Step 67. The Role of the Operating System — Every Request Goes to the Kernel
Level 1 — Programming and the Computer’s Interior | Difficulty ★★★☆☆ | Estimated time: 3 hours
Prerequisites: Steps 56–66 complete; you can compile a C program and know the structure of executables and the memory map.
- What you need: a Linux terminal (WSL or Ubuntu) and gcc. Having the reference tool strace installed is even better, but we’ve prepared so you can verify all of today’s experiments another way without it.
- Caution: today’s practice is 100% safe. We do make one deliberately broken program (badboy), but the only thing that breaks is that program itself.
The programs we made read files, printed characters to the screen, and borrowed memory. But think about it. How does it read the disk? How does it draw on the screen? Nowhere in our code was there a command like "move the disk." The answer is that our program doesn’t do that work itself. It "requested" all of it from the operating system — precisely, from its heart, the kernel. Today we directly call the channel of those requests, the system call, and even measure the cost of a request.
1. Learning Objectives
By the end of this chapter, you will be able to:
- Explain what the kernel is and why programs can’t touch hardware directly
- State the distinction between user mode and kernel mode
- Explain what a system call is with examples, and call one yourself
- Draw the three-stage flow "library function → system call → kernel"
- Read strace output and trace a program’s requests
2. Background Knowledge — Today’s Tools and Concepts
Today’s Tools at a Glance
| Category | Details |
|---|---|
| Language/environment | C language + Linux terminal (verified on WSL Ubuntu 24.04) |
| Today’s commands/tools | strace (system call tracing — on environments without it, learn from the screen examples), grep to view the system call number table, /proc/self/fd to check file number tags, time to measure the cost of requests |
| Today’s C functions | syscall(), write(), printf(), fopen()/fgets(), fileno() |
| Concepts needed | Kernel, user mode/kernel mode, system calls, file descriptors (number tags), segmentation faults |
| Today’s deliverable | A system call trace log — a "function → system call" comparison table for three programs |
2-1. The Kernel — The Heart of the Operating System
The core part of an operating system is called the kernel. It’s the computer’s general manager: it hands out memory, reads files, handles the network, and decides the order among programs. Only this manager may touch hardware directly.
2-2. User Mode and Kernel Mode — Two Statuses
The CPU has two status modes. Our programs run in user mode (the lower status), and the kernel runs in kernel mode (the higher status). Programs in user mode are limited in what they can do — commands that touch hardware directly or peek into someone else’s memory are forbidden from executing. This status distinction isn’t entrusted to programs’ good intentions; the hardware called the CPU enforces it directly.
2-3. System Calls — How to Cross the Threshold
So what if a user-mode program wants to read a file? There’s a formal procedure for crossing the threshold: the system call. When you request in the prescribed way, "Dear kernel, please read this file," the CPU briefly switches to kernel mode, the kernel does the work, and it returns to user mode carrying the result. Every external activity — opening files (openat), reading (read), writing (write), borrowing memory (brk, mmap) — passes through this channel.
2-4. Why the Threshold Exists — The Oldest Fence
If this threshold didn’t exist, any program could do as it pleased with hardware and look at others’ memory. One program’s bug or malice would wreck the whole computer. Thanks to the threshold, even malware must make "requests," and the kernel can inspect those requests. And someone who knows this structure comes to ask the next question: "Then how does a privilege escalation attack get over this threshold?" Good question. That’s a later study topic; today, confirming the threshold’s existence is enough.
3. Follow Along
3-1. Peeking at the System Call Number Table
Requests to the kernel are numbered. Let’s open the number-table file built into Linux ourselves.
Input
grep -E "define __NR_(write|read|openat|brk|mmap|exit_group) " /usr/include/x86_64-linux-gnu/asm/unistd_64.h
Output (verified 2026-09-09):
#define __NR_read 0
#define __NR_write 1
#define __NR_mmap 9
#define __NR_brk 12
#define __NR_exit_group 231
#define __NR_openat 257
How to read it: __NR_write 1 means the number of the "please write" request is 1. Requests reach the kernel by number, not by name. This table is the kernel’s list of service counters.
Why: the first step in confirming that a system call isn’t an abstract concept but an actual counter with a number.
3-2. Requesting the Kernel Directly — Three Ways
Now let’s knock on that counter ourselves. We’ll call the same "please write to the screen" from three different layers.
Input (direct.c)
#include <stdio.h>
#include <unistd.h>
#include <sys/syscall.h>
int main(void) {
syscall(SYS_write, 1, "direct syscall: kernel, please write\n", 37);
write(1, "via write(): kernel, please write\n", 34);
printf("via printf: kernel, (secretly) please write\n");
return 0;
}
Input
gcc -g -O0 direct.c -o direct && ./direct
Output (verified 2026-09-09):
direct syscall: kernel, please write
via write(): kernel, please write
via printf: kernel, (secretly) please write
How to read it: all three lines appeared on screen. syscall(SYS_write, ...) directly knocked on the counter (write) we found in 3-1’s number table. write() is a thin function wrapping that counter. printf() is a thick wrapper that even handles formatting and output batching, but in the end it calls write inside. The first argument, 1, means "output counter number 1 (the screen)" — we’ll confirm this number tag’s identity in the next experiment.
Why: touching the three-stage flow "library function → system call → kernel" directly in code is today’s core. You’ve seen printf’s true face.
3-3. File Number Tags — 0, 1, 2, and 3
Input (reader.c)
#include <stdio.h>
int main(void) {
FILE *f = fopen("test.txt", "r");
char buf[100];
fgets(buf, 100, f);
printf("read: %s", buf);
printf("file descriptor (fd): %d\n", fileno(f));
fclose(f);
return 0;
}
Setup and run
echo "hello" > test.txt
gcc -g -O0 reader.c -o reader
./reader
ls -l /proc/self/fd/
Output (verified 2026-09-09):
read: hello
file descriptor (fd): 3
lr-x------ 1 root root 64 ... 0 -> pipe:[5984]
l-wx------ 1 root root 64 ... 1 -> pipe:[5985]
l-wx------ 1 root root 64 ... 2 -> pipe:[5986]
lr-x------ 1 root root 64 ... 3 -> /proc/.../fd
How to read it: the file opened with fopen received number tag 3. Open /proc/self/fd (the number-tag box of the currently running process) and the reason is visible — 0 (standard input), 1 (standard output), and 2 (error output) are numbers assigned to every process from birth, and the file we open gets the next number, starting from 3. These numbers are called file descriptors (file number tags).
Why: the step where the structure becomes visible — "when requesting the kernel, you pass together what to do (the system call number) and where to do it (the file number tag)." The 1 in write(1, …) pointing to "the screen" is also because of this convention.
3-4. strace — Peeking at the Work Log of Requests (Screen example)
The tool that records and shows every request is strace. Note: strace is not installed in this book’s verified lab (WSL), and per this book’s rule of not installing things during practice, the output below is a "screen example," not a live capture. If your environment has the strace command (or one where you can install it with sudo apt install strace), follow along yourself.
Input (example)
strace -e trace=write ./hello
Screen example (how it looks in a typical Ubuntu environment):
write(1, "Hello, C!\n", 10) = 10
+++ exited with 0 +++
How to read it: strace shows, in order, the requests a program made to the kernel. -e trace=write is a filter meaning "show only write requests." write(1, "Hello, C!\n", 10) = 10 is a request and answer: "please write ten characters to number 1 (the screen) → wrote ten characters." It wasn’t printf printing characters to the screen. printf prepares, and the kernel prints via write. If you run strace ./hello without options, dozens of lines appear including startup-preparation requests — in that case, read backwards from the end; our code’s requests are usually near the back.
Why: even without strace, we proved the same fact in code in 3-2 (printf and syscall reached the same screen). strace is merely a tool for "observing from outside" that three-stage flow; the principle is already in your hands.
3-5. Crossing the Threshold — How Forbidden Acts Get Blocked
What happens if you stab someone else’s land directly with a pointer, instead of making a request (system call)?
Input (badboy.c)
#include <stdio.h>
int main(void) {
int *p = (int *)0x1;
*p = 42;
printf("will this line show?\n");
return 0;
}
Compile and run
gcc badboy.c -o badboy
./badboy; echo "exit code: $?"
Output (verified 2026-09-09):
Segmentation fault (core dumped)
exit code: 139
How to read it: trying to write to address 1 (land that isn’t mine) got it immediately thrown out. "will this line show?" was never printed — with no chance even to call a system call, the memory management unit blocked it, saying "that’s not your land in user mode." Exit code 139 is 128 + 11, meaning it died by signal 11 (SIGSEGV). The threshold has two layers: the request channel (system calls) and direct blocking (this kind of memory protection).
Why: when a program makes you ask "why did it die?", the answer is almost always in the last line and the exit code. The habit of reading exit codes is the starting point of debugging.
3-6. Requests Have a Cost — Measuring with Time
Crossing the threshold isn’t free. Switching from user mode to kernel mode and back takes time. Let’s measure how much. We’ll write the same million characters two ways: once with a million system calls, and once by batching in a buffer and writing in chunks.
Input (cost.c — a system call per byte)
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
int main(void) {
int fd = open("/dev/null", O_WRONLY);
char c = 'x';
for (int i = 0; i < 1000000; i++) write(fd, &c, 1);
close(fd);
return 0;
}
Input (cost2.c — batching in a buffer)
#include <stdio.h>
int main(void) {
FILE *f = fopen("/dev/null", "w");
for (int i = 0; i < 1000000; i++) fputc('x', f);
fclose(f);
return 0;
}
Compile and time
gcc -O2 cost.c -o cost && gcc -O2 cost2.c -o cost2
time ./cost
time ./cost2
Output (verified 2026-09-09):
== write (syscall) 1 byte at a time, 1,000,000 calls ==
real 0m0.071s
user 0m0.029s
sys 0m0.042s
== batched writes (fputc), 1,000,000 calls ==
real 0m0.002s
user 0m0.002s
sys 0m0.000s
How to read it: the same million characters, but 0.071 seconds vs. 0.002 seconds — about a 35x difference. Look at the sys column especially — the write version spent 0.042 seconds in kernel mode (sys = time the kernel spent working), while the buffer version’s sys is 0. That’s because fputc gathers output in a temporary memory store (buffer) and writes it all at once at fclose. This cost is exactly why printf secretly batches output.
Why: the threshold’s existence explains the world of performance, not just security. You’ve confirmed with your own body why "how many system calls does it make?" is a fundamental question of performance tuning.
4. Missions & Exercises
Mission — A System Call Trace Log
- Make a table for the three programs hello, reader, and cost (the ones made today). Columns: program, the function we used, the actual system call, what it does. (Fill it in based on today’s number table, code, and run results)
- If you’re in an environment with strace, run
strace -e trace=openat,read,write ./readerand verify whether the predictions in your table were right. If not, note that fact in the log and compare with the screen example - In the reader run, mark what file number tags 0, 1, 2, and 3 were each used for
- Draw "the journey of printf("hello") until it appears on screen" as a library → system call → kernel diagram
- Answer in your notes: "Why doesn’t the operating system let programs touch hardware directly — three reasons."
Exercises
Q1. Explain the relationship between printf and write using the distinction "wrapper and request."
Q2. ./badboy died with Segmentation fault and left exit code 139. How do you interpret 139, and why was the printf line never printed?
Q3. Explain why a file opened with fopen receives number tag 3, together with the identities of 0, 1, and 2.
Q4. Writing the same million characters took 0.071 seconds with 1-byte writes and 0.002 seconds with fputc (verified 2026-09-09). What made this difference, and which column of the time output is the evidence?
5. Model Answers & Completion Criteria
Mission Model Answer
An example trace-log table (based on the 2026-09-09 verification):
| Program | Function we used | Actual system call | What it does |
|---|---|---|---|
| hello | printf | write (1) | Write characters to the screen (number tag 1) |
| reader | fopen | openat (257) | Open test.txt and receive number tag 3 |
| reader | fgets | read (0) | Read from number tag 3 |
| reader | printf | write (1) | Write the read contents to the screen |
| cost | write | write (1) | Write to /dev/null one byte at a time, a million times |
File number tags summarized: 0 = standard input, 1 = standard output (screen), 2 = error output — all assigned at birth. 3 = the test.txt that reader opened with fopen (fileno confirmed as 3 in the 2026-09-09 verification).
Diagram: printf("hello") → the C library builds the format and batches it in a buffer → the write(1, "hello...", ...) system call → the CPU switches to kernel mode → the kernel handles screen output → returns to user mode with the result.
Three example reasons: ① Safety — so one program’s bug can’t wreck the whole. ② Isolation — so it can’t freely peek at others’ memory and files. ③ Inspection — so the kernel can inspect and record every external activity (monitoring tools like antivirus and EDR watch exactly this channel).
How to verify: ① Do the system call numbers in the table match the numbers in the number-table file (unistd_64.h)? ② Does the diagram include "kernel mode switch"? ③ If you used strace, did the write line match the table’s prediction? If all three are "yes," it’s complete.
Exercise Solutions
Q1 solution. printf is the wrapper that "prepares" the request (format handling, output batching); write is the request "itself" delivered to the kernel. In the 3-2 experiment, the direct syscall(SYS_write) call and printf reached the same screen — evidence that both ultimately head to the same counter.
Q2 solution. 139 = 128 + 11, meaning it exited by signal 11 (SIGSEGV, invalid memory access). The printf line wasn’t printed because the memory management unit blocked immediately at *p = 42;, before reaching that line — it never even had a chance to make a request (system call) (verified 2026-09-09).
Q3 solution. Because 0, 1, and 2 are numbers automatically assigned when every process starts — standard input, standard output, and error output respectively. So a file a process newly opens gets the next free number, starting from 3. We verified by live capture that 0, 1, and 2 are already occupied in /proc/self/fd.
Q4 solution. The threshold-crossing cost of a million system calls made the difference. fputc batches in a memory buffer and writes in chunks at fclose, so it barely crosses the threshold. The evidence is the sys column of the time output — the write version has sys 0.042s (time the kernel spent working), the fputc version has sys 0.000s (verified 2026-09-09).
Completion Criteria Checklist
- [ ] I can explain the kernel and the user mode/kernel mode distinction
- [ ] I can explain what a system call is, together with the number table
- [ ] I requested the kernel directly with syscall(SYS_write, …)
- [ ] I can explain the meaning of file descriptors 0, 1, 2, and 3
- [ ] I can draw the library function → system call → kernel flow
- [ ] I confirmed with the time experiment that system calls have a cost
- [ ] Mission: I completed the system call trace log
6. Common Pitfalls & Fixes
Wall 1. Panic at the flood of strace output
Symptom (screen example): you run strace without options and dozens or hundreds of lines pour out.
execve("./hello", ["./hello"], ...) = 0
brk(NULL) = 0x55...
... (dozens of lines) ...
write(1, "Hello, C!\n", 10) = 10
exit_group(0) = ?
+++ exited with 0 +++
Cause: from the moment it starts, a program makes countless requests (library preparation, memory setup, etc.).
Fix: at first, accept it and skip past. Filtering to requests of interest like -e trace=write is the standard, and look from the "end" of the output. Our code’s requests are usually near the back.
Wall 2. Non-ASCII text looks like \354\225\210
Symptom (screen example): in strace output, non-ASCII text (e.g., Korean) appears as a number sequence like read(3, "\354\225\210\353\205\225...", 4096) = 15.
Cause: strace displays bytes in octal. The shape is unfamiliar to our eyes, but the content is fine.
Fix: remember Step 50. Non-ASCII characters are UTF-8 bytes. It’s not "broken" — it’s "seen as bytes." Focus on reading the = 15 (bytes read) after the string.
Wall 3. Thinking system calls and library functions are the same thing
Symptom: you think printf = system call.
Cause: confusing the wrapper with the contents.
Fix: remember the three-stage flow. printf (library) → write (system call) → kernel. In 3-2, the direct syscall call and printf producing the same result is evidence they’re different layers. They arrive at the same counter, but the thickness of the road differs.
Wall 4. Losing sight of why you’re learning this
Symptom: the commands work, but you think "so what does this have to do with security?"
Cause: you just haven’t drawn the connecting line yet.
Fix: connect it like this. All external activity passes through the kernel → the kernel is a checkpoint → malware’s behavior also shows up in this channel → so defensive tools (antivirus, EDR) monitor system calls. And privilege escalation attacks are techniques for fooling this checkpoint’s identity check. Today’s experiments are training in reading that checkpoint’s work rules.
Wall 5. The cost experiment’s times differ from expectation
Symptom: your numbers differ from the book’s 0.071 seconds (larger or smaller).
Cause: time measurements vary each run with CPU performance and the system load of the moment. The book’s figures are a single 2026-09-09 record.
Fix: look not at absolute values but at "the ratio of the two versions" and "the contrast in the sys column." Run it several times and the ratio mostly holds.
7. Summary
Today’s Concepts
| Concept | One-line description |
|---|---|
| Kernel | The only manager allowed to touch hardware directly |
| User mode / kernel mode | Two statuses enforced by the CPU — our programs are on the lower side |
| System call | The formal channel by which user mode requests work from the kernel (called by number) |
| File descriptor | A number tag pointing to a request’s target — 0 input, 1 output, 2 error, our files from 3 |
| Segmentation fault | Immediate blocking of an unauthorized memory access (exit code 139 = 128+11) |
| Buffering | The technique of batching output and writing in chunks to save system call costs |
Today’s Commands/Functions
| Command/function | What it does |
|---|---|
grep ... unistd_64.h |
View the system call number table |
syscall(SYS_write, 1, ...) |
Knock directly on a system call counter |
strace -e trace=name program |
Trace a program’s requests (optional tool) |
ls -l /proc/self/fd/ |
View the number-tag box of the currently running process |
time ./program |
Measure execution time — the sys column is time the kernel spent working |
echo $? |
Check the previous command’s exit code (139 = SIGSEGV) |
The Instinct That Matters More Than Commands
Remember that the threshold has two layers. The first layer is system calls — to touch resources like files, networks, and processes, you must pass the kernel’s threshold, and the very act of requesting gets inspected. The second layer is memory protection — even when you stab directly with a pointer without making any request, like badboy.c, you get blocked, and here what’s checked is not the "act" but "the address your hand reached for." The further you go in security study, the more these two layers split into concrete technology names like seccomp, SELinux, and DEP — all descendants of what you saw today: "requests are inspected / only permitted addresses."
The security connection: defenders monitor this channel — "this program suddenly opened a strange file and is writing to a strange place." Much of antivirus and EDR is this monitoring technology, and attackers, conversely, try to bypass the channel or target weaknesses of the checkpoint (the kernel) itself. Today you saw the terrain of that battlefield. Experiments stay on your own programs in your own VM. Finally, remember Linux’s philosophy "everything is a file" — screen and keyboard alike are opened, read, and written like files. The number tags 0, 1, and 2 you saw today are the entrance to that philosophy.
Once every box is checked, Step 67 is complete. Click the checkbox in the sidebar to save your progress.