InfoSec 3001 — Information Security from the Offensive Perspective

Step 182. Assembly 1: Registers, mov/push/pop/call/ret, Stack Frames — The Alphabet of the Common Language

Step 182. Assembly 1: Registers, mov/push/pop/call/ret, Stack Frames — The Alphabet of the Common Language

Level 3 — Real-World CTF and Advanced Attack Skills | Difficulty ★★★★☆ | Estimated time: 6 hours

Prerequisites: Step 64 (first meeting with assembly — C vs assembly side by side), Step 62 (stacks and buffers). You’ve extracted a .s file with gcc -S before.

⚠️ All exercises in this chapter are for your own lab and legal platforms only. Applying them to unauthorized systems is a crime.

  • What you need: WSL Ubuntu (gcc, gdb). The measured environment is Ubuntu 24.04, gcc 13.3.0, gdb 15.1, x86-64.
  • Caution: today’s practice is 100% safe. You translate C you wrote into assembly, read it, and observe registers while it runs — nothing more.

In Step 64 we had our "first meeting" with assembly — you saw that an if is a cmp plus a jump. Starting today you learn it properly, as the common language of Pwn and Reversing. Good news: most instructions you’ll meet in the field number barely ten. And once you understand the stage those instructions move on — the registers and the stack frame — the entire "why" of the buffer overflow you experienced hands-on in Step 62 gets explained.

1. Learning Objectives

By the end of this chapter, you will be able to:

  • Know the names and roles of the x86-64 general-purpose registers (rax, rdi, rsp, rbp, etc.)
  • Explain mov, push/pop, call/ret, cmp+branches, add/sub, xor, and lea, one line each
  • Confirm the calling convention — arguments in rdi, rsi, rdx, rcx, r8, r9 in order; return value in rax — in assembly
  • Draw the stack frame that a function prologue (push rbp; mov rbp, rsp; sub rsp, N) builds
  • Look at 10 lines of sourceless assembly and infer "what does this function do"

2. Background Knowledge — Today’s Tools and Concepts

Today’s Tools at a Glance

Category Details
Language/environment Assembly (x86-64, Intel syntax) + C + WSL bash (measured: gcc 13.3.0, gdb 15.1)
Today’s commands gcc -S -masm=intel -O0 file.c, gcc -g -O0 file.c -o file, gdb b *function, ni, info registers
Concepts needed 16 general-purpose registers, 10 instruction types, calling convention, stack frames, Intel vs AT&T syntax
Today’s artifact 10 instruction cards + a stack frame drawing + assembly commentary notes

2-1. The Register Map — Sixteen Palm-Sized Boxes

In Step 64 you met registers as "the CPU’s palm-sized boxes." Today you see the full map. x86-64 has 16 general-purpose registers, with role agreements:

Register Agreed role
rax Return value, the main stage of computation
rdi rsi rdx rcx r8 r9 Function arguments — in order from the first
rsp Stack-top pointer (moved by push/pop)
rbp Current function’s reference point (the stack frame’s anchor)
rbx, r10–r15 Temporary work

Names starting with e, like eax and edi, are the lower 32 bits of the same box. Sixty-four bits is overkill for handling one int, so you use just the lower part — that’s why both names appear mixed in assembly.

2-2. Instruction Cards — Ten and You Can Start Reading

Instruction Meaning One-line example
mov a, b Copy b into a mov eax, 3 — put 3 in eax
lea a, [b] Put b’s address (location, not value) in a when making addresses of strings/arrays
push x Push x onto the stack (rsp decreases by 8) push rbp
pop x Pop the stack top into x (rsp increases by 8) pop rbp
call f Push the return address onto the stack and jump to f call mix
ret Pop and jump to the address at the stack top end of a function
cmp a, b Compute a−b, keep only the result marks (flags) cmp eax, 59
je/jne/jmp etc. Jump based on flags / jump unconditionally jle .L2
add/sub Addition / subtraction sub rsp, 16
xor a, a XOR with itself = the idiom for zeroing out xor eax, eax

call and ret are today’s protagonists. They’re not mere jumps but round-trip tickets using the stack — on the way out, the return address gets tucked onto the stack; on the way back, it gets pulled out and used. This agreement is the stack frame’s first floor.

2-3. The Stack Frame — One Function’s Desk

Every time a function gets called, a workspace of its own gets laid out on the stack. This is the stack frame. Standard layout:

high address ─┬─ the previous function's frame
              ├─ return address        ← what call pushed
              ├─ saved old rbp         ← push rbp
current rbp → ┤
              ├─ local variables       ← secured by sub rsp, N
low address   └─ (rsp, the top)

Three lines of prologue — push rbp (save the old reference point), mov rbp, rsp (declare my reference point), sub rsp, N (widen the desk) — build this structure. The reason a buffer overflow in Step 62 overwrote "the address to return to" is in this picture: the return address sits on the floor right above the local variables (the buffer).

2-4. Intel vs AT&T — Two Notations, Opposite Directions

A review and a settling of what you met in Step 64. Intel syntax mov eax, 3 means "put 3 into eax"; AT&T syntax movl $3, %eax has the order reversed. We’ll unify on -masm=intel again today, but field material (especially gdb’s default output) is often AT&T. When you see % and $, switch on the "reversed order" warning light — today’s practice compares the two side by side.

3. Follow Along

3-1. Test Code — A Three-Argument Function

Let’s build a function with three arguments and observe the calling convention and stack frame at once.

Input (frame.c)

int mix(int a, int b, int c) {
    int t = a + b;
    int u = t * c;
    return u;
}

int main(void) {
    int r = mix(3, 4, 5);
    return r;
}

Compile — extract in both syntaxes

gcc -S -masm=intel -O0 frame.c -o frame_intel.s
gcc -S -O0 frame.c -o frame_att.s

3-2. Reading in Intel Syntax — Dissecting mix

grep -v -e "^\." -e cfi frame_intel.s

mix:
	endbr64
	push	rbp
	mov	rbp, rsp
	mov	DWORD PTR -20[rbp], edi
	mov	DWORD PTR -24[rbp], esi
	mov	DWORD PTR -28[rbp], edx
	mov	edx, DWORD PTR -20[rbp]
	mov	eax, DWORD PTR -24[rbp]
	add	eax, edx
	mov	DWORD PTR -8[rbp], eax
	mov	eax, DWORD PTR -8[rbp]
	imul	eax, DWORD PTR -28[rbp]
	mov	DWORD PTR -4[rbp], eax
	mov	eax, DWORD PTR -4[rbp]
	pop	rbp
	ret

(Measured 2026-09-09. Assembler directive lines like .file are omitted.)

Line-by-line commentary:

  1. push rbp; mov rbp, rsp — the prologue. Save the old reference point on the stack and declare mine.
  2. mov -20[rbp], edi and two more lines — set the arguments a, b, c, which arrived in registers, down on my desk on the stack. rdi→a, rsi→b, rdx→c. The calling convention in the flesh.
  3. Two mov lines + add eax, edx — t = a + b. The result goes to eax, then gets stored at -8[rbp] (variable t).
  4. imul eax, -28[rbp] — u = t × c. The result goes to -4[rbp] (variable u).
  5. mov eax, -4[rbp] — load the return value into rax (eax). The "results go back via rax" agreement.
  6. pop rbp; ret — restore the old reference point, return home to the return address at the stack top.

Prediction: why use addresses below rbp, like -20[rbp] and -24[rbp]? Because in the 2-3 picture, local variables live on the floor below rbp. Write the five slots a, b, c, t, u into the picture yourself.

3-3. The Same Function in AT&T Syntax — Reading in Reverse

sed -n "/^mix:/,/ret/p" frame_att.s | grep -v cfi

mix:
	endbr64
	pushq	%rbp
	movq	%rsp, %rbp
	movl	%edi, -20(%rbp)
	movl	%esi, -24(%rbp)
	movl	%edx, -28(%rbp)
	movl	-20(%rbp), %edx
	movl	-24(%rbp), %eax
	addl	%edx, %eax
	...
	popq	%rbp
	ret

(Measured 2026-09-09.)

How to read the output: same function, different look. movl %edi, -20(%rbp) means "put edi into -20(%rbp)" — departure on the left, arrival on the right. Exactly opposite to Intel. The register %, the constant $, and the length suffix on instructions (l=32-bit, q=64-bit) are AT&T’s signals. Since gdb’s default output is this syntax, starting tomorrow in Step 183 you learn to switch on the converter.

3-4. The Calling Side — Keeping the Agreement in main

main:
	endbr64
	push	rbp
	mov	rbp, rsp
	sub	rsp, 16
	mov	edx, 5
	mov	esi, 4
	mov	edi, 3
	call	mix
	mov	DWORD PTR -4[rbp], eax
	mov	eax, DWORD PTR -4[rbp]
	leave
	ret

(Measured 2026-09-09, the main portion of the same frame_intel.s.)

How to read the output: the three lines right before call mix — the three arguments get loaded into edx (third), esi (second), edi (first), looking reversed (the order is the compiler’s discretion; the convention enforces only the register assignment). Right after call, the result gets pulled from eax the moment it returns. And main’s prologue has sub rsp, 16 — widening the desk for variable r. leave is "copy rbp into rsp, then pop rbp" — the exact reverse of the prologue, i.e., folding the desk.

3-5. Watching a Stack Frame Come Into Being with gdb

This time, instead of a picture, let’s look at the real thing mid-execution. Compile with debug info and stop at mix’s entry point:

gcc -g -O0 frame.c -o frame
gdb -batch -x watch.gdb ./frame

Input (watch.gdb — command collection file)

set disassembly-flavor intel
b *mix
run
info registers rsp rbp
x/1gx $rsp
ni
ni
info registers rsp rbp
x/2gx $rsp
ni
info registers rsp rbp

Output (excerpt, measured 2026-09-09 — addresses differ per run)

Breakpoint 1, mix (a=0, b=0, c=0) at frame.c:1
=== right after entry (before push rbp) ===
rsp            0x7fffffffe668      0x7fffffffe668
rbp            0x7fffffffe680      0x7fffffffe680
0x7fffffffe668:	0x0000555555555174      ← stack top = return address

=== one line past push rbp ===
rsp            0x7fffffffe660      ← decreased by 8
rbp            0x7fffffffe680
0x7fffffffe660:	0x00007fffffffe680	0x0000555555555174
                 ↑ old rbp saved          ↑ return address above it

=== one line past mov rbp, rsp ===
rsp            0x7fffffffe660
rbp            0x7fffffffe660      ← rbp = rsp: my reference point declared

How to read the output: b *mix is a breakpoint on the function’s first machine instruction. ① Right after entry, the return address is already at the stack top — what call pushed (0x…5174 is the address of the instruction after call in main). ② One line of push rbp decreased rsp by 8 and saved the old rbp (0x…e680) at the new top. ③ After mov rbp, rsp, rbp=rsp — from here the reference point for notations like -20[rbp] is fixed. The 2-3 picture is now confirmed in numbers.

3-6. Inferring Ten Sourceless Lines

Finally, let’s read some assembly with its origin hidden:

func:
	push	rbp
	mov	rbp, rsp
	mov	DWORD PTR -4[rbp], edi
	mov	eax, DWORD PTR -4[rbp]
	add	eax, DWORD PTR -4[rbp]
	pop	rbp
	ret

The inference process: takes one argument (edi) → loads that value into eax → adds the same value once more → leaves it in eax and rets. This function is "a function that doubles its input." It could have been translated as imul eax, 2, but the -O0 compiler chose add. This kind of inference — register tracking + operation reading — is the opening move of every Reversing challenge.

4. Missions & Exercises

Mission — 10 Instruction Cards + a Stack Frame Drawing

  1. Make cards for the 10 instructions in 2-2 — front: instruction, back: one-line meaning + one real usage example from today’s practice
  2. Using the measured values from 3-5, draw mix’s stack frame — mark the return address, the old rbp, the slots for locals a·b·c·t·u, and each address (-20[rbp], etc.)
  3. In the 3-6 style, extract a simple function you write yourself (average of three numbers, etc.) with gcc -S, hide the source, and practice inferring "what function is this" from the assembly alone

Exercises

Exercise 1. Explain how the call instruction differs from an ordinary jump (jmp), using "stack" as your keyword.

Exercise 2. State the two things that happen in a single line of push rbp (the change to rsp, the change to memory).

Exercise 3. When calling a five-argument function int f(a,b,c,d,e), write the registers each argument rides in, in order.

Exercise 4. What idiom is xor eax, eax, and guess why it’s used so often instead of mov eax, 0.

5. Model Answers & Completion Criteria

Mission Model Answer

An example of the stack frame drawing (using the 3-5 measured addresses):

0x...e680 ── old rbp (the caller's reference point)  ← actually the pre-call frame
0x...e668 ── return address 0x...5174        ← pushed by call
0x...e660 ── saved old rbp = 0x...e680       ← push rbp     ┐ mix's frame
              (rbp = rsp = 0x...e660)                       │  reference point fixed
 -4[rbp] ── u                            ← imul's result    │
 -8[rbp] ── t                            ← add's result     │
-20[rbp] ── a (from edi)  -24[rbp] ── b  -28[rbp] ── c      ┘

An example card back: "call — pushes the return address onto the stack and jumps. Example: call mix in main → 0x…5174 goes onto the stack (3-5 measurement)."

How to verify: ① do all ten cards carry an "example from today’s practice" (cards that only copy dictionary definitions are incomplete)? ② in the drawing, is the return address above (higher address than) the old rbp? ③ in the inference exercise, did you write down your prediction before checking the answer?

Exercise Answers

Answer 1. jmp only goes, leaving no way back. call pushes the next instruction’s address (the return address) onto the stack before jumping. That’s why the called function can pop that address with ret and come back exactly. In 3-5, the return address already sitting at the stack top right after entry is the evidence.

Answer 2. ① rsp decreases by 8 (the stack grows toward lower addresses). ② The old rbp value gets stored at the new rsp location. In the measurement, it shrank from 0x…e668 to 0x…e660, and 0x…e680 was recorded at 0x…e660.

Answer 3. rdi, rsi, rdx, rcx, r8, r9, in that order. (From the seventh argument on, they go onto the stack — that’s a story for the advanced stage.)

Answer 4. It’s the idiom for "zeroing out a register." XORing with itself always yields 0. The machine code is shorter (fewer bytes) and often faster than mov eax, 0, so compilers favor it. When you meet it while reading assembly, interpret it as "putting in 0."

Completion Criteria Checklist

  • [ ] Among the 16 general-purpose registers, I know the roles of rax, rdi–r9, rsp, rbp
  • [ ] I can explain the 10 instruction types (mov, lea, push, pop, call, ret, cmp, jcc, add/sub, xor)
  • [ ] I can state the six argument registers in order and the return-value-in-rax convention
  • [ ] I can explain the direction difference between Intel and AT&T syntax
  • [ ] I can draw how the three prologue lines build a stack frame
  • [ ] I verified in gdb the rsp change and stack contents before/after push rbp
  • [ ] I’ve tried inferring 10 lines of sourceless assembly
  • [ ] Mission: I completed the 10 instruction cards and the stack frame drawing

6. Common Pitfalls & Fixes

Wall 1. Reading AT&T output in Intel order and understanding it backwards

Symptom: you read movl %edi, -20(%rbp) as "put the memory value into edi."
Cause: AT&T is departure→arrival order, the reverse of Intel. Compare the two syntaxes of the same function in 3-3.
Fix: if you see % and $, it’s AT&T. In gdb you can switch to Intel syntax with set disassembly-flavor intel — that’s the first line of watch.gdb in 3-5.

Wall 2. [rbp-4] and -4[rbp] look different and confuse me

Symptom: gcc -S output and objdump/gdb output use different notations.
Cause: a notation difference only — the meaning is the same: "4 bytes below rbp."
Fix: they differ only in bracket-inside-vs-outside and decimal-vs-hex. Just read "how far from whose reference point."

Wall 3. Not knowing the difference between b *mix and b mix

Symptom: you set b mix and it stops at a point where the prologue has already finished.
Cause: b mix breaks on the function’s first line of source, with debug info placing you past the prologue. The asterisk (*) breaks on the function’s first machine instruction address.
Fix: when observing the prologue itself, use b *function-name — today’s 3-5 is that case. When you only want to see variables, b function-name is more convenient.

Wall 4. Panicking because my addresses differ from the book’s

Symptom: addresses like 3-5’s 0x7fffffffe660 differ in your environment.
Cause: the OS assigns stack addresses on every run (ASLR). Differing is normal.
Fix: look not at absolute values but at relationships — did rsp shrink by 8? did rbp=rsp happen? is the return address at the stack top? Relationships are always the same.

Wall 5. Wearing yourself out trying to memorize every prologue and directive line

Symptom: you dig into lines like endbr64 and .cfi and your progress stops.
Cause: today’s goal is reading the skeleton, not a full translation.
Fix: endbr64 is a branch-protection marker, .cfi is a debugging guidance line — both can be skipped for now. The reading order is four phrases: "arguments coming down (mov …edi) → operations (add/imul) → loading the result (eax) → pop/ret."

7. Summary

Today’s Concepts

Concept One-line explanation
General-purpose registers 16 boxes inside the CPU — with role agreements (rax=return, rdi~=arguments)
Calling convention Arguments: rdi→rsi→rdx→rcx→r8→r9; return value: rax
Stack frame One function’s desk — return address + old rbp + local variables
Prologue push rbp; mov rbp,rsp; sub rsp,N — setting up the desk
call/ret Round-trip tickets pushing/popping the return address on the stack
xor idiom xor eax, eax = zeroing out
Intel vs AT&T Two notations with arrival/departure order reversed

Today’s Commands

Command What it does
gcc -S -masm=intel -O0 file.c Extract Intel-syntax assembly
gcc -S -O0 file.c Extract AT&T-syntax (default) assembly
gcc -g -O0 file.c -o file Compile with debug info
gdb b *function Break on a function’s first machine instruction
gdb ni Execute one machine instruction
gdb info registers rsp rbp See register values
gdb x/2gx $rsp See two 8-byte values at the stack top

An Instinct More Important Than Commands

What today’s measurements confirmed comes down to one thing — that the abstract concept of a "function call" is a physical procedure of stacking addresses and moving reference points on the stack. Eyes that have seen one line of push shrink rsp by 8 and record the old value will, from now on, instantly understand a buffer overflow’s attack principle as "which slot of the picture gets overwritten."

Don’t throw away the stack frame drawing you made today. When you handle tools that freeze a running program and peer onto this stage, that drawing becomes your map.

Once every box is checked, Step 182 is complete. Click the checkbox in the sidebar to save your progress.

← 이전 스텝
다음 스텝 →