InfoSec 3001 — Information Security from the Offensive Perspective

Step 233. AES Structure and Mode-Specific Vulnerabilities — Anatomy of the World Standard and a Map of Failures

Step 233. AES Structure and Mode-Specific Vulnerabilities — Anatomy of the World Standard and a Map of Failures

Level 3 — Real-World CTF & Advanced Offensive Skills | Difficulty ★★★☆☆ | Estimated time: 4 hours

Prerequisites: Steps 231~232 (Cryptopals Sets 1~2) — today we open up the internals of the very AES you spent two days breaking.

⚠️ All exercises in this chapter are for your own lab and legal platforms only. Applying them to unauthorized systems is a crime.

  • What you need: Python 3 (measured: 3.12.14), pycryptodome.
  • Caution: today is a concept chapter — the goal is not implementing AES yourself but "reading its structure and mapping where each mode fails." Every experiment output is measured.

AES is the US standard (NIST FIPS 197) and the de facto single worldwide standard for symmetric encryption — TLS, disk encryption, Wi-Fi all sit on top of AES. And yet, over the last two days, we broke this "world standard" again and again. That’s no contradiction — what broke was never the AES algorithm but how the modes were used. Today you’ll understand AES’s internal structure (SPN, rounds) and organize exactly where and how the four modes — ECB/CBC/CTR/GCM — each collapse.

1. Learning Objectives

By the end of this chapter, you will be able to:

  • State AES’s block and key sizes and the round counts (10/12/14) they correspond to
  • Explain the role of the SPN structure’s four operations (SubBytes, ShiftRows, MixColumns, AddRoundKey)
  • Prove with an image experiment why ECB preserves patterns
  • Connect the structures of CBC and CTR to their signature vulnerabilities (oracles, keystream reuse)
  • Run the CTR nonce-reuse attack and explain what GCM (AEAD) adds

2. Background Knowledge — Today’s Tools and Concepts

Today’s Tools at a Glance

Category Details
Language/environment Python 3 (measured: 3.12.14) + pycryptodome
Today’s commands AES.new(key, AES.MODE_ECB/CBC/CTR/GCM), Crypto.Util.Counter
Concepts needed SPN (Substitution-Permutation Network), rounds, block-cipher modes, nonce, AEAD
Today’s deliverable An ECB pattern-leak image proof + a measured CTR reuse attack + a per-mode vulnerability table

2-1. AES at a Glance — Blocks, Keys, Rounds

AES is a symmetric block cipher that works on 128-bit (16-byte) blocks. Key sizes come in three flavors — 128/192/256 bits — with 10/12/14 rounds respectively. The 16 input bytes go into a 4×4 state matrix, and each round applies four operations to scramble them.

SubBytes   : per-byte S-box substitution  — nonlinearity (linear relations die going through the table)
ShiftRows  : shift the rows by offsets    — diffuses each row's bytes into other columns
MixColumns : mix each column by matrix multiplication — one byte's change spreads across the whole column
AddRoundKey: XOR with the round key       — the only point where the secret (the key) enters

The final round omits MixColumns. This structure is called an SPN (Substitution-Permutation Network) — repeat substitution (S-box) and permutation (row shifts, column mixing), and after 10 rounds a single input-bit change spreads across the entire output (the avalanche effect). The algorithm itself has seen no practical attack in over 20 years. Every vulnerability comes from the modes.

2-2. Modes — How to Use a Block Cipher on Long Data

AES encrypts only 16 bytes. The "splicing rules" for applying it to long data are the modes:

  • ECB: encrypts each block independently. Same block = same ciphertext. Never use it.
  • CBC: C[i] = Enc(P[i] ⊕ C[i-1]). Chaining hides patterns, but with no integrity it’s exposed to bit-flipping and padding oracles (proven in Step 232).
  • CTR: encrypts a counter to make a keystream and XORs it with the plaintext — effectively a stream cipher. No padding needed, parallelizable. But nonce reuse is fatal.
  • GCM: CTR + an authentication tag (GHASH). An AEAD (authenticated encryption) that also detects tampering. The modern standard choice.

2-3. nonce — a "Number Used Once"

The nonce in CTR/GCM is the keystream’s starting point. Encrypt two plaintexts with the same key + same nonce and the same keystream gets reused: C1 ⊕ C2 = P1 ⊕ P2 — the keystream cancels out and only the XOR of the plaintexts remains. Know one plaintext and the other is recovered as-is. We measure this today.

3. Follow Along

All output in this chapter was measured 2026-09-09 on Python 3.12.14 + pycryptodome.

3-1. The Fate of Identical Plaintext Blocks — ECB vs CBC

from Crypto.Cipher import AES
import secrets

key = secrets.token_bytes(16)
pt = b"same block here!" * 8          # a 16-byte block repeated 8 times
ct_ecb = AES.new(key, AES.MODE_ECB).encrypt(pt)
ct_cbc = AES.new(key, AES.MODE_CBC, iv=secrets.token_bytes(16)).encrypt(pt)
for name, ct in (("ECB", ct_ecb), ("CBC", ct_cbc)):
    blocks = [ct[i:i+16] for i in range(0, len(ct), 16)]
    print(f"{name}: 8 identical plaintext blocks -> {len(set(blocks))} unique ciphertext blocks")

ECB: 8 identical plaintext blocks -> 1 unique ciphertext blocks
CBC: 8 identical plaintext blocks -> 8 unique ciphertext blocks

How to read the output: same key, same plaintext. Under ECB, 8 blocks collapsed into 1 kind (every ECB attack from Steps 231~232 stands on this property), while CBC’s chaining split them all apart. These two lines are today compressed whole.

3-2. The ECB Penguin Experiment — Pattern Leakage Made Visible

A CTF classic. Encrypt an image with ECB and the outline stays perfectly visible. Instead of an original bitmap, we reproduce it with a 240×240 BMP drawn directly in code (solid background + a white square in the center).

import struct
W = H = 240
row_size = (W * 3 + 3) & ~3
pixels = bytearray()
for y in range(H):
    row = bytearray()
    for x in range(W):
        on = (60 <= x < 180) and (60 <= y < 180)      # the center square
        row += bytes([255]*3) if on else bytes([30]*3)
    row += b"x00" * (row_size - W * 3)
    pixels += row

bmp_header = b"BM" + struct.pack("<IHHI", 54 + len(pixels), 0, 0, 54)
bmp_header += struct.pack("<IiiHHIIiiII", 40, W, H, 1, 24, 0, len(pixels), 2835, 2835, 0, 0)

def encrypt_bmp(mode, pixels, key):
    pad = (16 - len(pixels) % 16) % 16
    body = bytes(pixels) + bytes([pad or 16]) * (pad or 16)
    if mode == "ECB":
        ct = AES.new(key, AES.MODE_ECB).encrypt(body)
    else:
        ct = AES.new(key, AES.MODE_CBC, iv=secrets.token_bytes(16)).encrypt(body)
    return bmp_header + ct[:len(pixels)]   # keep the 54-byte header so the image still opens

open("orig.bmp", "wb").write(bmp_header + bytes(pixels))
open("ecb.bmp", "wb").write(encrypt_bmp("ECB", pixels, key))
open("cbc.bmp", "wb").write(encrypt_bmp("CBC", pixels, key))

for name in ("orig", "ecb", "cbc"):
    body = open(f"{name}.bmp", "rb").read()[54:]
    blocks = [body[i:i+16] for i in range(0, len(body) - len(body) % 16, 16)]
    print(f"{name}.bmp: {len(set(blocks))} unique of {len(blocks)} blocks "
          f"(duplication {100*(1-len(set(blocks))/len(blocks)):.1f}%)")

orig.bmp: 4 unique of 10800 blocks (duplication 100.0%)
ecb.bmp: 4 unique of 10800 blocks (duplication 100.0%)
cbc.bmp: 10800 unique of 10800 blocks (duplication 0.0%)

How to read the output: with a solid background and a square, the original has only 4 kinds of blocks. The ECB ciphertext has exactly 4 kinds too — encrypted, yet the structure is untouched. Open the generated ecb.bmp in an image viewer. The square’s outline remains sharp on top of the ciphertext (rendering verified 2026-09-09), while the CBC version is pure noise. This is the visual proof of "encrypted, yet still readable."

3-3. CTR Mode — AES as a Stream Cipher

CTR doesn’t encrypt the plaintext. It encrypts counter values to make a keystream and XORs it with the plaintext:

keystream[i] = Enc(key, nonce + i)
C[i] = P[i] XOR keystream[i]

Decryption regenerates the same keystream and XORs again — encryption and decryption are the same operation. No padding needed either (it’s a stream, so byte-level), and blocks are independent so it parallelizes. The price is nonce management — the subject of the very next experiment.

3-4. The CTR Nonce-Reuse Attack

from Crypto.Util import Counter

ctr1 = Counter.new(128, initial_value=100)   # fixed nonce/counter
c1 = AES.new(key, AES.MODE_CTR, counter=ctr1).encrypt(b"Transfer 1000 USD to Alice")
ctr2 = Counter.new(128, initial_value=100)   # the same nonce reused (a mistake!)
c2 = AES.new(key, AES.MODE_CTR, counter=ctr2).encrypt(b"Transfer 9000 USD to Mallory")

x = bytes(a ^ b for a, b in zip(c1, c2))     # c1 ⊕ c2 = m1 ⊕ m2
m1 = b"Transfer 1000 USD to Alice"            # one plaintext the attacker knows
m2_rec = bytes(a ^ b for a, b in zip(x, m1))
print("CTR reuse: c1^c2^m1 =", m2_rec)

CTR reuse: c1^c2^m1 = b'Transfer 9000 USD to Mallo'

How to read the output: the same keystream covered both, so it cancels in c1 ⊕ c2. XOR back the known plaintext m1 and the second plaintext is recovered (the last character is outside zip‘s range because m2 is one byte longer than m1 — measured as-is). If "1000" and "9000" are transfer instructions differing by a single digit, an attacker can go beyond reading to exploiting the difference for manipulation. One nonce reuse destroys all confidentiality.

3-5. GCM — The Mode That Also Stops Tampering

gcm = AES.new(key, AES.MODE_GCM)
ct4, tag = gcm.encrypt_and_digest(b"important message")
bad = bytearray(ct4); bad[0] ^= 1
try:
    AES.new(key, AES.MODE_GCM, nonce=gcm.nonce).decrypt_and_verify(bytes(bad), tag)
    print("GCM tamper detection: failed (passed through)")
except ValueError as e:
    print("GCM tamper detection:", repr(str(e)))

GCM tamper detection: 'MAC check failed'

How to read the output: flip a single ciphertext bit and the authentication tag check fails before any decryption result is produced. Step 232’s padding oracle and bit-flipping needed "a server that decrypts tampered ciphertexts for you" — GCM changes that server’s first action to "discard the tampering." That’s why modern TLS uses AES-GCM (or ChaCha20-Poly1305).

3-6. A Map of Mode-Specific Vulnerabilities

Mode Structure Signature vulnerability Modern verdict
ECB Independent blocks Pattern leak (3-2), byte-at-a-time Never use
CBC Chained to previous block Padding oracle, bit-flipping (Step 232) Legacy. If you must, Encrypt-then-MAC
CTR Counter keystream XOR Nonce reuse (3-4), no integrity Only as GCM’s encryption component
GCM CTR + GHASH tag Nonce reuse leaks even the auth key Modern standard (AEAD)

One caution: GCM is also fatally vulnerable to nonce reuse (it escalates to leaking the authentication key). Even a "good mode" collapses if you break the rules, and there’s only one rule — a key + nonce combination, exactly once.

4. Missions & Exercises

Mission — AES Mode Comparison Lab Notes

  1. Organize 3-1’s block-statistics experiment into a function block_uniqueness(ct, bs=16), and compare unique block counts across ECB/CBC/CTR with the same repeated plaintext
  2. Reproduce 3-2’s BMP experiment and open ecb.bmp and cbc.bmp in an image viewer — confirming the square with your own eyes is this mission’s completion condition
  3. In 3-4’s CTR reuse attack, this time assume you know neither plaintext and just print c1 ⊕ c2 — observe the English-XOR-English pattern (space XOR letter = case flip)
  4. (Optional) Encrypt two messages with a reused nonce in GCM and test whether tag verification still works (does only confidentiality break while integrity checks pass?)

Exercises

Exercise 1. Of AES’s four round operations, identify "the operation the key participates in" and "the operation that creates nonlinearity," and answer what would happen without each.

Exercise 2. Explain why the square stays visible in an ECB-encrypted bitmap, connecting it to the block statistics (unique block count).

Exercise 3. Show with a formula why decryption in CTR mode is exactly the same operation as encryption. Also answer how this property makes implementation mistakes (keystream reuse) more dangerous.

Exercise 4. Explain why the padding oracle (Step 232) doesn’t work against GCM mode, based on "the processing order of a tampered ciphertext."

5. Model Answers & Completion Criteria

Mission Model Answer

def block_uniqueness(ct, bs=16):
    blocks = [ct[i:i+bs] for i in range(0, len(ct) - len(ct) % bs, bs)]
    return len(blocks), len(set(blocks))

Grading criteria: ① in the three-mode comparison, only ECB’s unique block count converges to 1 (with repeated plaintext) ② the square outline in ecb.bmp confirmed by eye ③ in the c1 ⊕ c2 observation, confirm that "XOR with a space (0x20) flips a letter’s case" — e.g., 'A' ⊕ ' ' = 'a'. This observation is the decryption clue (crib dragging) when you know neither plaintext.

How to verify: in mission 4, reused-nonce GCM leaks plaintext via ciphertext XOR but tag verification still passes normally — confirming that confidentiality and integrity are separate axes is this experiment’s correct answer.

Exercise Answers

Answer 1. Only AddRoundKey involves the key — without it the cipher becomes a public function independent of the key that anyone can invert. Nonlinearity comes from SubBytes (the S-box) — without it, ShiftRows/MixColumns/AddRoundKey are all linear (XOR and matrix multiplication), so no matter how many rounds you repeat, the whole thing collapses into a single linear transformation solvable by simultaneous equations. A strong cipher is the repetition of "nonlinear substitution + linear diffusion + key injection" — that’s SPN’s design principle.

Answer 2. A bitmap’s solid regions repeat the same pixel pattern across thousands of blocks. ECB is a fixed table mapping the same input block to the same output, so a solid region remains "one unique block" even after encryption — in our measurement the original’s 4 kinds of blocks stayed 4 kinds after ECB. Even if the pixel values changed, the boundary information "same color up to here" wasn’t erased by a single bit, and that boundary renders as the square’s outline.

Answer 3. From encryption C = P ⊕ KS (KS is the keystream): C ⊕ KS = P ⊕ KS ⊕ KS = P — XOR is its own inverse, so XORing the same keystream again decrypts. Since Enc and Dec take the same code path in the implementation, neither compiler nor library catches a nonce-management mistake (recalling the same nonce). Worse, when reuse happens the output still looks like perfectly fine ciphertext — the mistake becomes quietly fatal.

Answer 4. The padding oracle’s fuel is "a server that decrypts the tampered ciphertext and then reports the padding-check result." GCM verifies the authentication tag before decrypting, and if even one bit differs it discards the input with MAC check failed without ever producing plaintext (measured in 3-5). To be an oracle, a query must reach the decryption path — tampered input is blocked at that entrance, so the query never materializes.

Completion Criteria Checklist

  • [ ] I can state AES’s 16-byte block, 128/192/256-bit keys → 10/12/14 rounds correspondence
  • [ ] I can explain the roles of the four SPN operations (substitution, diffusion, key injection)
  • [ ] I confirmed the ECB/CBC unique-block-count difference by measurement
  • [ ] I confirmed the square outline in ecb.bmp with my own eyes
  • [ ] I recovered the second plaintext via CTR nonce reuse
  • [ ] I measured GCM’s MAC check failed
  • [ ] I can reproduce the mode-vulnerability table with the book closed
  • [ ] Mission: comparison lab notes complete (three-mode stats + BMP observation + XOR observation)

6. Common Pitfalls & Fixes

Wall 1. I encrypted a BMP and the image viewer won’t open it

Symptom: opening ecb.bmp gives a "corrupted file" error.
Cause: you encrypted the BMP header (the first 54 bytes) too — viewers parse the file via the header’s BM magic and size fields.
Fix: keep the header in plaintext and encrypt only the body (pixels), as in 3-2. In this experiment’s framing, the header is "the file format’s skeleton," not an encryption target.

Wall 2. CTR throws TypeError or the reuse doesn’t work

Symptom: the second encryption differs from the first.
Cause: pycryptodome’s Counter object advances its internal state as it encrypts — using the same Counter object twice encrypts with the continuing counter.
Fix: to reproduce nonce reuse as in 3-4, create a fresh Counter.new(128, initial_value=100) each time. Conversely, understand that in production this very "reuse" is the vulnerability — keep the two perspectives separate.

Wall 3. ValueError from feeding ECB/CBC without padding

Symptom: ValueError: Data must be aligned to block boundary in ECB mode (measured in Step 232).
Cause: ECB/CBC only accept inputs that are multiples of 16 bytes. CTR/GCM are stream-family and have no such restriction, so this error doesn’t occur there — padding requirements genuinely differ per mode.
Fix: always apply pkcs7_pad (Step 232) first for block modes.

Wall 4. Using only decrypt in GCM and skipping verification

Symptom: a tampered ciphertext still produces a decryption result.
Cause: calling only decrypt() decrypts without tag verification — GCM’s security lives in the decrypt_and_verify pair.
Fix: always use decrypt_and_verify(ct, tag). Decryption without verification resurrects the CBC-era oracle problem as-is.

Wall 5. The conclusion "it’s AES-256, so it’s safe"

Symptom: you judge a design’s safety by key length alone.
Cause: today’s entire measurement is the counterexample — ECB (3-2) and CTR reuse (3-4) break identically with a 256-bit key. Attacks don’t find the key; they exploit mode rule violations.
Fix: the checklist is not key length but mode, nonce, authentication — "Is it not ECB? Is the nonce unique? Is tampering verified (AEAD)?"

7. Summary

Today’s Concepts

Concept One-line explanation
SPN Repetition of substitution (S-box) + permutation (rows/columns) + key XOR — 10~14 rounds
Avalanche effect A 1-bit input change spreads across the entire output — a condition of strong ciphers
ECB Independent blocks → pattern preservation — never use
CBC Chained to the previous block — no integrity, vulnerable to oracles/flipping
CTR Counter keystream XOR — stream encryption, nonce reuse forbidden
GCM (AEAD) CTR + authentication tag — confidentiality and integrity together, the modern standard
nonce A starting value whose combination with a key must be used only once

Today’s Commands & Code

Command What it does
AES.new(key, AES.MODE_ECB) ECB (for study/comparison only, forbidden in production)
AES.new(key, AES.MODE_CBC, iv=iv) CBC (legacy)
AES.new(key, AES.MODE_CTR, counter=ctr) CTR stream mode
AES.new(key, AES.MODE_GCM) + encrypt_and_digest AEAD encryption + tag
decrypt_and_verify(ct, tag) Verified decryption — the correct way to use GCM
Counter.new(128, initial_value=v) Create a CTR counter

An Instinct More Important Than Commands

Looking back at the middle of the Crypto track, a pattern emerges — what broke was always an implementation that violated a mathematical premise. RSA fell in key generation that violated "n doesn’t factor"; AES fell in usage that violated "blocks must not be independent" (ECB), "tampering must be verified" (CBC), and "nonces must be unique" (CTR). A strong algorithm is a set of conditions; the side that keeps them is the defender, the side that breaks them writes the challenge. Don’t memorize the mode-vulnerability table — memorize the violated rule in each cell.

Once every box is checked, Step 233 is complete.

← 이전 스텝
다음 스텝 →