Step 224. Strategies for Analyzing Obfuscated Code — Read It Head-On and You Lose

Level 3 — Real CTF and Advanced Attack Skills | Difficulty ★★★☆☆ | Estimated time: 4 hours

Prerequisites: Step 215 (Reading Function Structure with Ghidra), Step 220 (Packing/Unpacking), Step 222 (.NET/Python Binary Reversing). You can read and write Python code.

⚠️ All exercises in this chapter are for your own lab and legal platforms only. Applying them to unauthorized systems is a crime.

• What you need: Python 3 (measured: 3.12.14), a text editor. Today’s analysis target is a harmless serial verification script you write yourself.
• Caution: real-world obfuscated binaries (with packers and anti-debugging aboard) are handled together with Step 220’s techniques. Today we shrink it down to Python source to focus on the "strategy" itself.

Solving crackmes in Steps 217~218, you’ll meet it — code where variable names are _0x4a2f, strings are number arrays, and jumps happen by state number inside a while True. Read it head-on, line by line, and you lose. Obfuscation is the craft of "making reading painful," and the analyst’s response is not reading but bypassing. Today you build the three types of obfuscation yourself (name destruction, string encryption, control-flow flattening) and measure each one’s bypass strategy hands-on.

1. Learning Objectives

By the end of this chapter, you will be able to:

• Distinguish the three types of obfuscation (name destruction, string encryption, control-flow flattening)
• Explain the call-pattern clue for finding the "decryption function" in string encryption
• Reuse the decryption routine to batch-restore encrypted strings (static bypass)
• Abstract control flow flattened into a state machine, block by block, to restore the original order
• Apply the analysis standard of "not perfect deobfuscation, just the answer to my question"

2. Background Knowledge — Today’s Tools and Concepts

Today’s Tools at a Glance

2-1. Obfuscation Tangles Three Things

Obfuscation leaves a program’s behavior intact while making only the reading hard. The types you meet in the field are broadly three.

First, name destruction — check_serial becomes _0x7e1, and key becomes _0x3a. The "meaningful names" that are the footholds of reading disappear.

Second, string encryption — a string like "FAIL: length" is stored as a number array like [28, 27, 19, ...] and appears only at runtime through a decryption function. First-pass recon with strings is neutralized.

Third, control-flow flattening — code that originally flowed top to bottom is rewritten as a giant while True + state variable. Each block is shaped like "if state is 0, do this and set state to 1," so the execution order can’t be known from the code’s layout order.

2-2. Strategy ① — Find the Decryption Function

If every string is a number array, a decryption function that turns those arrays into strings must exist somewhere in the code — because the program must decrypt strings to use them.

The finding clue is the call pattern — some function takes an array argument and gets called repeatedly wherever a string is needed (print, input, return). That function is the decryption function. Once found, there are two bypass paths.

• Static bypass: if the decryption algorithm is simple (a single XOR, etc.), implement the same transform in Python and batch-restore all the arrays.
• Dynamic bypass: if the algorithm is complex, set a breakpoint at the decryption function’s "exit" with a debugger and catch the restored plaintext from memory at runtime. Today we measure the static bypass and touch the dynamic one only conceptually, as an extension of Step 216 (x64dbg).

2-3. Strategy ② — Abstract Flattening Block by Block

Read flattened code head-on and you get lost in a forest of while and if _0x55 == N. Instead, look at it like this.

1. Summarize what each state-number block does in one line ("state 0: length check")
2. Write down which state number each block moves to next (0 → 1 → 2 → …)
3. Erase the state numbers and rearrange the summaries in order

Then the original sequential code emerges. The key is treating each block as a black box from an "input → output" perspective.

2-4. Strategy ③ — Don’t Aim for Perfection

The most important strategy. Deobfuscation’s goal is not restoring code cleanly but answering my question. Once you’ve answered "where is the serial comparison logic and what are its conditions," the rest of the tangle doesn’t need untangling. Aim for perfect restoration and the time is endless; dig out only the answer you need and it finishes in tens of minutes.

3. Follow Along

3-1. The Analysis Target — An Obfuscated Serial Verifier

Below is today’s analysis target s224_obf.py. All three types are present (every Python output in this chapter was measured 2026-09-09 on Python 3.12.14).

def _0x4a2f(_0x1b):
    return bytes(_0x9c ^ 0x5A for _0x9c in _0x1b).decode()

_0xd1 = [28, 27, 19, 22, 96, 122, 54, 63, 52, 61, 46, 50]
_0xd2 = [28, 27, 19, 22, 96, 122, 41, 47, 55]
_0xd3 = [28, 27, 19, 22, 96, 122, 63, 62, 61, 63]
_0xd4 = [28, 27, 19, 22, 96, 122, 34, 53, 40]
_0xd5 = [21, 17]
_0xd6 = [41, 63, 40, 51, 59, 54, 96, 122]

def _0x7e1(_0x3a):
    _0x55 = 0
    while True:
        if _0x55 == 0:
            if len(_0x3a) != 8:
                return _0x4a2f(_0xd1)
            _0x55 = 1
        elif _0x55 == 1:
            _0x77 = 0
            for _0x12 in _0x3a:
                _0x77 += ord(_0x12)
            if _0x77 != 728:
                return _0x4a2f(_0xd2)
            _0x55 = 2
        elif _0x55 == 2:
            if _0x3a[0] != 'R' or _0x3a[7] != '9':
                return _0x4a2f(_0xd3)
            _0x55 = 3
        elif _0x55 == 3:
            if ord(_0x3a[3]) ^ ord(_0x3a[4]) != 0x1F:
                return _0x4a2f(_0xd4)
            _0x55 = 4
        else:
            return _0x4a2f(_0xd5)

_0x9b = input(_0x4a2f(_0xd6))
print(_0x7e1(_0x9b))

First, run it and check only the behavior — before analysis, see "what this program does on the outside."

printf 'shortn' | python s224_obf.py
printf 'RCDe9fF9n' | python s224_obf.py

serial: FAIL: length
serial: FAIL: sum

How to read the output: it takes input and rejects with "length" if the length is wrong and "sum" if the sum is wrong. That it’s a serial verifier shows from behavior alone. Now "what exactly are the verification conditions" is our question.

3-2. First Recon — Finding the Decryption Function from the Call Pattern

Scan the code and check the three types.

• Name destruction: _0x7e1, _0x3a, _0x55 — meaningless ✅
• String encryption: not a single human-readable string in the code; six number arrays instead ✅
• Control-flow flattening: _0x7e1 is built from while True + the state variable _0x55 ✅

And find the decryption function. Look at _0x4a2f — it’s called as the argument of input(...), at four returns, whenever an array is involved. Its body is one line: bytes(each byte ^ 0x5A). A decryption function with XOR key 0x5A. A simple form where the static bypass works.

3-3. Static Bypass — Batch-Restoring the Strings

Reuse the decryption routine as-is to restore all six arrays.

tables = {
    "_0xd1": [28, 27, 19, 22, 96, 122, 54, 63, 52, 61, 46, 50],
    "_0xd2": [28, 27, 19, 22, 96, 122, 41, 47, 55],
    "_0xd3": [28, 27, 19, 22, 96, 122, 63, 62, 61, 63],
    "_0xd4": [28, 27, 19, 22, 96, 122, 34, 53, 40],
    "_0xd5": [21, 17],
    "_0xd6": [41, 63, 40, 51, 59, 54, 96, 122],
}
for name, data in tables.items():
    print(name, "->", bytes(b ^ 0x5A for b in data).decode())

_0xd1 -> FAIL: length
_0xd2 -> FAIL: sum
_0xd3 -> FAIL: edge
_0xd4 -> FAIL: xor
_0xd5 -> OK
_0xd6 -> serial: 

How to read the output: every number array became a sentence. And the strings tell you the verification order — checks in the order length, sum, edge, xor, ending with OK. No debugger — six lines of Python peeled off the whole string-encryption layer.

3-4. Un-flattening — Block Abstraction and Rearrangement

Now untangle _0x7e1‘s state machine. Summarize each state block in one line and follow the next state.

state 0: if len != 8 → "FAIL: length"  → next state 1
state 1: if char-code sum != 728 → "FAIL: sum"  → next state 2
state 2: if first char not 'R' or last char not '9' → "FAIL: edge"  → next state 3
state 3: if key[3] XOR key[4] != 0x1F → "FAIL: xor"  → next state 4
state 4: return "OK"

Erase the state numbers and rearrange in order, and the original code is restored.

def check(key):                    # restored pseudocode
    if len(key) != 8:
        return "FAIL: length"
    if sum(ord(c) for c in key) != 728:
        return "FAIL: sum"
    if key[0] != 'R' or key[7] != '9':
        return "FAIL: edge"
    if ord(key[3]) ^ ord(key[4]) != 0x1F:
        return "FAIL: xor"
    return "OK"

How to read it: the while True forest became five lines of an ordinary verification function. As per 2-4’s standard — we’ve answered our question ("what are the verification conditions"), so deobfuscation ends here. Nicely restoring the variable names to their originals is not something we do.

3-5. Completion — Finding a Serial That Satisfies the Conditions

Build an answer from the restored conditions and verify it. With four conditions, hand-matching is hard, so search with Python.

import itertools, string
chars = string.ascii_letters
for c3 in chars:
    c4 = chr(ord(c3) ^ 0x1F)          # condition: key[3] ^ key[4] == 0x1F
    if c4 not in chars:
        continue
    for combo in itertools.product(chars, repeat=3):
        last = 728 - ord('R') - ord('9') - ord(c3) - ord(c4) - sum(map(ord, combo))
        if 0x61 <= last <= 0x7A:      # remaining char constrained to lowercase
            key = 'R' + combo[0] + combo[1] + c3 + c4 + combo[2] + chr(last) + '9'
            print('key:', key)
            raise SystemExit

key: RaaezAk9

Verify by feeding it to the original obfuscated script.

printf 'RaaezAk9n' | python s224_obf.py

serial: OK

How to read the output: OK appeared. Without reading a single line of the obfuscated code "head-on" — with only decryption-routine reuse and block abstraction — we grasped the core logic and derived the answer. This is the full run of today’s strategy.

4. Missions & Exercises

Mission — Build Your Own Obfuscated Sample and Write a Dissection Report

1. Modify today’s verifier into your own sample: change one verification condition (e.g., sum 728 → a different number) and change the XOR key too (0x5A → a different value)
2. Reopen that file from an analyst’s viewpoint and write a dissection report with five items: ① the three-type check ② the basis for identifying the decryption function (call pattern) ③ the batch string-restoration output ④ the restored pseudocode ⑤ the correct serial and OK output
3. At the report’s end, write in two sentences "if it had been not XOR but a complex cipher that only unlocks at runtime, which bypass would you have used"

Exercises

Exercise 1. Explain the difference between obfuscation and encryption (Step 169) in terms of "where the decryption function/key lives."

Exercise 2. Name two clues for finding the decryption function in a string-encrypted program.

Exercise 3. Between the static and dynamic bypass, which must you choose if the decryption algorithm is "AES decryption with a key received over the network at runtime"? Why?

Exercise 4. In 3-4’s state machine, if you rearranged state 1 (the sum check) to execute before state 0 (state numbers kept, only _0x55‘s initial value changed to 1), would the result differ? What changes for the analyst?

1. Type check: name destruction ✅ / string encryption (6 arrays) ✅ / flattening (while+state variable) ✅
2. Decryption function: _0x4a2f — repeatedly called with array arguments at input/return sites. Body is one XOR line
3. Batch restoration: bytes(b ^ 0x33 ...) → obtained 6 strings including "FAIL: length"
4. Pseudocode: rearranged states 0→4 into a five-line verification function
5. Answer: serial generated by a search script → run result OK (output attached)

6. Common Pitfalls & Fixes

Wall 1. Restoring the arrays throws UnicodeDecodeError

Symptom: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x.. in position ..
Cause: you picked the wrong XOR key (e.g., 0xA5 instead of 0x5A), or the array is the product of a different transform, not XOR.
Fix: re-read the decryption function body’s key and operation character by character. If the restoration "looks like letters but is partially broken," the key is partially wrong; if it’s entirely broken, the transform itself is different. In that case, copying the decryption function wholesale as if importing it is the safe move.

Wall 2. Input doesn’t go through with printf '...' | python

Symptom: EOFError: EOF when reading a line (measured 2026-09-09, when run without input).
Cause: input() needs standard input, but you didn’t pipe any.
Fix: pipe it with a newline included, like printf 'serialn' | python s224_obf.py, or run interactively.

Wall 3. In the state machine, you miss a "next state" and read it as an infinite loop

Symptom: while summarizing blocks, a block appears whose destination you can’t tell.
Cause: you missed a _0x55 = N assignment, or you didn’t look at the else branch (the default next state).
Fix: underline the assignment at each block’s end first. Gather the next-state numbers, and only then read the block contents — content only has meaning after the order is fixed.

Wall 4. The answer search finds nothing

Symptom: the 3-5 search script ends with no output.
Cause: the allowed character range is too narrow to satisfy the sum condition. For example, with only uppercase letters, a 6-character sum can’t reach 728 (confirmed in measurement — lowercase must be allowed for it to work).
Fix: widen the allowed character set (uppercase/lowercase/digits) and adjust the allowed range of the last character (e.g., 0x61~0x7A).

Wall 5. You exhaust yourself restoring "perfectly"

Symptom: you guess and fix every variable name back, add comments, and spend hours.
Cause: you forgot 2-4’s standard.
Fix: write the question first — "what are the verification conditions." Once that answer (five lines of pseudocode) comes out, stop. Pretty restoration is for reports, not for analysis.

7. Summary

Today’s Concepts

Today’s Commands & Code

An Instinct More Important Than Commands

Obfuscation makes things "unreadable," not "unsolvable." The decryption function is necessarily inside the program, and flattening hides order but can’t change logic. So the analyst’s weapon isn’t meticulousness but choosing the bypass route — don’t read head-on; reuse, abstract, and untangle only as much as needed. This instinct carries straight into malware analysis in later chapters — malware is tangled far harder than today’s sample, but the order of response (recon → decryption routine → bypass choice) is the same.

Once every box is checked, Step 224 is complete.