Step 48. Regular Expressions — A Magnet That Finds by Shape

Level 1 — Programming and the Computer’s Inner Workings | Difficulty ★★★☆☆ | Estimated time: 3.5 hours

Prerequisites: Steps 41–46 complete. You can use string manipulation (split, in) and lists.

What you need: a computer with Python installed and a text editor. Nothing new to install.
Caution: today’s exercise is 100% safe. However, the unfamiliar symbols may look like cipher at first — being confused at the start is perfectly normal.

Say you want to extract only "things that look like IP addresses" from a million-line log. You can’t do it with in. Because you’re not looking for the exact string "1.2.3.4" — you’re looking for everything with the shape number.number.number.number. Finding by shape — that’s today’s topic, and the "language that describes shapes" is regular expressions (regex). It appears in almost every text task of security study — log analysis, flag hunting, input validation — and once learned, it’s a tool for life.

1. Learning Objectives

By the end of this chapter, you will be able to:

Explain the meaning of d, w, ., +, *, {n,m} and assemble patterns
Extract desired pieces (IPs, emails, dates) from text with re.findall
Capture only part of a pattern with parenthesis groups
Know the difference between re.search and re.fullmatch and use the right one for validation
Learn the work method of building patterns up from the simplest one

2. Background Knowledge — Today’s Tools and Concepts

Today’s Tools at a Glance

Category	Details
Language/environment	Python 3 + the standard `re` module (no installation needed)
Today’s functions	`re.findall` (extract all), `re.search` (the first one), `re.fullmatch` (full-match validation)
Today’s symbols	`d` `w` `.` `+` `*` `{n,m}` `[ ]` `( )` `.`
Concepts needed	metacharacters, raw strings (r""), group capture, escaping

2-1. Metacharacters — The Meaning Table of Symbols

The ingredients of regex are symbols with special meanings (metacharacters). Today’s core six:

d : one digit (0–9)
w : one word character (letters, digits, underscore)
. : any single character
+ : one or more of the preceding thing
* : zero or more of the preceding thing
{n,m} : at least n and at most m of the preceding thing

Ordinary characters just mean themselves. So d+.d+ is the pattern "one or more digits, a dot, one or more digits." It’s not cipher — just shorthand.

2-2. Raw Strings — The Backslash Protector

Regex uses lots of , but in Python strings also gets special treatment (e.g., n is a line break). To prevent the collision, regex patterns are always written as r"..." (raw strings). The r in front is the sign "interpret this literally." It’s less a rule than a vaccination — skip it and someday, without fail, there’s an accident.

2-3. The re Module — Three Ways of Asking

Python’s regex tool is the standard library re. There are three questions you’ll ask often.

re.findall(pattern, text): everything matching the pattern, all of it, as a list
re.search(pattern, text): the first match. None if nothing matches
re.fullmatch(pattern, text): does the entire text match the pattern?

Today we use findall as the main weapon — for "extracting," it’s the right answer.

2-4. Groups — Taking Only Part of the Pattern

A part wrapped in parentheses ( ) becomes the sign "give me just this part separately." This is called group capture. If you want only the year out of the date 2026-09-08, wrap the year part in parentheses like (d+)-d+-d+.

3. Follow Along

3-1. First Pattern — Extracting Numbers

Input (re1.py)

import re

text = "Ports 80 and 443 are open, and 22 is closed"
result = re.findall(r"d+", text)
print(result)

Output (verified on 2026-09-09):

['80', '443', '22']

How to read it: d+ — findall found every chunk where "a digit (d) appears one or more times (+) in a row." Remember also that what gets extracted is not numbers but strings. To calculate, you need int() conversion.

3-2. Specifying Counts — Exactly How Many Digits

Input

text = "Phone 010-1234-5678, PO box 12345"
print(re.findall(r"d{4}", text))
print(re.findall(r"d{3}-d{4}-d{4}", text))

Output (verified on 2026-09-09):

['1234', '5678', '1234']
['010-1234-5678']

How to read it: d{4} is "exactly 4 digits." The phone number’s middle and last four digits, plus the first four digits of the box number (12345), got caught — three in total. The second pattern is the phone-number shape itself, "3 digits – 4 digits – 4 digits," so exactly one gets caught. The more specific the pattern, the more precise the catch.

Predict: what would d{2,4} catch? "At least 2, at most 4." The verified result is ['010', '1234', '5678', '1234'] — notice that from 12345, only the leading 1234 gets caught. Meeting a long chunk, regex’s default nature is to grab greedily up to the maximum (4).

3-3. Extracting IP Addresses — Today’s Main Drill

Input (re3.py)

import re

log = "Connection 192.168.0.23 success, connection 10.0.0.7 failed, attack attempt 1.2.3.4 detected"
ips = re.findall(r"d+.d+.d+.d+", log)
print(ips)

Output (verified on 2026-09-09):

['192.168.0.23', '10.0.0.7', '1.2.3.4']

How to read it: d+.d+.d+.d+ — "digit chunk, dot, digit chunk, dot, digit chunk, dot, digit chunk." The before a dot means "a literal dot character." A plain . means "any character," so when you want the dot itself, you pin it down as ..

A limitation to note: this pattern only looks at shape. Experiment and you’ll see even a fake IP like 999.999.999.999 gets caught (confirmed by testing on 2026-09-09). Rigorous validation (whether each chunk is 255 or less) is left as a later task.

3-4. Extracting Email Shapes

Input

text = "Contact admin@example.com or support@test.org for inquiries"
emails = re.findall(r"w+@w+.w+", text)
print(emails)

Output (verified on 2026-09-09):

['admin@example.com', 'support@test.org']

How to read it: w+@w+.w+ — "character chunk, @, character chunk, dot, character chunk." Simple, but quite usable. A truly complete email regex is a several-hundred-character monster, but in practice this level of simple pattern handles most of the work.

3-5. Capturing Parts with Groups

Input

text = "Date: 2026-09-08, Date: 2025-12-31"
years = re.findall(r"(d+)-d+-d+", text)
print(years)

Output (verified on 2026-09-09):

['2026', '2025']

How to read it: the whole pattern is a date shape, but only the parenthesized year part went into the list. "Find by the whole shape, but take only a part" — that’s the trick of group capture.

What if you use three parentheses? Verified (2026-09-09):

[('2026', '09', '08'), ('2025', '12', '31')]

You get a list of (year, month, day) tuples.

3-6. search and Judgment — "Is It There or Not"

Input

line = "ALERT: access denied from 10.0.0.7"
if re.search(r"ALERT", line):
    print("This is an alert line!")
found = re.search(r"d+.d+.d+.d+", line)
print(found.group())

Output (verified on 2026-09-09):

This is an alert line!
10.0.0.7

How to read it: search returns "found it / didn’t find it," so it pairs well with if. The found content is pulled with .group(). When nothing is found, None comes back — build the habit of checking whether it found something before calling .group() (we see the accident at Wall 3).

3-7. Character Classes — Finding "One of These"

Square brackets [ ] mean "one of the characters inside."

Input

import re
text = "Files: a1.txt, b2.txt, c10.txt"
print(re.findall(r"[abc]d", text))
print(re.findall(r"[a-z]d+", text))

Output (verified on 2026-09-09):

['a1', 'b2', 'c1']
['a1', 'b2', 'c10']

How to read it: [abc] is a or b or c. [a-z] is any lowercase letter from a to z. The first pattern is "a/b/c + one digit," so from c10 only c1 gets caught; the second catches it whole thanks to +. Inside brackets, - is a range and ^ means "not these" — [^0-9] is "one character that is not a digit."

3-8. Validation Patterns — "Is This Input Valid"

There’s a use different from extraction (findall): validation — checking "does the user’s input follow the rule?"

Input

import re

def is_valid_port(text):
    return re.fullmatch(r"d{1,5}", text) is not None

print(is_valid_port("8080"))
print(is_valid_port("80a0"))
print(is_valid_port(""))

Output (verified on 2026-09-09):

True
False
False

How to read it: re.fullmatch asks "does the entire text match the pattern exactly?" A partial match isn’t enough. The difference from search matters — search asks "is it inside?", fullmatch asks "is the whole thing this shape?" For validation, always use fullmatch.

Why: input validation is Step 46’s defensive programming deployed forward. Expressing "only accept what follows the rule" in regex keeps weird input from entering the program’s interior.

4. Missions & Exercises

Mission — Complete the IP Extraction Function

Complete the extract_ips(text) function. Requirements:

Write a function that takes arbitrary text, extracts every IPv4 shape, and returns them as a list
Remove duplicates before returning (hint: Step 45’s set)
Add a feature that separately picks out private IPs — check whether an extracted IP starts with "192.168." or "10." using the string’s startswith
Create your own fake log file (log_sample.txt, at least five lines with several IPs mixed in), read it in, and test the function
Print a summary of the results: "total IP count, distinct IP count, and the list of private IPs among them"

Bonus challenge: interpret the pattern flag{[^}]+}, which finds the flag{...} shape.

Exercises

Question 1. State the meaning of each of the six symbols d, w, ., +, *, {n,m}.

Question 2. Why do we use raw strings (r"...") when writing regex patterns?

Question 3. Explain the difference between re.search and re.fullmatch, and state which one should be used for "user input validation."

Question 4. The pattern d+.d+.d+.d+ catches even 999.999.999.999 as an IP. Explain the reason for this limitation, and state why this pattern is still useful.

5. Model Answers & Completion Criteria

Mission Model Answer

import re

def extract_ips(text):
    found = re.findall(r"d+.d+.d+.d+", text)
    return list(set(found))

def private_only(ips):
    result = []
    for ip in ips:
        if ip.startswith("192.168.") or ip.startswith("10."):
            result.append(ip)
    return result

with open("log_sample.txt", encoding="utf-8") as f:
    log = f.read()

ips = extract_ips(log)
privates = private_only(ips)
print(f"Total IP count (with duplicates): {len(re.findall(r'd+.d+.d+.d+', log))}")
print(f"Distinct IP count: {len(ips)}")
print(f"Private IPs: {privates}")

How to verify: ① Write the same IP three times in the fake log and check that "distinct IP count" comes out as 1. ② Mix 192.168.x.x, 10.x.x.x, and 8.8.8.8 (public) and check that only the private ones get picked. ③ Check that a fake with only three chunks, like "1.2.3", is not caught.

Bonus commentary: flag{[^}]+} means "flag, a literal opening brace ({), one or more characters that are not a closing brace ([^}]+), a literal closing brace." In verification, from the result is flag{h3ll0_w0rld} and, exactly flag{h3ll0_w0rld} was caught (2026-09-09).

Exercise Answers

Answer 1. d one digit, w one word character (letters, digits, underscore), . any single character, + one or more of the preceding, * zero or more of the preceding, {n,m} at least n and at most m of the preceding.

Answer 2. In Python strings, is interpreted with special meanings like n (line break), and since regex also uses heavily, the two collide. r"..." is the sign "interpret literally," preventing the pattern’s d from being mangled by Python.

Answer 3. search asks "is the pattern somewhere inside the text?", fullmatch asks "is the entire text exactly the pattern?" For input validation, use fullmatch — because search would pass input like "8080abc" where only the front part matches.

Answer 4. Because this pattern only looks at "shape" and doesn’t check each chunk’s value range (0–255). Why it’s still useful: most "IP shapes" appearing in logs are real IPs, so this simple pattern is sufficient as a first-pass collector. The practical approach is to split stages — collect first, then add range validation.

Completion Criteria Checklist

[ ] I can explain the meanings of d, w, ., +, *, {n,m}
[ ] I know why raw strings (r"") are needed
[ ] I can extract pieces matching a pattern with findall
[ ] I can capture only a part with groups ( )
[ ] I distinguish search from fullmatch and use fullmatch for validation
[ ] I know a literal dot must be found with .
[ ] Mission: I completed the extract_ips function and the private-IP classification

6. Common Pitfalls & Fixes

Wall 1. The pattern catches nothing at all

Symptom: it should clearly be there, but an empty list [] comes out.
Cause: three classics. You dropped the r so d got mangled; a spelling mistake in the pattern; or the shape you’re looking for and the pattern differ subtly (whitespace, letter case).
Fix: build the pattern up from the simplest one. Start with d, and if that works, d+, then add .. The iron rule of regex work is never trying to complete it in one shot.

Wall 2. Looking for a dot, but wrong things get caught

Symptom: things like "151" and "1a1" get caught too. Verified comparison (2026-09-09):

pattern r"1.1"  -> ['151', '1a1', '1.1']
pattern r"1.1" -> ['1.1']

Cause: . means "any single character." A literal dot is ..
Fix: when finding a character with special meaning (., *, +, ?, {, }, [, ], (, ), ^, $, , |) as the character itself, put in front. This is called escaping.

Wall 3. Error saying NoneType has no group

Symptom (verified on 2026-09-09):

AttributeError: 'NoneType' object has no attribute 'group'

Cause: search found nothing and returned None, and you called .group() on it directly. It’s like opening the lid of a box that doesn’t exist.
Fix: use .group() only after an if found: check. Step 46’s defensive programming works here too.

Wall 4. A too-short pattern catches only half

Symptom: you tried to catch an IP but only the first two chunks come out — or conversely, surrounding characters get dragged in.
Cause: the pattern is shorter (less specific) or longer than the actual shape. d+.d+ catches only the first two chunks of an IP.
Fix: first write the "exact shape" of what you want to catch in words. "Digit chunk dot digit chunk dot digit chunk dot digit chunk." That sentence is the pattern’s blueprint as-is.

Wall 5. Long digit chunks get cut off

Symptom: you want to catch 12345 but only 1234 comes out.
Cause: d{4} means "exactly 4," so it catches only the first four digits of a five-digit number (the case in 3-2’s verification where PO box 12345 was caught as 1234).
Fix: choose according to intent. Exactly 4 digits: {4}; "at least 4 digits": {4,}; a range: {4,6}. Also remember the default nature: meeting a long chunk, it grabs greedily up to the maximum.

7. Summary

Today’s Concepts

Concept	One-line description
Regular expression (regex)	A "language describing shapes" — finding by shape
Metacharacter	A symbol with special meaning (d, w, ., +, *, {n,m})
Raw string	r"…" — the vaccination against backslash collisions
Group capture	Taking only the parenthesized ( ) part separately
Escape	Finding a special character as itself (., etc.)
Character class	[ ] — "one of the characters inside"

Today’s Functions & Symbols

Tool	What it does
`re.findall(pattern, text)`	Everything matching, as a list
`re.search(pattern, text)`	The first one. None if nothing
`re.fullmatch(pattern, text)`	Full-match check (for validation)
`d` `w` `.`	digit / word character / any single character
`+` `*` `{n,m}`	one or more / zero or more / count range
`[abc]` `[^0-9]`	one of these / something that is not these
`.` `{`	a literal dot / a literal brace

Instincts More Important Than Commands

Half of regex study is not writing but reading other people’s patterns. When you meet w+@ in code on the internet, don’t skip past it — unpack it symbol by symbol: "character chunk, at sign." Even professionals don’t memorize regex; they look it up. The goal is to become someone who looks things up to use them, but can read them.

The security connection: CTF answers are usually hidden in a promised shape like flag{...}, and finding an intruder’s IP in a log is exactly today’s skill. Move today’s patterns into the live test bench regex101.com, and you’ll see the caught parts highlighted in color right away — instinct attaches quickly. Leave your first completed d+.d+.d+.d+ in your notebook — it’s your entry stamp into the world of regex.

Once every box is checked, Step 48 is complete.