Step 4. Text Processing — Finding a Needle in the Logs

Level 0 — Understanding How to Operate a Computer and How It’s Structured | Difficulty ★★☆☆☆ | Estimated time: 2–3 hours

Prerequisites: Step 3 complete (the pipe). We’ll work in Windows PowerShell.

What you need: a Windows PC, PowerShell.
Safety: today is again only reading and creating practice files, so it’s safe. You’ll make the practice files yourself — we’ll build a 50-line fake log and practice investigating inside it.
Caution: you may run into an incident where non-English text looks garbled. Don’t panic — today you’ll conquer even its cause (encoding).

Today’s star is "logs." Everything in the computer world leaves records — login successes/failures, program errors, connection history. And those records are all text files. An incident investigation starts with a single line like "show me only the lines containing ERROR," and as tens of thousands of lines shrink to a few dozen, a story starts to emerge. The "pull the lines you want out of text" skill you’ll learn today is the alpha and omega of security analysis.

1. Learning Objectives

By the end of this chapter, you can:

Read files with Get-Content, and peek at just the beginning of long files
Pick out only the lines containing a given word with Select-String
Count result hits and save results to a new file to create an "investigation document"
Investigate many files at once using the filename:linenumber:content format
Explain the cause of garbled non-English text (encoding mismatch) and respond with -Encoding

2. Background Knowledge — Today’s Tools and Concepts

Today’s Tools at a Glance

Category	Details
Language/environment	PowerShell 5.1, text file (log) processing
Today’s commands	`Get-Content` (read), `Select-String` (search), `Measure-Object` (count), `Out-File` (save)
Today’s options	`-TotalCount` (just the beginning), `-Path`/`-Pattern` (search target/pattern), `-Encoding` (specify encoding)
Concepts you need	text file = a sequence of lines, encoding (character↔number conversion table), `\|` (or) inside a pattern

Translated into the language of a crime scene, today’s commands look like this: Select-String is tool #1 of incident investigation — "hooking only the suspicious lines out of tens of thousands of log lines." Measure-Object is the tool that produces sentences like "300 failed logins between 2 and 4 a.m." Out-File -Encoding utf8 is the device for "preserving evidence without corruption." A security professional’s daily life involves far more time wrestling with logs using commands like today’s than wielding fancy hacking tools.

2-1. The Structure of a Text File — "Lines" Are the Unit

Open up a text file and it’s ultimately a sequence of lines. And all of today’s tools work "line by line":

Get-Content: reads a file as a list of lines (one line = one object)
Select-String: examines each line and picks out only the ones matching the condition

In Step 3 you learned "list → filter," right? Text processing is exactly the same thinking. Only the material changed from processes to "lines."

2-2. Encoding — The Conversion Table Between Characters and Numbers

Computers can’t store characters. They can only store numbers. So we need a "character ↔ number" conversion table, and that table is an encoding.

The problem is there are several kinds of tables:

Encoding	Characteristics
UTF-8	The current world standard. Handles nearly every language and emoji
UTF-16	Common inside Windows
EUC-KR / CP949	The old Korean Windows standard
Latin-1 / Windows-1252	Legacy Western European Windows standard

90% of why text looks garbled: the file was saved as numbers using table A (say, UTF-8), but the reading program interprets it with table B (say, CP949). Same numbers, different table — so the characters come out bizarre. It’s like trying to open a lock with the wrong key.

The fix: when reading, explicitly say "read with this table." We’ll practice it today.

3. Follow Along

3-1. Preparing the Crime Scene — Making a Fake Log

Create a practice ground and prepare a fake log. These are commands from Step 2:

cd $HOME
New-Item -ItemType Directory textlab
cd textlab

Now let’s make a 50-line log. You could use Notepad, but since we’re practicing the CLI, we’ll build it in PowerShell. Copy the block below in one piece and paste it into PowerShell (multiple lines will run at once):

1..50 | ForEach-Object {
  $m = "{0:D2}" -f $_
  if ($_ % 7 -eq 0) { Add-Content log.txt "2026-09-08 02:$m ERROR failed login: admin from 10.9.9.9" }
  elseif ($_ % 5 -eq 0) { Add-Content log.txt "2026-09-08 03:$m WARNING disk usage high" }
  else { Add-Content log.txt "2026-09-08 04:$m INFO normal operation" }
}

You don’t need to understand this syntax at all right now (you’ll learn it in Step 8). All you need to know: "a 50-line file where every 7th line has ERROR, every 5th line (that isn’t a multiple of 7) has WARNING, and the rest have INFO."

Check:

Get-Content log.txt -TotalCount 8

2026-09-08 04:01 INFO normal operation
2026-09-08 04:02 INFO normal operation
2026-09-08 04:03 INFO normal operation
2026-09-08 04:04 INFO normal operation
2026-09-08 03:05 WARNING disk usage high
2026-09-08 04:06 INFO normal operation
2026-09-08 02:07 ERROR failed login: admin from 10.9.9.9
2026-09-08 04:08 INFO normal operation

-TotalCount 8 — "not all of it, just the first 8 lines." An option for peeking at long files.

3-2. Reading Everything and Why That’s a Problem — Get-Content

Let’s read the whole thing:

Get-Content log.txt

50 lines whoosh past. It’s only 50 lines now, but real logs have 50,000. Feel in your bones that "reading everything with your eyes" is not an option. That’s why we need the next machine.

3-3. Investigation Begins — Select-String

Get-Content log.txt | Select-String "error"

2026-09-08 02:07 ERROR failed login: admin from 10.9.9.9
2026-09-08 02:14 ERROR failed login: admin from 10.9.9.9
...

Only the lines containing "error" were picked out. Same role as Step 3’s Where-Object, but simpler because it’s dedicated to text search.

Wait — we saved it as "ERROR" (uppercase) earlier, so why did searching for "error" (lowercase) catch it? Select-String is case-insensitive by default. (If you want case sensitivity, add the -CaseSensitive option.)

It can be even shorter — Select-String accepts a file path directly:

Select-String -Path log.txt -Pattern "error"

-Path — the file to search
-Pattern — the text/pattern to find

3-4. How Many? — Counting

When gauging the scale of an attack, counts matter:

Select-String -Path log.txt -Pattern "error" | Measure-Object

Count    : 7

ERROR went into every multiple-of-7 line, so 7 hits. "7 ERRORs" — the Measure-Object you learned in Step 3 works exactly the same here. Assembly thinking, remember?

Predict first: how many lines will Select-String -Path log.txt -Pattern "WARNING" return? Predict, then run it.
(Answer: 9 lines. There are 10 multiples of 5 (5, 10, …, 50), but 35 is also a multiple of 7, so it fell into the ERROR slot. Recall the generation rules and verify — doubting "what happens when conditions overlap?" is an analyst’s instinct.)

3-5. Preserving Evidence — Out-File

Let’s save the extracted results to a file. Raw material for a report:

Select-String -Path log.txt -Pattern "error" | Out-File errors.txt
Get-Content errors.txt

2026-09-08 02:07 ERROR failed login: admin from 10.9.9.9
...

It was written to errors.txt instead of the screen, and we read it back to verify. This flow:

search (Select-String) → save (Out-File) → verify (Get-Content)

This is the basic workflow of "leaving investigation results as a document." Real incident response reports are an expanded version of this structure.

3-6. The Garbled-Text Incident — An Encoding Experiment

This time let’s deliberately create an incident. Make a file containing non-English text (Korean, in this example):

Set-Content korean.txt "비밀번호 초기화 필요"
Get-Content korean.txt

In most cases it displays fine. (PowerShell wrote and read it with the same table.) But garbling happens when you open this file in another program, or open a file downloaded from the internet — because the writing table and the reading table differ (review 2-2).

How to specify the reading table:

Get-Content korean.txt -Encoding UTF8

-Encoding UTF8 — "read it with the UTF-8 table." The working professional’s habit goes like this:

When non-English text in a file someone else made looks garbled, change the encoding and read it again. (Alternate between attempts like -Encoding UTF8 and -Encoding Default.)

You can also fix the table when saving. UTF-8 is the world standard, so we recommend this habit from now on:

... | Out-File result.txt -Encoding utf8

3-7. Investigating Many Files at Once

Here’s where Select-String’s real power shows:

Select-String -Path *.txt -Pattern "error"

It searches every .txt file in this folder for "error." The result even shows which file and which line number (actual output):

errors.txt:1:2026-09-08 02:07 ERROR failed login: admin from 10.9.9.9
log.txt:7:2026-09-08 02:07 ERROR failed login: admin from 10.9.9.9
log.txt:14:2026-09-08 02:14 ERROR failed login: admin from 10.9.9.9
...

How to read it: filename:linenumber:content. Thanks to this feature, you can investigate hundreds of log files in one shot.

4. Missions & Exercises

Mission — A Day in the Life of a Log Analyst

Incident: you’ve been ordered to "find suspicious activity in our server logs and submit a report." Work in the textlab folder.

From log.txt, extract only the lines containing "WARNING" and save them to warnings.txt.
Calculate the total count of "ERROR" and "WARNING" combined. (Hint: write the pattern as "ERROR|WARNING" to search both at once — the | inside a pattern means "or." Try it!)
Save the combined results of both kinds to incidents.txt. (The UTF-8 encoding habit!)
Verify the report: use Get-Content incidents.txt -TotalCount 3 to check that the first 3 lines saved correctly.
Think about it: if the real server log were 100,000 lines, which of today’s commands would you type first? Organize the order out loud.

Exercises

Exercise 1. How many lines in log.txt contain "INFO"? (Out of 50 lines: 7 ERROR, 9 WARNING)

Exercise 2. What’s the result count of Select-String -Path log.txt -Pattern "ERROR|WARNING"? What does the | inside the pattern mean?

Exercise 3. You opened a text file with Korean text downloaded from the internet, and it shows garbled like 비?번호. Give one cause and one remedy.

Exercise 4. In the results of Select-String -Path *.txt -Pattern "error", how do you read log.txt:7:2026-09-08...?

5. Model Answers & Completion Criteria

Mission Walkthrough

# 1
Select-String -Path log.txt -Pattern "WARNING" | Out-File warnings.txt -Encoding utf8
# 2 — Count : 16 (ERROR 7 + WARNING 9)
Select-String -Path log.txt -Pattern "ERROR|WARNING" | Measure-Object
# 3
Select-String -Path log.txt -Pattern "ERROR|WARNING" | Out-File incidents.txt -Encoding utf8
# 4
Get-Content incidents.txt -TotalCount 3

Model answer for step 5’s think-about-it: "don’t read everything" is the starting point. ① Grasp the format first with Get-Content -TotalCount, ② narrow down suspicious patterns ("failed", "ERROR") with Select-String, ③ measure the scale with Measure-Object, ④ record the results with Out-File. Narrow → count → record — this order works exactly the same at 100,000 lines.

Exercise Answers

Answer 1. 50 − 7 − 9 = 34 lines. Verify by command: Select-String -Path log.txt -Pattern "INFO" | Measure-Object → Count 34.

Answer 2. 16 hits (ERROR 7 + WARNING 9). The | inside a pattern is a regular-expression symbol meaning "or" — it searches for "lines containing ERROR or WARNING" in one pass.

Answer 3. Cause: the encoding (table) the file was saved with differs from the encoding of the program reading it. Remedy: specify the table when reading — find the table that reads correctly by varying -Encoding, as in Get-Content file -Encoding UTF8.

Answer 4. filename:linenumber:content — "found in line 7 of the file log.txt, and here’s what that line says." When investigating many files at once, seeing at a glance which file and where — that’s the power of this format.

Completion Checklist

[ ] I can read a file with Get-Content and view just the beginning with -TotalCount
[ ] I can pick out only lines containing a given word with Select-String
[ ] I can read the filename:linenumber:content format
[ ] I can count hits with Measure-Object
[ ] I can save results with Out-File and verify them
[ ] I know the cause of garbled text (encoding mismatch) and the response (-Encoding)
[ ] I completed mission steps 1–5

6. Common Pitfalls & Fixes

Wall 1. Text shows up garbled as □□ or strange characters

It’s an encoding mismatch. The table that made the file and the table reading it differ. Alternate between Get-Content file -Encoding UTF8 and -Encoding Default. If one of them reads correctly, you’ve discovered the file’s encoding. (This trial and error is itself the professional workflow.)

Wall 2. Select-String results include the result file I saved earlier

If a previously saved result file (like errors.txt) sits in the search folder, it becomes a search target too and inflates your results (that’s exactly why errors.txt got caught in section 3-7’s output). Get into the habit of saving investigation results to a different folder.

Wall 3. I saved with Out-File and the non-English text is garbled

Out-File‘s default encoding also varies by environment. Making Out-File file -Encoding utf8 a habit solves most cases.

Wall 4. The word is definitely there but it wasn’t caught

① Check the spelling. ② If you attached -CaseSensitive, the case must match exactly (error ≠ ERROR). ③ Sometimes the file’s encoding is unusual and the search fails — read it with -Encoding specified and pipe it along: Get-Content file -Encoding UTF8 | Select-String "word".

Wall 5. The result is so long the beginning got cut off

Save with | Out-File and view the file, or combine parts from Step 3: ... | Select-Object -First 20. The longer the result, the more the "trimming" habit shines.

7. Summary

Commands You Learned Today

Command	What it does	Example
`Get-Content`	Read a file	`Get-Content log.txt -TotalCount 10`
`Select-String`	Pick out lines containing a pattern	`Select-String -Path log.txt -Pattern "error"`
`Measure-Object`	Count	`... \| Measure-Object`
`Out-File`	Save results to a file	`... \| Out-File result.txt -Encoding utf8`

Key Concepts

Concept	One-line summary
Text file = a sequence of lines	The tools work "line by line"
Encoding	The character↔number conversion table. Garbled text = table mismatch
`-Encoding`	Specify the read/write table (make UTF-8 a habit)
Result format	`filename:linenumber:content` — investigative clues

Instincts That Matter More Than Commands

"Don’t read everything. Decide your extraction conditions first." Even a 100,000-line log becomes a story once you narrow it to "only failed logins" or "only from that IP." And always leave results in a file — an investigation becomes a report only when it’s recorded.

Two nice-to-knows: the -Pattern slot in Select-String actually accepts a pattern language called regular expressions (regex) — the final boss of text processing, which you’ll formally learn in Level 1. And you’ll use this same skill almost unchanged on Linux. Only the name changes to grep — the mindset of "hooking lines by pattern" is a lingua franca across every operating system.

Once every box is checked, Step 4 is complete. Click the checkbox in the sidebar to save your progress.