Step 4. Text Processing — Finding a Needle in the Logs

Level 0 — Understanding How to Operate a Computer and How It’s Structured | Difficulty ★★☆☆☆ | Estimated time: 2–3 hours

Prerequisites: Step 3 complete (the pipe). We’ll work in Windows PowerShell.

• What you need: a Windows PC, PowerShell.
• Safety: create practice files and reports only in a new dedicated folder. Output files with the same name may be overwritten, so do not use real logs or personal documents. You’ll make the practice files yourself — we’ll build a 50-line fake log and practice investigating inside it.
• Caution: you may run into an incident where non-English text looks garbled. Don’t panic — today you’ll conquer even its cause (encoding).

This section’s star is “logs.” Depending on logging configuration, login attempts, application errors, and connections can leave records. Logs can use text, Windows event-log formats, databases, and other storage. Today we use only synthetic text logs. An incident investigation starts with a single line like “show me only the lines containing ERROR,” and as tens of thousands of lines shrink to a few dozen, a story starts to emerge. The “pull the lines you want out of text” skill you’ll learn today is the alpha and omega of security analysis.

1. Learning Objectives

By the end of this unit, you can:

• Read files with Get-Content, and peek at just the beginning of long files
• Pick out only the lines containing a given word with Select-String
• Count result hits and save results to a new file to create an “investigation document”
• Investigate many files at once using the filename:linenumber:content format
• Explain the cause of garbled non-English text (encoding mismatch) and respond with -Encoding

2. Background Knowledge — Today’s Tools and Concepts

This section’s Tools at a Glance

Translated into the language of a crime scene, this section’s commands look like this: Select-String is tool #1 of incident investigation — “hooking only the suspicious lines out of tens of thousands of log lines.” Measure-Object is the tool that produces sentences like “300 failed logins between 2 and 4 a.m.” Out-File -Encoding utf8 is a tool for saving readable analysis results. A security professional’s daily life involves far more time wrestling with logs using commands like this section’s than wielding fancy hacking tools.

2-1. The Structure of a Text File — “Lines” Are the Unit

Open up a text file and it’s ultimately a sequence of lines. And all of this section’s tools work “line by line”:

• Get-Content: reads a file as a list of lines (one line = one object)
• Select-String: examines each line and picks out only the ones matching the condition

In Step 3 you learned “list → filter,” right? Text processing is exactly the same thinking. Only the material changed from processes to “lines.”

2-2. Encoding — The Conversion Table Between Characters and Numbers

Computers can’t store characters. They can only store numbers. So we need a “character ↔ number” conversion table, and that table is an encoding.

The problem is there are several kinds of tables:

One cause of garbled text: the file was saved as numbers using table A (say, UTF-8), but the reading program interprets it with table B (say, CP949). Same numbers, different table — so the characters come out bizarre. It’s like trying to open a lock with the wrong key.

The fix: when reading, explicitly say “read with this table.” We’ll practice it today.

3. Follow Along

Before you begin. The log is a synthetic fixture. Counts describe that file, not the security of your computer. Encoding examples deliberately include non-ASCII characters.

3-1. Preparing the Crime Scene — Making a Fake Log

As in Step 2, create a uniquely named folder and a reports subfolder for outputs. Record the path held in $textlab. All account and IP values are fake log text; no command connects to them:

$textlab = Join-Path $HOME ('infosec-step04-' + [guid]::NewGuid().ToString('N'))
New-Item -ItemType Directory -Path $textlab -ErrorAction Stop
Set-Location -LiteralPath $textlab
New-Item -ItemType Directory reports
pwd

Now let’s make a 50-line log. You could use Notepad, but since we’re practicing the CLI, we’ll build it in PowerShell. Copy the block below in one piece and paste it into PowerShell (multiple lines will run at once):

1..50 | ForEach-Object {
  $m = "{0:D2}" -f $_
  if ($_ % 7 -eq 0) { "2026-09-08 02:$m ERROR failed login: admin from 10.9.9.9" }
  elseif ($_ % 5 -eq 0) { "2026-09-08 02:$m WARNING disk usage high" }
  else { "2026-09-08 02:$m INFO normal operation" }
} | Set-Content log.txt -Encoding UTF8

The loop syntax is covered in Step 8. The final Set-Content replaces this practice log.txt with 50 lines, so re-running the generation block does not append another 50. All timestamps are in hour 02 to keep row order chronological. All you need to know: “a 50-line file where every 7th line has ERROR, every 5th line (that isn’t a multiple of 7) has WARNING, and the rest have INFO.”

Check:

Get-Content log.txt -Encoding UTF8 -TotalCount 8

2026-09-08 02:01 INFO normal operation
2026-09-08 02:02 INFO normal operation
2026-09-08 02:03 INFO normal operation
2026-09-08 02:04 INFO normal operation
2026-09-08 02:05 WARNING disk usage high
2026-09-08 02:06 INFO normal operation
2026-09-08 02:07 ERROR failed login: admin from 10.9.9.9
2026-09-08 02:08 INFO normal operation

• -TotalCount 8 — “not all of it, just the first 8 lines.” An option for peeking at long files.

3-2. Reading Everything and Why That’s a Problem — Get-Content

Let’s read the whole thing:

Get-Content log.txt -Encoding UTF8

50 lines whoosh past. It’s only 50 lines now, but real logs have 50,000. Feel in your bones that “reading everything with your eyes” is not an option. That’s why we need the next machine.

3-3. Investigation Begins — Select-String

Get-Content log.txt -Encoding UTF8 | Select-String "error"

2026-09-08 02:07 ERROR failed login: admin from 10.9.9.9
2026-09-08 02:14 ERROR failed login: admin from 10.9.9.9
...

Only the lines containing “error” were picked out. Same role as Step 3’s Where-Object, but simpler because it’s dedicated to text search.

Wait — we saved it as “ERROR” (uppercase) earlier, so why did searching for “error” (lowercase) catch it? Select-String is case-insensitive by default. (If you want case sensitivity, add the -CaseSensitive option.)

It can be even shorter — Select-String accepts a file path directly:

Select-String -Path log.txt -Pattern "error" -Encoding UTF8

• -Path — the file to search
• -Pattern — a regular expression by default. Add -SimpleMatch to search literal text, including characters such as a dot.

3-4. How Many? — Counting

When gauging the scale of an attack, counts matter:

Select-String -Path log.txt -Pattern "error" -Encoding UTF8 | Measure-Object

Count    : 7

Count here measures matching lines, not every occurrence of a word or the number of unique attacks. ERROR went into every multiple-of-7 line, so 7 lines match. “7 ERRORs” — the Measure-Object you learned in Step 3 works exactly the same here. Assembly thinking, remember?

Predict first: how many lines will Select-String -Path log.txt -Pattern "WARNING" return? Predict, then run it.
(Answer: 9 lines. There are 10 multiples of 5 (5, 10, …, 50), but 35 is also a multiple of 7, so it fell into the ERROR slot. Recall the generation rules and verify — doubting “what happens when conditions overlap?” is an analyst’s instinct.)

3-5. Save Analysis Results — Out-File

Search results are MatchInfo objects. Sending them directly to Out-File can include filename/line-number prefixes and display spacing. Use Select-Object -ExpandProperty Line to save just the original line text, one result per output line:

Select-String -Path log.txt -Pattern "error" -Encoding UTF8 |
  Select-Object -ExpandProperty Line |
  Out-File .\reports\errors.txt -Encoding UTF8
Get-Content .\reports\errors.txt -Encoding UTF8

2026-09-08 02:07 ERROR failed login: admin from 10.9.9.9
...

It was written to reports\errors.txt instead of the screen, and we read it back to verify. This flow:

search (Select-String) → save (Out-File) → verify (Get-Content)

reports\errors.txt must contain seven original log lines. Re-running this export overwrites that practice report. This plain-text export omits source-file paths and line numbers; a real investigation must preserve provenance and collection time separately.

3-6. Read the Same Bytes with Different Encodings

The fixture is UTF-8 without a byte-order mark. Its accented word, café, makes an encoding mismatch visible. Compare explicit UTF-8 reading with the default for your PowerShell version; a different display is not a different original file.

$utf8NoBom = [Text.UTF8Encoding]::new($false)
[IO.File]::WriteAllText((Join-Path $textlab 'unicode.txt'), 'Reset the café password', $utf8NoBom)
Get-Content unicode.txt -Encoding ASCII
Get-Content unicode.txt -Encoding UTF8

ASCII cannot represent this non-ASCII text correctly; the UTF8 read must return the original sentence. Changing the read encoding does not convert or repair the file itself. If a file was already overwritten with question marks, an encoding option cannot reconstruct the lost characters.

Match the write and read encodings for new files. For existing files, check the producing application, format documentation, and BOM before reading a copy. UTF8 includes a BOM in 5.1 but normally omits it in 7. -Encoding Default is not a shared remedy supported the same way by both versions.

3-7. Investigating Many Files at Once

Here’s where Select-String’s real power shows:

Set-Content other.txt "2026-09-08 05:00 ERROR sample service failure" -Encoding UTF8
Select-String -Path .\*.txt -Pattern "error" -Encoding UTF8

We added one fake ERROR to other.txt. Searching the current folder’s .txt files now finds seven lines from log.txt and one from other.txt: eight in total. The reports subfolder is not searched by this nonrecursive pattern. Example output format:

log.txt:7:2026-09-08 02:07 ERROR failed login: admin from 10.9.9.9
log.txt:14:2026-09-08 02:14 ERROR failed login: admin from 10.9.9.9
...

How to read it: filename:linenumber:content. Thanks to this feature, you can investigate hundreds of log files in one shot.

4. Missions & Exercises

Mission — A Day in the Life of a Log Analyst

Incident: you’ve been ordered to “find suspicious activity in our server logs and submit a report.” Work in the new $textlab folder. This mission analyzes only log.txt, excluding other files. Export only the original line text and specify UTF8.

1. From log.txt, extract only the lines containing “WARNING” and save them to reports\warnings.txt.
2. Calculate the total count of “ERROR” and “WARNING” combined. (Hint: write the pattern as "ERROR|WARNING" to search both at once — the | inside a pattern means “or.” Try it!)
3. Save the combined results of both kinds to reports\incidents.txt. (The UTF-8 encoding habit!)
4. Verify the report: use Get-Content .\reports\incidents.txt -Encoding UTF8 -TotalCount 3 to check that the first 3 lines saved correctly.
5. Think about it: if the real server log were 100,000 lines, which of this section’s commands would you type first? Organize the order out loud.

Exercises

Exercise 1. How many lines in log.txt contain “INFO”? (Out of 50 lines: 7 ERROR, 9 WARNING)

Exercise 2. What’s the result count of Select-String -Path log.txt -Pattern "ERROR|WARNING"? What does the | inside the pattern mean?

Exercise 3. You opened a text file with non-ASCII text downloaded from the internet, and it shows garbled like caf?. Give one cause and one remedy.

Exercise 4. In the results of Select-String -Path *.txt -Pattern "error", how do you read log.txt:7:2026-09-08...?

# 1
Select-String -Path log.txt -Pattern "WARNING" -Encoding UTF8 |
  Select-Object -ExpandProperty Line |
  Out-File .\reports\warnings.txt -Encoding UTF8
# 2: Count = 16
Select-String -Path log.txt -Pattern "ERROR|WARNING" -Encoding UTF8 | Measure-Object
# 3
Select-String -Path log.txt -Pattern "ERROR|WARNING" -Encoding UTF8 |
  Select-Object -ExpandProperty Line |
  Out-File .\reports\incidents.txt -Encoding UTF8
# 4
Get-Content .\reports\incidents.txt -Encoding UTF8 -TotalCount 3

6. Common Pitfalls & Fixes

Wall 1. Text shows up garbled as □□ or strange characters

Check encoding, font support, and whether the original is damaged. Preserve the original and consult the producing application’s encoding settings. Changing the read option does not convert the source. Already lost characters may require a fresh original or backup.

Wall 2. Select-String results include the result file I saved earlier

If a previously saved result file (like errors.txt) sits in the search folder, it becomes a search target too and inflates your results (which is why this lesson stores outputs in the reports subfolder). Get into the habit of saving investigation results to a different folder.

Wall 3. I saved with Out-File and the non-English text is garbled

Out-File‘s default encoding also varies by environment. Save new files with Out-File file -Encoding utf8 and read them with the same encoding. Re-saving already misdecoded text as UTF8 does not restore the original characters.

Wall 4. The word is definitely there but it wasn’t caught

① Check the spelling. ② If you attached -CaseSensitive, the case must match exactly (error ≠ ERROR). ③ Sometimes the file’s encoding is unusual and the search fails — read it with -Encoding specified and pipe it along: Get-Content file -Encoding UTF8 | Select-String "word".

Wall 5. The result is so long the beginning got cut off

Save with | Out-File and view the file, or combine parts from Step 3: ... | Select-Object -First 20. The longer the result, the more the “trimming” habit shines.

7. Summary

Commands You Learned Today

Key Concepts

Instincts That Matter More Than Commands

“Don’t read everything. Decide your extraction conditions first.” Even a 100,000-line log becomes a story once you narrow it to “only failed logins” or “only from that IP.” And always leave results in a file — an investigation becomes a report only when it’s recorded.

Two nice-to-knows: the -Pattern slot in Select-String actually accepts a pattern language called regular expressions (regex) — the final boss of text processing, which you’ll formally learn in Level 1. And you’ll use this same skill almost unchanged on Linux. grep uses a similar approach, but options, regex dialects, and output types differ; it is not a direct command-name substitution. The mindset of “hooking lines by pattern” is a lingua franca across every operating system.

Once every box is checked, Step 4 is complete.

Practice verification record

Historical record. The dated record below describes the earlier lesson version. The current instructions include subsequent editorial corrections; that record does not certify every revised example.

2026-09-10 · Ran the synthetic-log generation and report workflow in order on Windows 11 with Windows PowerShell 5.1.26100.9444 and PowerShell 7.6.5. Verified 50 log lines: 7 ERROR, 9 WARNING, and 34 INFO, plus a 16-line combined report. Also checked regeneration, exclusion of report files from source searches, correct Korean UTF8 reading, and incorrect ASCII decoding. No production logs or personal data were used.

References: Microsoft Select-String · Out-File · Character encoding