InfoSec 3001 — Information Security from the Offensive Perspective

Step 145. Directory Busting & Information Exposure — The “If They Don’t Know the Address, It’s Safe” Fallacy

Step 145. Directory Busting & Information Exposure — The "If They Don’t Know the Address, It’s Safe" Fallacy

Level 2 — Introduction to Security and Attack Skill Basics | Difficulty ★★☆☆☆ | Estimated time: 3 hours

Prerequisites: Step 121 (hidden path discovery) and Step 144 (file inclusion) complete. You can read status codes 200/301/403/404.

  • What you need: Python 3 (local web server practice), gobuster on Kali (lab practice), a text editor
  • ⚠️ All exercises in this chapter are for your own lab and legal platforms only. Applying them to unauthorized systems is a crime.

Developers often believe this: "If they don’t know the address, they can’t find it." But attackers knock on everything with a dictionary — the very technique we built ourselves in Step 121. Today we look at what that discovery actually catches. A .git repository uploaded wholesale to the web, a backup file with the password intact, a robots.txt that says "don’t look" and thereby becomes a map. Today we build a local server that deliberately exposes these things, and loot them one by one through an attacker’s eyes (this book was measured on 2026-09-09).

1. Learning Objectives

By the end of this chapter, you will be able to:

  • Explain the principle of forced browsing and the basic usage of gobuster
  • Confirm firsthand why .git folder exposure means "the entire source leaks"
  • Explain the danger of backup files (.bak, ~) and meta files (.DS_Store)
  • Use the fact that even a 403 response carries the information "it exists" in discovery
  • Organize the TOP 5 information-exposure mistakes attackers look for first

2. Background Knowledge — Today’s Tools and Concepts

Today’s Tools at a Glance

Category Details
Language/environment Python 3 (http.server — local reproduction) / Kali gobuster (lab)
Today’s commands gobuster dir -u address -w wordlist -x php,txt,bak, curl
Concepts needed Forced browsing, reading status codes, .git structure, directory listing
Today’s artifact An exposure-mistakes TOP 5 document + a discovery record table

2-1. Forced Browsing — Knocking Where There Are No Links

The field name of the directory busting you learned in Step 121 is forced browsing. You append each dictionary word to the path, send a request, and read the status code. gobuster is a tool that does this with tens of thousands of words in parallel:

gobuster dir -u http://target/ -w /usr/share/wordlists/dirb/common.txt -x php,txt,bak

-x combines extensions too, knocking on config.php, config.txt, and config.bak together. It is the core option for backup-file discovery.

2-2. .git Exposure — The Accident of a Whole Repository on the Web

Accidents where git clone or a deploy script leaves a .git folder in the web root are genuinely common. .git holds the source’s entire history. When you find a server that answers /.git/config, the attacker uses a tool like GitToolsDumper to download the whole repository and restore the source. Even passwords that were "committed for now and deleted later" survive in the commit history. Today we expose it ourselves and confirm what leaks.

2-3. Backup Files and Meta Files — The Byproducts of Development

These are the traces editors and deploy tools leave behind.

  • config.php.bak, index.php~: the server does not execute .bak as PHP and serves it as text — the source and DB password, verbatim.
  • .DS_Store: a meta file macOS creates in every folder. Inside is the list of file names in that folder — a map for learning file names that have no links.
  • robots.txt: guidance telling search engines "don’t crawl," but to an attacker it is "the list of hidden paths right here."

2-4. The Informational Value of 403

Just as 301 effectively meant "it exists" in Step 121, 403 (Forbidden) is also a signal that "it exists." A nonexistent path returns 404 — so a 403 means the path exists but an access rule is blocking it. When you organize discovery results, 403 is not a discard value; collect it in a separate column. It becomes the starting point for studying ways around the defense rules (header manipulation, path mangling).

3. Follow Along

3-1. Target of the Simulation — A Website Exposing Its Development Traces

We build a site that looks like an ordinary company homepage on the outside but is riddled with development artifacts inside. Python’s standard library alone does it.

Input (building the exposed site)

mkdir -p exposed_site/.git/refs/heads exposed_site/admin
cd exposed_site
echo "<h1>Our Company Homepage (Lab)</h1>" > index.html
# Things the developer left behind by mistake
echo "DB_PASSWORD = 'sup3r-secret!'" > app.py
echo "<?php $db_pass = 'backup-pass-123'; ?>" > config.php.bak
printf '[remote "origin"]nturl = https://dev:hunter2@github.com/company/secret-repo.gitn' > .git/config
echo "ref: refs/heads/main" > .git/HEAD
echo "9f2ab1c7d4e5a0b3c8d1e2f3a4b5c6d7e8f9a0b1" > .git/refs/heads/main
echo "User-agent: *
Disallow: /admin/
Disallow: /.git/" > robots.txt
python -m http.server 8082 --bind 127.0.0.1

How to read it: the homepage has not a single link to these files. But you will soon confirm that "there is no link" and "it cannot be reached" are completely different statements.

3-2. .git Exposure — The Repository’s Door Is Open

The first thing an attacker checks is the existence of .git itself.

Input

curl http://127.0.0.1:8082/.git/config
curl http://127.0.0.1:8082/.git/HEAD
curl http://127.0.0.1:8082/.git/refs/heads/main

Output (measured 2026-09-09):

GET /.git/config -> 200
[remote "origin"]
	url = https://dev:hunter2@github.com/company/secret-repo.git

GET /.git/HEAD -> 200
ref: refs/heads/main

GET /.git/refs/heads/main -> 200
9f2ab1c7d4e5a0b3c8d1e2f3a4b5c6d7e8f9a0b1

How to read it: count the damage in three lines. From config, the remote repository URL and a credential-bearing URL (dev:hunter2@) leaked. From HEAD, the current branch; from refs/heads/main, the latest commit hash. At a real incident site, GitToolsDumper steps in here and restores the entire repository (every source file and the commit history) — .git exposure is not "one file leaked" but "the whole project leaked."

3-3. Reading Backup Files and Source Directly

Input

curl http://127.0.0.1:8082/config.php.bak
curl http://127.0.0.1:8082/app.py
curl http://127.0.0.1:8082/robots.txt

Output (measured 2026-09-09):

GET /config.php.bak -> 200
<?php $db_pass = 'backup-pass-123'; ?>

GET /app.py -> 200
DB_PASSWORD = 'sup3r-secret!'
flag{source_code_leak}

GET /robots.txt -> 200
User-agent: *
Disallow: /admin/
Disallow: /.git/

How to read it: the server does not execute a .bak file and serves the original text — which is why the PHP source is visible as-is. And look at robots.txt. The Disallow list that says "don’t look" is a discovery priority list for an attacker. In fact, thanks to this file, we got the existence of /admin/ and /.git/ confirmed for us.

3-4. Directory Listing — A Server Whose Folders Are Transparent

What happens when you request a folder with no index.html? Depending on configuration, the server shows the entire list of the folder’s contents.

Input

curl http://127.0.0.1:8082/.git/refs/heads/

Output (measured 2026-09-09):

<title>Directory listing for /.git/refs/heads/</title>
<h1>Directory listing for /.git/refs/heads/</h1>
<ul>
<li><a href="main">main</a></li>
</ul>

How to read it: under the title "Directory listing for …", the files inside the folder appeared as links. Apache’s Options Indexes and some development servers turn this behavior on by default. As a defender, turning off this listing is a baseline setting — without knowing file names, an attacker must knock with a dictionary, and a listing removes that labor.

3-5. Scanning the Lab with gobuster (lab practice, output example)

Do this on your Kali against a lab target such as DVWA. The output is an output example.

Input

gobuster dir -u http://DVWA-address/ -w /usr/share/wordlists/dirb/common.txt -x php,txt,bak

/.git                 (Status: 301)
/config               (Status: 301)
/docs                 (Status: 301)
/login.php            (Status: 200)
/robots.txt           (Status: 200)
/setup.php            (Status: 200)
/config.inc.php.bak   (Status: 200)   ← the backup file -x bak caught

How to read it: the habit of reading the Status column is the core. 301 means the directory exists, 200 means the file exists, 403 means "exists but forbidden." common.txt has a few thousand entries, so it finishes quickly in a lab. Big lists slow a lab server down — a small list is plenty while you’re learning the principle.

4. Missions & Exercises

Mission — Hunting Information Exposure and the TOP 5 Document

  1. Build the exposed site from 3-1, and first confirm that the homepage source (HTML) contains no links to .git, .bak, or robots.txt
  2. With no links, using addresses alone, read /.git/config, /config.php.bak, and /app.py, and record at least three leaked secrets (passwords, flags)
  3. Interpret robots.txt’s Disallow list as "the attacker’s discovery priority list" and write that in your notes
  4. Scan the lab (DVWA) with gobuster and record the 200/301/403 results classified by status code
  5. Write the "TOP 5 exposure mistakes attackers look for first" — with one line each on "what leaks"

Exercises

Exercise 1. Give two reasons why exposing a .git folder is far more dangerous than exposing one ordinary file.

Exercise 2. Why is the original text of config.php.bak visible as-is? Explain by comparing it with the same file having the .php extension.

Exercise 3. Explain why robots.txt actually helps attackers, and how robots.txt should be written instead.

Exercise 4. Explain why a 403 response in discovery results should be recorded as a "finding."

5. Model Answers & Completion Criteria

Mission Model Answer

An example leak record (measured 2026-09-09): from .git/config, a credential-bearing remote URL (dev:hunter2@...); from config.php.bak, a DB password; from app.py, a password and a flag — all three obtained with no links, by guessing addresses alone.

An example TOP 5 document:

1. .git/ exposure     — the entire source + old passwords in commit history
2. .bak / ~ backups   — unexecuted original source, DB passwords
3. robots.txt         — the list of hidden paths (Disallow = a map)
4. Directory listing  — every file name in the folder
5. admin/, setup.php  — leftover admin/install pages

How to verify: ① is there a record of confirming the absence of links first? ② did you write the leaked secrets as actual values? ③ are the gobuster results classified by status code? ④ does the TOP 5 carry "what leaks" for each item?

Exercise Answers

Answer 1. First, .git holds not just the current source but the entire commit history, so restoration tools can download the whole project. Second, the history retains passwords and keys that were deleted later, so even secrets absent from the current source get looted. Credentials can also leak directly, like the remote URL in config.

Answer 2. A web server decides by extension whether to "execute or serve as text." .php is executed by the PHP engine and only the result goes out, but .bak is an unknown extension, so the original text is served as-is. That is why backup files are a source-leak channel.

Answer 3. robots.txt is a public file anyone can read, so its Disallow list becomes exactly "the list of paths you want hidden." The response: use robots.txt only for search-engine control, and don’t write sensitive paths in it — block them with authentication and access control. Hiding is not defense.

Answer 4. 404 means "it doesn’t exist," but 403 means "it exists but is forbidden" — the path’s existence itself is confirmed. A confirmed path becomes a candidate for later bypass attempts (header manipulation, path mangling, other HTTP methods), so it should be collected separately from 404s in the discovery record.

Completion Criteria Checklist

  • [ ] I built the exposed site and read .git/config by address alone, with no links
  • [ ] I confirmed original source leaking from a backup file (.bak)
  • [ ] I saw a directory-listing screen ("Directory listing for …") myself
  • [ ] I interpreted robots.txt’s Disallow through an attacker’s eyes
  • [ ] I recorded gobuster results classified into 200/301/403
  • [ ] I wrote the exposure-mistakes TOP 5 document

6. Common Pitfalls & Fixes

Wall 1. I requested /.git/config and got a 404

Symptom: the .git you clearly created is reported missing.
Cause: the folder where you started the server is the web root. You may have started the server above exposed_site, or there may be a path typo.
Fix: first check that the homepage opens with curl http://127.0.0.1:8082/, and inspect where the server is running. Some servers block paths starting with . by default — which is itself a defense setting.

Wall 2. gobuster is too slow

Symptom: you ran a big wordlist and the lab server can’t hold up.
Cause: tens of thousands of requests are a load even on a light lab server.
Fix: in a lab, common.txt (a few thousand) is enough. Learn the principle with a small dictionary, and try adjusting the thread count with the -t option.

Wall 3. Lots of 301s and no 200s

Symptom: the results are all 301 redirects.
Cause: the directory candidates exist, but the relocation notice appends a / and the body may not follow.
Fix: a 301 is also a "directory exists" finding. Follow it with curl -L, or check gobuster’s -f (append slash) option.

Wall 4. I found a backup file but it only downloads, not displays

Symptom: the browser saves the .bak instead of showing it.
Cause: a Content-Type issue only — the leak itself already happened.
Fix: fetch it with curl and the original text is visible. Distinguish "doesn’t appear on screen" from "safe."

Wall 5. The doubt: "But we made this on purpose"

Symptom: the exercise works so well that its real-world relevance feels doubtful.
Cause: look at real incident statistics — .git exposure and backup files are still staple bug-bounty report types today.
Fix: the point of the exercise is grasping the structure of "why it leaks." Becoming the defender and blocking the 3-1 site’s exposures one by one (turning off listing, adding .git deny rules) is the best review.

7. Summary

Today’s Concepts

Concept One-line explanation
Forced browsing Knocking on paths with a dictionary to find link-less resources
.git exposure The accident of a repository folder published on the web — source + history leak wholesale
Backup files (.bak, ~) Source copies that leak verbatim because they aren’t executed
.DS_Store A macOS meta file — leaks the list of file names in a folder
robots.txt A public "list of hidden paths" — not defense, a map
The value of 403 "Exists but forbidden" — an existence-confirmation signal
Directory listing A setting that shows the whole contents of folders with no index file

Today’s Commands

Command What it does
gobuster dir -u address -w common.txt Dictionary discovery of hidden paths
-x php,txt,bak Extension combinations to find even backup files
curl address/.git/config Check for repository exposure
curl address/config.php.bak Check a backup file’s original text
curl -L address/path Follow a 301 to the final response

An Instinct More Important Than Commands

Everything you saw leak today has one thing in common: "what wasn’t cleaned up." .git rode along in the deploy, the pre-edit copy stayed as .bak, and the admin page under development was left as-is. A truth that comes before attack technique — attackers don’t go after what you deployed; they go after what you left behind.

The defender’s checklist is today’s mission TOP 5 flipped over. Clean up hidden files before deploy, block . paths and listings at the web server, and don’t write sensitive paths in robots.txt. In front of flashy attack tools, humble settings like these are the real castle wall.

Once every box is checked, Step 145 is complete.

← 이전 스텝
다음 스텝 →