Step 73. Understanding HTTP Completely — The Web’s Rules of Conversation

Level 1 — Programming and the Computer’s Insides | Difficulty ★★★☆☆ | Estimated time: 4 hours

Prerequisites: Steps 71–72 complete; you know a web page’s structure (HTML) and behavior (JS). You can type commands in a terminal (Git Bash or PowerShell).

What you need: a browser, a terminal, curl (built into Windows 10 and later — check with curl --version), and Python (for the practice server). Today’s conversation partner is not somewhere on the internet — it is a server you launch inside your own computer.
Caution: today’s practice is 100% safe. Every communication happens only inside 127.0.0.1 (the special address that points at your own computer), and we never connect to an external site, not even once.

In the last two chapters we built a page and made it move. But how that page travels from a server to our screen is still a black box. Today we open that box. All web communication runs on one simple rule — the client (the guest) makes a request, and the server (the kitchen) returns a response. And that conversation is entirely text a human can read. Today we eavesdrop on it, speak it ourselves, and interpret the answers.

1. Learning Objectives

By the end of this chapter, you will be able to:

Launch a practice web server on your computer with python -m http.server
Draw the structure of a request (method + path + headers + body) and a response (status code + headers + body)
Read the raw conversation with curl -v, distinguishing > (request) from < (response)
Explain the meaning of status codes 200/301/404/500 and why they are sources of information
Explain the difference between GET and POST in terms of "where the data rides"

2. Background Knowledge — Today’s Tools and Concepts

Today’s Tools at a Glance

Category	Details
Language/environment	HTTP (the web document transfer protocol) — the grammar browsers and servers speak. All practice happens on localhost (127.0.0.1)
Today’s tools	`curl` (the terminal’s all-purpose communication tool), `python -m http.server` (a one-line practice web server), the browser DevTools Network tab
Today’s commands	`curl -v URL`, `curl -I URL`, `curl -A "..." URL`, `curl -X POST -d "..." URL`
Concepts needed	The three parts of request/response (first line, headers, body), methods (GET/POST), status codes, headers (Host, User-Agent, Cookie)
Today’s output	The practice pages in the `webroot/` folder and a notebook of communication observations

2-1. The Anatomy of a Request

A request a browser sends to a server has three parts.

GET /index.html HTTP/1.1
Host: 127.0.0.1:8001
User-Agent: curl/8.11.0

First line (the request line): in the order method path version. The method is "what do you want to do" — GET (give it to me), POST (process this data), and so on. The path is "which page," and the version is "which grammar am I speaking."
Headers: the list of Name: value pairs continuing from the second line. They are the markings on the outside of the envelope. Host (which site), User-Agent (who sent this), Cookie (the ID slip) all go here.
Body: the contents attached only when data rides along, as with POST. A GET usually has no body.

2-2. The Anatomy of a Response

The server’s answer has the same structure.

HTTP/1.0 200 OK
Content-type: text/html
Content-Length: 270

<!DOCTYPE html>...

First line (the status line): the version, the status code, and a short description.
Headers: Content-Type (the kind of contents), Content-Length (the length), and so on.
Body: the actual web page (HTML) or data (JSON). The HTML we built in Step 71 is exactly what rides home in this body.

2-3. Status Codes — Reading Between the Three Digits

A status code is the server’s one-word answer. The hundreds digit sets the general meaning.

Code	Meaning	Analogy
200	Success	"Here you are"
301/302	Redirect (moved)	"That one moved over there"
403	Forbidden	"You may not enter"
404	Not found	"There’s no such page"
500	Server error	"An accident happened in the kitchen"

In security study, these numbers are intelligence. A 403 means "something is there but it’s blocked," which is distinguishable from 404, so it becomes a clue when probing for hidden pages. A 500 means "the server broke because of my input," and that can become a lead for an attack.

2-4. GET and POST — The Postcard and the Parcel

The difference between the two methods in one line: GET carries its data on the address (URL); POST carries it in the body.

GET: attached after the address as ?name=value, like .../search?q=hammer. Like a postcard, it is visible to anyone passing by, and it remains in both the browser history and the server logs. That’s why it’s used for searching and looking things up.
POST: the data goes into the body and is invisible in the address bar. It travels wrapped like a parcel. It’s used for requests that "cause change on the server," like logging in or writing a post.

No misconceptions allowed: POST is not encrypted. It merely isn’t visible in the address bar; open the envelope (intercept it) and the contents are plain. Real protection is the job of HTTPS (encrypted HTTP).

3. Follow Along

3-1. Launching the Practice Server — Your Computer Becomes a Server

First, let’s create the experiment partner. Make a folder called webroot in your working folder and save index.html inside it (the craft from Step 71).

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <title>Practice Server Home</title>
</head>
<body>
  <h1>Our home practice server</h1>
  <p>This page lives only at 127.0.0.1.</p>
</body>
</html>

Then, in the terminal, move into that folder and switch the server on.

Input

cd webroot
python -m http.server 8001

Screen example (may differ by version and environment):

Serving HTTP on :: port 8001 (http://[::]:8001/) ...

How to read it: python -m http.server 8001 means "turn on a web server that offers this folder through exit number 8001." 127.0.0.1 is the special address pointing to "this computer itself," and 8001 is the exit number (port) inside it. This window is busy working as a server and won’t take other commands — open one more new terminal window and run the experiments below there. To stop the server, press Ctrl+C in this window.

Why: you can’t experiment freely against someone else’s site, but my server is mine. It’s a perfect practice ground where failure and strange requests cause no problems at all.

3-2. curl -v — Seeing the Conversation Raw

Now let’s send a request directly from the terminal, without a browser. curl is a tool that "communicates on your behalf when you give it an address."

Input (in the new terminal window)

curl -v http://127.0.0.1:8001/index.html

Output (measured 2026-09-09; the progress lines in the middle are omitted):

*   Trying 127.0.0.1:8001...
* Connected to 127.0.0.1 (127.0.0.1) port 8001
> GET /index.html HTTP/1.1
> Host: 127.0.0.1:8001
> User-Agent: curl/8.11.0
> Accept: */*
>
< HTTP/1.0 200 OK
< Server: SimpleHTTP/0.6 Python/3.12.14
< Date: Wed, 09 Sep 2026 04:34:10 GMT
< Content-type: text/html
< Content-Length: 270
<
<!DOCTYPE html>
<html>
... (the rest of the HTML body)

How to read it: lines starting with > are what I sent (the request), lines starting with < are what the server sent (the response), and lines starting with * are connection preparations (safe to ignore). -v is the verbose option — it means "show me all of this envelope’s contents." Notice that the response’s Server: header has the server’s identity (SimpleHTTP/0.6 Python/3.12.14) written by its own hand. What continues at the very bottom with no headers is the response body — the raw HTML.

Why: this is training to see the "raw conversation" the browser doesn’t draw on the screen. This text, not the screen, is the web’s true appearance.

3-3. Status Code Experiment — A Page That Exists and One That Doesn’t

Input

curl -I http://127.0.0.1:8001/index.html
curl -I http://127.0.0.1:8001/no-such-page.html

Output (measured 2026-09-09):

HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.12.14
Content-type: text/html
Content-Length: 270

HTTP/1.0 404 File not found
Server: SimpleHTTP/0.6 Python/3.12.14
Connection: close
Content-Type: text/html;charset=utf-8
Content-Length: 335

How to read it: -I is the option to receive only the headers (a HEAD request). Use it when you want just the envelope’s face, no body. The existing page returns 200; the missing one returns 404 — and this server’s 404 description reads "File not found."

Why: this trains you to see a status code not as "an error message on the screen" but as "a signal machines read." The way a hidden-path-finding tool (a directory scanner) works is exactly this signal reading — poke candidate addresses one by one and sort the 200s from the 404s.

3-4. The Server-Side Log — What Rides on the Address Stays Behind

This time we send data by GET and look at what is visible on the server side.

Input

curl "http://127.0.0.1:8001/index.html?name=admin&level=3"

Now look at the first terminal window, where the server is running. A line of record piles up with every connection (measured 2026-09-09):

::ffff:127.0.0.1 - - [09/Sep/2026 13:34:23] "HEAD /no-such-page.html HTTP/1.1" 404 -
::ffff:127.0.0.1 - - [09/Sep/2026 13:38:38] "GET /index.html?name=admin&level=3 HTTP/1.1" 200 -

How to read it: each line is "who (IP) sent what request when, and what the status code was." Look at the second line: the name=admin&level=3 that rode on the address remains entirely in the server’s record. The server can read and log all data that rides on the address.

Why: that’s why you must never send a password by GET. Data riding on the address stays in the browser’s visit history, in server logs, and in the records of equipment in between. Behind the convention "GET for lookup, POST for change" there are security reasons like this.

3-5. Sending a POST — This Server Doesn’t Understand It

Input

curl -X POST -d "id=admin&pw=1234" http://127.0.0.1:8001/echo

Output (measured 2026-09-09, part of the body):

<h1>Error response</h1>
<p>Error code: 501</p>
<p>Message: Unsupported method ('POST').</p>

It stays in the server log too: code 501, message Unsupported method ('POST')

How to read it: 501 is the status code for "I don’t know that method." This simple file server understands only GET and HEAD and can’t do POST (receiving data). Our id=admin&pw=1234 did ride along in the body, but there was no one on the other end to read it.

Why: a method is an agreement. Even if the client speaks POST, if the server doesn’t know that agreement, the conversation never takes place. For a login form to actually work, you need a server program that unpacks and processes the POST body — building such a server and talking to it yourself is the first exercise once you learn requests.

3-6. Header Disguise — Who Am I?

With curl you can change request headers at will.

Input

curl -v -A "MyFirstBot/1.0" http://127.0.0.1:8001/index.html

Output (measured 2026-09-09, request part only):

> GET /index.html HTTP/1.1
> Host: 127.0.0.1:8001
> User-Agent: MyFirstBot/1.0
> Accept: */*

How to read it: we changed the User-Agent with the -A option, and the "sender" box of the request envelope changed exactly as told. The server does not know who we are by itself — it merely accepts what we wrote in the header.

Why: a header is only "information the client claims," not the truth — this is today’s core lesson. A server that distinguishes browsers by User-Agent, a server that trusts an origin by Referer — all of them can be fooled. Not trusting headers without verification is the starting point of security design.

3-7. The Network Tab — Eavesdropping on the Browser’s Conversation Too

Finally, let’s look at the browser’s conversation.

Input: press F12 → with the Network tab open, type http://127.0.0.1:8001/index.html into the address bar.

Screen example: one line for index.html appears in the list under the tab. Click it, and in the Headers sub-tab you see Request Headers (the headers I sent) and Response Headers (the headers the server sent) side by side; the Response tab holds the returned raw HTML.

How to read it: each line in the list is one request-response pair. The User-Agent the browser sent is the very same box we just imitated with curl — a browser, in the end, is just one client speaking the same grammar.

Why: this closes the loop by confirming, inside the browser too, that the phenomenon of "a page appearing" is the result of request and response. curl and the Network tab — these two windows become your eyes in all web analysis from now on.

4. Missions & Exercises

Mission — An HTTP Observation Report

Write an observation report against our home practice server (a note file http_report.txt is recommended).

Add a second page hello.html to webroot and turn the server on
Request index.html with curl -v and copy down every > line of the request and every < line of the response
Use curl -I to provoke an existing page (200) and a missing page (404) once each, and copy down the status lines
Send curl "http://127.0.0.1:8001/index.html?secret=abcd", then find that line in the server log and copy it down
Send a request with a changed User-Agent using -A, and copy down the changed request line
At the end of the report, one picture: draw in your own words "browser → [request: method + path + headers + body] → server → [response: status code + headers + body] → browser"

Exercises

Q1. State what goes into each of the three parts of a request message (first line, headers, body).

Q2. Explain the difference between GET and POST from two perspectives: "where the data rides" and "where it remains on record."

Q3. You requested some path and got a 403. Compared to 404, what information does this number give you, and why does it become a clue for probing?

Q4. "Sending by POST keeps the data out of the address bar, so a password is safe" — explain where this statement goes wrong.

5. Model Answers & Completion Criteria

Mission Model Answer

The core content the report must contain is as follows (it should all look the same as the values measured on 2026-09-09).

Request > lines: GET /index.html HTTP/1.1, Host: 127.0.0.1:8001, User-Agent: curl/..., Accept: */*
Response < lines: HTTP/1.0 200 OK, Server: SimpleHTTP/0.6 Python/..., Content-type: text/html, Content-Length: ...
Status line of the 404 experiment: HTTP/1.0 404 File not found
Server log line: "GET /index.html?secret=abcd HTTP/1.1" 200 - — the point is that secret remained in the log as-is
Disguise experiment: the request’s User-Agent line changed to the string I chose
The picture: the three parts of the request (first line/headers/body) and of the response must sit on the arrows

How to verify: ① Does every copied record point at 127.0.0.1 rather than an external site? ② Did you confirm that secret=abcd remained in the server log? ③ Does the picture contain all four request elements (method, path, headers, body) and all three response elements (status code, headers, body)? If all three are "yes," it’s complete.

Exercise Solutions

Q1 solution. The first line holds the method, path, and version ("what, where, in which grammar"); the headers hold supplementary information in Name: value form (Host, User-Agent, Cookie, etc.); the body holds the data a request like POST carries. A GET usually has an empty body.

Q2 solution. GET carries its data on the address (?name=value) and POST carries it in the body. So GET data remains, address and all, in the browser history and the server logs (measured in 3-4), while POST data does not remain in address records — it travels in the body. Note, however, that neither has anything to do with encryption.

Q3 solution. 403 means "that path exists but access is forbidden," and 404 means "it does not exist." So a 403 itself becomes the information "there is something hidden," a clue for discovering the existence of protected resources.

Q4 solution. It confuses "not visible" with "protected." The POST body is also transmitted in plaintext, so if someone opens the envelope midway, its contents read just fine. Safety is provided not by where the data rides but by HTTPS (encryption of the communication channel).

Completion Criteria Checklist

[ ] I can launch and stop a practice server with python -m http.server
[ ] I can draw the three parts of a request and a response (first line/headers/body)
[ ] I can read curl -v output distinguishing request (>) from response (<)
[ ] I can state the meanings of status codes 200/301/404/500
[ ] I confirmed that GET data remains in the server log
[ ] I can send a changed User-Agent with -A
[ ] Mission: I completed the HTTP observation report

6. Common Pitfalls & Fixes

Wall 1. The terminal sits there blankly, not taking commands

Symptom: you typed python -m http.server 8001 and that window no longer takes other commands.
Cause: it’s not broken. The server must keep working while it’s on, so it occupies that window.
Fix: leave the server window alone and open a new terminal window to type curl. To stop the server, press Ctrl+C in the server window.

Wall 2. "Address already in use" appears

Symptom: when starting the server, it dies with a message like OSError: [WinError 10048] ....
Cause: that exit number (port) is already in use by another program (or a previous server that never stopped).
Fix: change the port number to another one, like 8002 or 8010. A port is just a chat-room number — use any free number.

Wall 3. curl -v output is so long it’s scary

Symptom: a flood of asterisks (*) and angle brackets makes you want to give up reading.
Cause: -v shows absolutely everything, including connection preparation.
Fix: read only the lines starting with > and <. The asterisk lines are preparation — ignore them. This distinction is today’s core reading skill.

Wall 4. I sent a POST and got a 501 back

Symptom (measured 2026-09-09): <p>Message: Unsupported method ('POST').</p>

Cause: the other server doesn’t understand POST. python -m http.server is a simple file server that knows only GET and HEAD.
Fix: this is not a typo but a matter of the server’s ability. To receive POST you need a server built to process POST — an exercise where we build such a practice server ourselves is coming soon.

Wall 5. It’s a 301 but nothing shows

Symptom: you made a request with curl and only a Location: header came back, with no body.
Cause: browsers automatically follow redirects (moving notices), but curl does not follow them by default.
Fix: add the -L option to follow the notice — like curl -L URL. With -v you can also watch how many steps it took to arrive.

7. Summary

Today’s Concepts

Concept	One-line description
HTTP	The conversation grammar of browsers and servers — a repetition of requests and responses
Request = method + path + headers + body	"What (method) where (path)" + supplementary info (headers) + contents (body)
Response = status code + headers + body	Result signal (status code) + description (headers) + contents (body)
Status code	The server’s one word — 200 success / 301 moved / 403 forbidden / 404 not found / 500 server accident
GET vs POST	Riding on the address (lookup) vs riding in the body (change) — neither is encryption
Header	Information the client claims — the server must not trust it without verification
127.0.0.1	The special address pointing to my own computer — a safe practice ground

Today’s Commands

Command	What it does
`python -m http.server 8001`	Practice server offering the current folder (stop: Ctrl+C)
`curl -v URL`	View the full request and response (> request, < response)
`curl -I URL`	Receive headers only (HEAD request)
`curl -A "string" URL`	User-Agent disguise
`curl -X POST -d "name=value" URL`	Send a POST body
`curl -L URL`	Follow redirects

The Instinct That Matters More Than Commands

Today’s core instinct is that "behind every screen there is always a text conversation." A pretty page, a login — all of it is the result of text going back and forth, beginning with a single method path version line. Once you can read this conversation, every web phenomenon turns into an object of analysis. The security connection: nearly all web attacks are "the craft of tampering with requests" — changing headers, changing paths, changing bodies. What you did by hand with curl today is already the first step of tampering, and it is from this conversation structure that the reason a header like Cookie (the login ID slip) deserves special protection emerges. For the record, the conversation we saw today was plaintext. HTTPS is this whole conversation encrypted; an eavesdropper in the middle sees only ciphertext instead of contents.

Once every box is checked, Step 73 is complete. Click the checkbox in the sidebar to save your progress.