Step 337. Win/Loss Analysis and Designing the Next Challenge — Win or Lose, It’s All Data

Level 4 — Professional | Difficulty ★★★☆☆ | Estimated time: 2 days (half a day for the per-problem analysis table + half a day for the team operations meeting + 1 day for the improvement plan)

Prerequisites: Step 336’s championship bid completed — the results-record template (rank, per-problem results, operations evaluation, win-factor separation) must already be filled in. This is a theory-focused concepts chapter.

What you need: the competition results record (Step 336’s deliverable), the winning team’s Write-up (as it gets published), a team channel, and a Python environment (for the aggregation script). The results-aggregation script is a measured tool you run on your own records; every competition scene and scoreboard quote is a screen example.
Caution: this chapter’s danger is not technique — it’s emotion. A post-loss meeting requires a pre-agreed principle: "we analyze the system, not the person."
⚠️ All exercises in this chapter are for your own lab and legal platforms only. Applying them to unauthorized systems is a crime. Reproducing unsolved problems is done only with publicly released problem files in a local environment.

The competition is over. Win or lose, what the team has left now is data. Yet winning teams and losing teams usually commit opposite errors — the winners celebrate without asking why, and the losers share emotions instead of causes. Both are ways of throwing data away.

The principle is one. Even when you win, you analyze — "is the reason we won reproducible (skill), or was it luck?" If you lost, you look even more precisely — in which fields did you bleed points, how was your time management, which problems did the winning team solve that you didn’t? Today’s analysis becomes the blueprint for the next challenge — whether that’s a rematch or an international final.

1. Learning Objectives

By the end of this chapter, you will be able to:

Structure competition results into a per-problem analysis table (solve status, time spent, winning-team comparison)
Separate the winning factors of a won competition into skill/luck and assess reproducibility
Run a team operations meeting under the "system, not person" principle
Re-run Step 327’s gap analysis against the winning team’s Write-up
Convert analysis results into a remediation plan, a team charter revision, and the next target competition

2. Background Knowledge — Today’s Tools and Concepts

Today’s Tools at a Glance

Category	Details
Language/environment	Python 3 (results aggregation), a document editor (analysis table, meeting minutes), the winning team’s Write-up
Today’s command	`python step337_result_analysis.py` — aggregate per-field gains/losses, gap coordinates, and efficiency
Concepts needed	Separating win factors into skill/luck, the system-analysis principle, gap coordinates, the format of an improvement plan
Today’s deliverable	A results-analysis document (analysis table + meeting minutes) + a remediation plan + team charter v2 + the next target competition

2-1. Analyzing a Win — The Reproducibility Audit

The analysis of a won competition starts with one question — "can we buy this victory again at the next competition?" Split the win factors into two accounts.

Account	Contents	Example
Skill (reproducible)	Coverage, rule enforcement, libraries, training volume	"Our assignee solved the pwn the opponent couldn’t — a victory of field assignment"
Luck (not reproducible)	Problem-set direction, opponents’ mistakes, a lucky match between problems and experience	"The hard crypto problem came out as one even the winning team couldn’t solve"

Why is this separation needed — because if you mistake a win factor sitting in the luck account for skill, the next competition’s strategy gets built on that mistake. Even in a won competition, only "what we could control" becomes a pillar of the next plan. Celebrate briefly, analyze deeply — that is the winning team’s analytical etiquette.

2-2. Analyzing a Loss — Three Boxes

The analysis of a lost competition is structured by three questions. ① In which fields did we bleed points — the per-problem analysis table answers this. ② How was our time management — Step 326’s day-of log analysis answers this (operations ratio, no-progress stretches, abandonment timing). ③ Which problems did the winning team solve that we didn’t — this is the gap coordinate.

The third matters most. A rank difference is abstract, but "the list of problems they solved and we didn’t" is concrete. Apply Step 327’s comparison table to each problem on that list — is it a knowledge gap, a tool gap, or a time-management gap — and the remediation-plan items drop out automatically. A loss is also the only path to obtaining these coordinates. A won competition shows you what you’re good at; a lost competition shows you what you don’t have.

2-3. The Meeting’s Principle — The System, Not the Person

A team meeting right after a loss easily mixes in emotion. The device that prevents this is one pre-agreed sentence — "we analyze the system, not the person."

The sentence’s effect comes from the transformation of speech. "We lost because you couldn’t solve the crypto" is an analysis of a person; "when a crypto problem comes out, our team stalls" is an analysis of a system. The former invites defense and rebuttal; the latter invites countermeasures — because if swapping the assignee produces the same result, the problem is not the person but the training system. Enforce it as a grammar rule for the minutes — record only sentences whose subject is not a person’s name but "our team / this field / the rule."

2-4. Designing the Next Challenge — Analysis Ends in a Plan

The terminus of analysis is three documents. ① Per-field training goals — the remediation items derived from the gap coordinates (this is the next chapter Step 338’s input value). ② Operations-rule revision — team charter v2 — incorporating the rules that broke at this competition and the rules newly needed. ③ Selecting the next target competition — a domestic rematch, or an international final? The analysis supplies the grounds for selection — depending on whether the gap to the domestic top tier is at the operations level or the fundamentals level, the size of the next stage is decided.

What matters is agreeing on the execution schedule. An analysis document is not complete when the meeting ends — it’s complete when the remediation items have entered each team member’s calendar.

3. Follow Along

3-1. The Per-Problem Analysis Table — Turn Results into Rows

Analysis’s first deliverable is the per-problem analysis table. Every problem in the competition becomes a row; fill in six columns.

■ Per-problem analysis table template (screen example):

| Field | Problem | Points | Our solve | Time (min) | Winning-team solve |
|------|------|------|-----------|----------|-------------|
| web  | chain-reaction | 300 | O | 150 | O |
| web  | session-forge  | 400 | X | —   | O |
| ...  | ...  | ...  | ... | ... | ... |

Column rules:
- For problems we didn't solve, "time" is the time from engagement to abandonment (not 0)
- Verify the winning-team column against the official scoreboard's per-problem stats

The "winning-team solve" column is this table’s core — a problem we didn’t solve is a regret, but a problem the winning team solved and we didn’t is a gap, and a problem even the winning team couldn’t solve is a competition-wide unsolved, not a cause of defeat. Distinguishing the three states (only we failed / both solved / both failed) is the resolution of the analysis.

3-2. The Results-Aggregation Script — Summarize the Table in Numbers

Once the table is filled, summarize it with aggregation. Save as step337_result_analysis.py — replace the sample data in GAMES with your own analysis table.

# step337_result_analysis.py — competition win/loss analysis aggregator
# GAMES: (field, problem name, points, our solve, our minutes, winning-team solve)
GAMES = [
    ("web",    "chain-reaction", 300, True,  150, True),
    ("web",    "session-forge",  400, False, 0,   True),
    ("pwn",    "heap-vault",     500, False, 0,   True),
    ("pwn",    "stack-basic",    200, True,  60,  True),
    ("rev",    "obf-box",        350, True,  180, True),
    ("crypto", "ecc-trap",       500, False, 0,   False),
    ("crypto", "rsa-again",      250, False, 0,   True),
    ("misc",   "signal-noise",   300, True,  90,  True),
]

print("=== Competition Results Analysis ===n")
ours = sum(p for _, _, p, solved, _, _ in GAMES if solved)
theirs = sum(p for _, _, p, _, _, win in GAMES if win)
print(f"Our team's points: {ours}  /  winning team's points (on this problem set): {theirs}"
      f"  /  difference: {theirs - ours}n")

print("[Per-field gains and losses]")
fields = sorted({g[0] for g in GAMES})
for f in fields:
    rows = [g for g in GAMES if g[0] == f]
    got = sum(p for _, _, p, s, _, _ in rows if s)
    lost = sum(p for _, _, p, s, _, w in rows if not s and w)
    zero = all(not s for _, _, _, s, _, _ in rows)
    tag = "  <- 0-solve field (top-priority remediation)" if zero and lost else ""
    print(f"  {f:<10} gained {got:>4} / lost to the winning team {lost:>4}{tag}")

print("n[Problems the winning team solved and we didn't = gap coordinates]")
for field, name, pts, s, m, w in GAMES:
    if not s and w:
        print(f"  {field}/{name} ({pts} pts) — target for winning-team Write-up comparison")

print("n[Efficiency of the problems we solved]")
for field, name, pts, s, m, w in GAMES:
    if s:
        print(f"  {field}/{name}: {pts} pts / {m} min = {pts/m:.1f} pts/min")
print("nNext step: attach a 'why we couldn't solve it' type (knowledge/tools/time management)"
      " to each gap-coordinate problem — Step 327's frame.")

Here’s the measured output from running it on the example data:

=== Competition Results Analysis ===

Our team's points: 1150  /  winning team's points (on this problem set): 2300  /  difference: 1150

[Per-field gains and losses]
  crypto     gained    0 / lost to the winning team  250  <- 0-solve field (top-priority remediation)
  misc       gained  300 / lost to the winning team    0
  pwn        gained  200 / lost to the winning team  500
  rev        gained  350 / lost to the winning team    0
  web        gained  300 / lost to the winning team  400

[Problems the winning team solved and we didn't = gap coordinates]
  web/session-forge (400 pts) — target for winning-team Write-up comparison
  pwn/heap-vault (500 pts) — target for winning-team Write-up comparison
  crypto/rsa-again (250 pts) — target for winning-team Write-up comparison

[Efficiency of the problems we solved]
  web/chain-reaction: 300 pts / 150 min = 2.0 pts/min
  pwn/stack-basic: 200 pts / 60 min = 3.3 pts/min
  rev/obf-box: 350 pts / 180 min = 1.9 pts/min
  misc/signal-noise: 300 pts / 90 min = 3.3 pts/min

Next step: attach a 'why we couldn't solve it' type (knowledge/tools/time management) to each gap-coordinate problem — Step 327's frame.

How to read it: the four blocks each answer a different question. ① The 1150-point total gap — "by how much did we lose." ② Per-field gains and losses — crypto carries the "0-solve field" tag. The diagnosis that the top-tier difference is decided by the weakest field applies verbatim to this team. ③ The three gap-coordinate problems — this list is both the target list for Step 327’s gap analysis and the input to Step 338’s training plan. ④ Efficiency — rev/obf-box was solved but expensive at 1.9 pts/min. The inefficiency of "solved problems" is also material for the next plan — if the winning team solved the same problem in 60 minutes, that difference is a tool gap.

3-3. The Team Operations Meeting — In the Grammar of Systems

Once aggregation is done, hold the team meeting. Here’s an agenda that turns 2-3’s principle into an enforcement device.

■ Team operations meeting agenda (screen example — 60 minutes):

[00–10 min] Reading the principle aloud: "We analyze the system, not the person."
            Grammar rule: no person's name as a subject in the minutes
[10–30 min] Rule inspection: was each rule kept at this competition?
            — enumeration / handoff / time caps / scoreboard / meals
            Rules kept are marked "keep"; for rules that broke, record "when they broke"
[30–45 min] Identifying bottlenecks: the 3 longest no-progress stretches in the log
            — what was the team doing at that moment, did a rule exist?
[45–60 min] Deriving amendments: broken rules → polished into charter v2 clauses
            Closing declaration: "reflections end here; the next agenda item is the plan"

How to read it: look at two devices. ① The grammar rule — forcing records to read "the team stalled on hard crypto" rather than "B couldn’t solve it." This grammar determines the meeting’s temperature. ② The closing declaration — the ritual of closing the meeting with a plan, not reflections. A loss meeting that ends in consolation recurs; one that ends in amended clauses does not.

3-4. Comparing Against the Winning Team’s Write-up — The Gap Analysis Repeated

When the winning team’s (and the top teams’) Write-ups are published, run Step 327’s comparison table on 3-2’s three gap-coordinate problems. There is nothing new in this chapter — the frame is already learned; the only difference is that this time it’s applied to the competition where we aimed for the championship.

■ Gap-coordinate comparison results (screen example):

pwn/heap-vault (500 pts):
  Winning team's solution: tcache poisoning — a technique already in our library
  Our state: never even engaged — assignee B was tied up on crypto
  Gap type: time management (not knowledge!) → prescription: assignment-distribution rule

crypto/rsa-again (250 pts):
  Winning team's solution: common-factor attack — one line of gcd
  Our state: didn't know the pattern existed
  Gap type: knowledge → prescription: reinforce the crypto pattern catalog + past-problem reproduction drills

How to read it: the two problems’ prescriptions are completely different — both are "unsolved problems," yet one is an operations-rule amendment and the other is field training. If you had lumped them together as "let’s study crypto more" without the comparison, heap-vault’s real cause of defeat (operations) would have been lost forever. The value of gap analysis lies in the accuracy of the prescription.

3-5. The Improvement Plan and the Next Goal — The Terminus of Analysis

Close the analysis with three documents.

■ Improvement plan (screen example):

1. Per-field training goals (→ input to Step 338):
   - crypto: organize a 20-pattern catalog + reproduce & vary 10 past problems
   - pwn: 5 heap-family variant problems (also to prevent recurrence of the time-management mistake)
2. Team charter v2 amendments:
   - New: assignment-distribution rule — no one engages 2 fields simultaneously
   - Revised: add a "gap-coordinate candidates" entry item to the scoreboard-check meeting
3. Next target competition: domestic major rematch in 3 months
   — grounds: the gap was confirmed at the coverage/operations level, not fundamentals
   — the international final comes after that (once coverage is complete)

Execution schedule: crypto training every Wed & Sat (owner B, backup A) — registered in calendars

The last line is the completion condition — has the training been converted from "plan" to "schedule"? A goal in a document doesn’t exist yet; only a goal in a calendar gets executed.

4. Missions & Exercises

Mission — The Results-Analysis Document and the Improvement Plan

Fill in the analysis table for every problem in the competition using 3-1’s template — down to the winning-team-solve column.
Transfer the table into step337_result_analysis.py, run it, and confirm the gap-coordinate list and the 0-solve fields.
If you won, write a paragraph separating the win factors into skill/luck; if you lost, write the three-box analysis (field, time management, gap coordinates).
Hold the team operations meeting using 3-3’s agenda and leave minutes — verify with the minutes that the grammar rule (subject restriction) was kept.
Write a winning-team Write-up comparison table for each gap-coordinate problem, and finalize 3-5’s three documents (training goals, charter v2, next competition) together with an execution schedule.

Exercises

Exercise 1. If you don’t separate win factors into skill/luck after a won competition, what error arises at the next competition? Explain with a concrete scenario.

Exercise 2. In the per-problem analysis table, why are "problems even the winning team couldn’t solve" excluded from the causes of defeat? Answer what distortion of the analysis this distinction prevents.

Exercise 3. Explain why the grammar rule of "analyze the system, not the person" (restricting subjects in the minutes) changes the meeting’s outcome, through the relationship between the form of speech and the countermeasures it produces.

Exercise 4. In 3-4’s heap-vault case, what is missed when you lump everything into "let’s study crypto more" without the comparison, and how does classifying the gap type raise the prescription’s accuracy?

5. Model Answers & Completion Criteria

Mission Model Answer

Check against these verification criteria.

Completeness of the table: is every problem a row, and is the "winning-team solve" column filled from official stats? A table with only our records is half a table.
Aggregation executed: does the script output survive, and was the gap-coordinate list connected to follow-up work (the comparison table)?
Form of the win/loss analysis: if won, is there a skill/luck separation paragraph; if lost, the three-box analysis? A personal reflection doesn’t count.
Grammar of the minutes: are there no sentences whose subject is a person’s name, and does every broken rule have its "moment of breakage" recorded?
Conversion into a plan: are the three documents finalized, and have the training items entered calendars with owners and weekdays specified?

Exercise Answers

Answer 1. A team that mistakes a luck-based win factor for skill loses the next competition with the same strategy once that luck disappears. Draw the scenario — if you won this time because the hard crypto problem came out as one even the winning team couldn’t solve, the illusion forms that "we can win without reinforcing crypto." When a mid-level crypto problem appears at the next competition, that team is defenseless. Separating win factors is an audit procedure that restricts the pillars of the next strategy to "what was controllable" — only reproducible factors like coverage and rule enforcement become grounds for the next plan; what sat in the luck account is received with gratitude but kept out of the plan.

Answer 2. A problem neither team solved only shows the competition’s difficulty ceiling — it’s not a defect specific to our team; no team "lost points" on it. Without this distinction, the distortion that arises is a flood of the remediation list — if every unsolved problem becomes a remediation target, the list grows unexecutable, and the very problems that separated us from the winning team (the gap coordinates) get buried in that long list. Since the analysis’s purpose is not a complete problem list but identifying the difference from the winning team, the "they solved it and we didn’t" condition is the filter that narrows the list to an executable size.

Answer 3. Because a sentence’s subject determines the reaction that sentence invites. "B couldn’t solve it" is a sentence whose subject is a person — it summons the listener’s defensive instinct, and the meeting converts from truth-seeking into a blame contest. "The team stalled on crypto" is a sentence whose subject is a system — it summons the listener’s problem-solving instinct; this sentence’s natural next sentence is not rebuke but "why did it stall / what was needed?" And substantively, system analysis is also more accurate — if swapping the assignee still stalls on the same problem, the defect lies in the training system, not the person. The grammar rule is not meeting etiquette; it’s an accuracy device for the analysis.

Answer 4. What gets missed is heap-vault’s real cause of defeat — time management. That problem was not a knowledge gap (we had the same technique in our library as the winning team). If you lump "unsolved problem = field to study more," the operations defect enters no training list, and at the next competition the recurrence happens: an assignee gets tied up on one problem again and misses another. Classifying the gap type (knowledge/tools/time management) raises accuracy because each type has a different corresponding prescription — training for knowledge, library reinforcement for tools, rule amendments for time management. Only once the type is fixed can the prescription be fixed, and the comparison table is the procedure that judges that type.

Completion Criteria Checklist

[ ] I completed the analysis table for every problem, down to the "winning-team solve" column
[ ] I ran the aggregation script and confirmed the 0-solve fields and gap coordinates
[ ] I wrote the analysis matching the outcome (skill/luck separation or the three-box analysis) as a document
[ ] I held the team operations meeting and left minutes where the grammar rule was kept
[ ] I completed the winning-team Write-up comparison table for every gap-coordinate problem
[ ] I finalized the team charter v2 amendments
[ ] I decided the next target competition together with the selection grounds
[ ] The training goals were converted into a schedule with owners and weekdays specified

6. Common Pitfalls & Fixes

Wall 1. The meeting turned into an emotional fight

Symptom: it starts with "why did you grab that problem back then" and ends on a bad note.

Cause: you started the meeting without the pre-agreement, or the agreement existed but had no enforcement device.

Fix: pause the meeting once and set up the devices first. ① Put the minutes’ grammar rule on screen — no person’s name as a subject. ② Fix the speech format — a "problem raised" is allowed only as two sentences: "fact (log timestamp) + system interpretation." ③ If emotions still rise, postpone the meeting 24 hours — moving a post-loss meeting to the next day is not avoidance, it’s design. This is a meeting where emotion easily mixes in, and the device must be set up in advance or right there on the spot.

Wall 2. The winning team’s Write-up isn’t getting published

Symptom: there’s only the official solution; the winning team’s detailed Write-up isn’t released.

Cause: a common occurrence — top teams delay or skip publication to protect strategy.

Fix: there are three fallback substitutes. ① The official author’s write-up — it gives the skeleton of the intended solution. ② Write-ups from teams ranked 2nd–5th — even if not the winning team, a record from a team that solved that problem makes the comparison possible. ③ Preserving your failed-reproduction records — keep your attempt history until a Write-up appears, and compare whenever one is published. Gap analysis is not deadline work — as long as the coordinates are saved, the material can be used whenever it arrives.

Wall 3. We won, but the team finds analyzing it a chore

Symptom: an atmosphere of "we won — why can’t we just celebrate instead of analyzing?"

Cause: the need to analyze a won competition hasn’t been shared — victory feels like an exemption from analysis, but it’s actually analysis’s golden hour.

Fix: celebration first, analysis within 48 hours — solve it by order. And the persuasion has one argument — a won competition’s log is a sample of our team’s operations at peak condition. If you don’t extract the "reproducible win factors" from that sample, you can’t reproduce the reason you won at the next competition. If analyzing a loss is removing the causes of defeat, analyzing a win is preserving the causes of victory — both are material for the next championship.

Wall 4. We analyzed, but no plan comes out — the meeting just drags on

Symptom: the list of problems is long, but "who does what by when" never gets decided.

Cause: there’s no conversion rule from analysis to plan — problems don’t become plans on their own.

Fix: enforce the conversion rule — before the meeting ends, every item in the analysis document must carry a tag of (owner, deadline, deliverable), and items that can’t take a tag move to the "deferred" box. Deferred is not deleted; it’s the first agenda item of the next meeting. And remember 3-5’s last line — a goal that hasn’t entered a calendar doesn’t exist yet.

Wall 5. The analysis result summarizes only as "insufficient skill" — it feels hopeless

Symptom: no structure emerges beyond "we lost because we’re not good enough."

Cause: insufficient analytical resolution — "insufficient skill" is not the conclusion of analysis; it’s the sentence you had before analysis began.

Fix: go back to 3-1’s analysis table — the resolution is enforced by the table’s columns. Even a competition that feels like "insufficient skill" decomposes into a mixture once you split it by problem — two problems of knowledge gap, one problem of operations loss, one problem of insufficient time. The moment it divides into a mixture, the hopelessness changes into a work list — because each item is small in size and different in prescription. Hopelessness is an emotion that comes when the target is large; analysis is the technique of splitting the target small.

7. Summary

Today’s Concepts

Concept	One-line explanation
Separating win factors	Two accounts of reproducible skill and luck — only skill enters the plan
Gap coordinates	Problems the winning team solved and we didn’t — analysis’s exact target
0-solve fields	What decides the top-tier difference — the top-priority remediation tag
System-analysis principle	Move the subject from person to system — grammar changes the meeting’s outcome
Charter v2	Amending the broken rules — a meeting must end in clauses not to recur
Plan conversion	An analysis item becomes a plan only with an (owner, deadline, deliverable) tag

Today’s Tools & Commands

Tool/command	What it does
`python step337_result_analysis.py`	Batch aggregation of per-field gains/losses, gap coordinates, and efficiency
Per-problem analysis table	Six columns: points, solve, time, winning-team comparison
Meeting agenda	Principle reading → rule inspection → bottleneck identification → amendment derivation
Gap-type classification	Knowledge/tools/time management — the judgment that decides the prescription
Three improvement-plan documents	Training goals · charter v2 · next target competition

The Core Instinct

If you compress the attitude of win/loss analysis into one sentence — don’t judge the result; mine it. The rank is finished, but the data is usable starting now. In won competitions and lost competitions alike, the mining yield is the same — rather, the lost one gives more. It’s the only place that shows you what you don’t have.

And confirm what this chapter has produced — gap coordinates and 0-solve fields, and a training schedule that has entered calendars. That is the next chapter’s input value. Training without analysis is like a hand without joints, and analysis without training is decoration. Now you head into the period of conquering those coordinates head-on — intensive weakness training.

Once every box is checked, Step 337 is complete.