Systematic debugging

Keeps the agent from guessing at fixes: reproduce the bug and find the root cause first, test one hypothesis at a time, fix with a failing test, and question the design after three failed fixes. Use for any bug, test failure or unexpected behavior.

DebuggingPack: Superpowers283 linessystematic-debugging
Author
Source
obra/superpowers/skills/systematic-debugging @ 22f0b46
License
MIT · Copyright (c) 2025 Jesse Vincent

About this skill

What it does

Systematic debugging keeps the agent from patching symptoms. Before any fix it has to find the root cause, in four phases: read the error and reproduce the bug, compare the broken code with a working example, test one hypothesis at a time, and only then fix the cause, with a failing test written first. After three fixes that did not work, the agent must stop and question the design instead of trying a fourth.

The folder ships the helpers the skill points to: tracing a bad value back up the call stack (root-cause-tracing.md), validation at several layers once the cause is known (defense-in-depth.md), waiting on a condition instead of a fixed timeout (condition-based-waiting.md, with a TypeScript example) and find-polluter.sh, which finds the test that leaves state behind for the next one.

When to use it, and when not

Use it for a failing test, a production bug, a broken build, a slowdown or anything that behaves unexpectedly — above all under time pressure, when the obvious quick fix has already failed once or the cause is not clear.

The skill itself says not to skip it because a bug looks simple: simple bugs have root causes too, and the process is quick for them. It is not for work where nothing is broken: a new feature or a planned change starts with brainstorming and writing-plans.

How to run it

The install command below puts the whole folder in place: SKILL.md, the helpers and the licence. The agent picks the skill up by its description whenever a bug comes up; to call it yourself:

  • Claude Code: /systematic-debugging, or ask for the skill by name.
  • Codex: $systematic-debugging.
  • Cursor: type / in the Agent chat and pick systematic-debugging.

Example: /systematic-debugging The checkout test fails on CI but passes locally. Find out why before changing anything.

The skill hands off to two other Superpowers skills, test-driven-development for the failing test and verification-before-completion before it reports the fix; install them too for the whole loop.

How it differs from similar skills

Root-cause debugging from our catalog takes the same path — reproduce, isolate, one hypothesis, fix, regression test — as a shorter checklist. Systematic debugging is stricter: an "iron law" of no fix before investigation, a list of thoughts that send the agent back to phase one, the three-fixes rule and logging at every boundary of a multi-component system. Pick it for an agent that tends to guess; pick root-cause debugging when a lighter checklist is enough. Diagnosing hard bugs puts most of the effort into a fast, deterministic command that goes red on this exact bug, then ranks 3–5 hypotheses at once for you to reorder. Debugging and error recovery has no separate hypothesis step and adds a stop-the-line rule and triage trees for test, build and runtime failures.

Install

Run one command — it saves SKILL.md and the files it uses into your agent's skills folder. Claude Code and Codex pick up a new skill without a restart; if it does not show up, restart the agent.

Claude Code — macOS, Linux
curl -fsSL --create-dirs -o ~/.claude/skills/systematic-debugging/SKILL.md https://anymodel.org/en/skills/superpowers/systematic-debugging/SKILL.md -o ~/.claude/skills/systematic-debugging/LICENSE https://anymodel.org/en/skills/superpowers/systematic-debugging/LICENSE -o ~/.claude/skills/systematic-debugging/condition-based-waiting-example.ts https://anymodel.org/en/skills/superpowers/systematic-debugging/condition-based-waiting-example.ts -o ~/.claude/skills/systematic-debugging/condition-based-waiting.md https://anymodel.org/en/skills/superpowers/systematic-debugging/condition-based-waiting.md -o ~/.claude/skills/systematic-debugging/defense-in-depth.md https://anymodel.org/en/skills/superpowers/systematic-debugging/defense-in-depth.md -o ~/.claude/skills/systematic-debugging/find-polluter.sh https://anymodel.org/en/skills/superpowers/systematic-debugging/find-polluter.sh -o ~/.claude/skills/systematic-debugging/root-cause-tracing.md https://anymodel.org/en/skills/superpowers/systematic-debugging/root-cause-tracing.md && chmod +x ~/.claude/skills/systematic-debugging/find-polluter.sh
Codex, Cursor — macOS, Linux
curl -fsSL --create-dirs -o ~/.agents/skills/systematic-debugging/SKILL.md https://anymodel.org/en/skills/superpowers/systematic-debugging/SKILL.md -o ~/.agents/skills/systematic-debugging/LICENSE https://anymodel.org/en/skills/superpowers/systematic-debugging/LICENSE -o ~/.agents/skills/systematic-debugging/condition-based-waiting-example.ts https://anymodel.org/en/skills/superpowers/systematic-debugging/condition-based-waiting-example.ts -o ~/.agents/skills/systematic-debugging/condition-based-waiting.md https://anymodel.org/en/skills/superpowers/systematic-debugging/condition-based-waiting.md -o ~/.agents/skills/systematic-debugging/defense-in-depth.md https://anymodel.org/en/skills/superpowers/systematic-debugging/defense-in-depth.md -o ~/.agents/skills/systematic-debugging/find-polluter.sh https://anymodel.org/en/skills/superpowers/systematic-debugging/find-polluter.sh -o ~/.agents/skills/systematic-debugging/root-cause-tracing.md https://anymodel.org/en/skills/superpowers/systematic-debugging/root-cause-tracing.md && chmod +x ~/.agents/skills/systematic-debugging/find-polluter.sh
Claude Code — Windows PowerShell
foreach ($f in 'SKILL.md','LICENSE','condition-based-waiting-example.ts','condition-based-waiting.md','defense-in-depth.md','find-polluter.sh','root-cause-tracing.md') { $p = Join-Path "$HOME\.claude\skills\systematic-debugging" $f; New-Item -ItemType Directory -Force (Split-Path $p) | Out-Null; Invoke-WebRequest "https://anymodel.org/en/skills/superpowers/systematic-debugging/$f" -UseBasicParsing -OutFile $p }
Codex, Cursor — Windows PowerShell
foreach ($f in 'SKILL.md','LICENSE','condition-based-waiting-example.ts','condition-based-waiting.md','defense-in-depth.md','find-polluter.sh','root-cause-tracing.md') { $p = Join-Path "$HOME\.agents\skills\systematic-debugging" $f; New-Item -ItemType Directory -Force (Split-Path $p) | Out-Null; Invoke-WebRequest "https://anymodel.org/en/skills/superpowers/systematic-debugging/$f" -UseBasicParsing -OutFile $p }

Any other agent: save the file into its skills folder, or give it the link to the file: https://anymodel.org/en/skills/superpowers/systematic-debugging/SKILL.md

SKILL.md

Raw

Systematic Debugging

Overview

Core principle: ALWAYS find root cause before attempting fixes. Symptom fixes are failure.

Violating the letter of this process is violating the spirit of debugging.

The Iron Law

NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST

If you haven't completed Phase 1, you cannot propose fixes.

When to Use

Use for ANY technical issue:

  • Test failures
  • Bugs in production
  • Unexpected behavior
  • Performance problems
  • Build failures
  • Integration issues

Use this ESPECIALLY when:

  • Under time pressure (emergencies make guessing tempting)
  • "Just one quick fix" seems obvious
  • You've already tried multiple fixes
  • Previous fix didn't work
  • You don't fully understand the issue

Don't skip when:

  • Issue seems simple (simple bugs have root causes too)
  • You're in a hurry (rushing guarantees rework)
  • Manager wants it fixed NOW (systematic is faster than thrashing)

The Four Phases

You MUST complete each phase before proceeding to the next.

Phase 1: Root Cause Investigation

BEFORE attempting ANY fix:

  1. Read Error Messages Carefully

    • Don't skip past errors or warnings
    • They often contain the exact solution
    • Read stack traces completely
    • Note line numbers, file paths, error codes
  2. Reproduce Consistently

    • Can you trigger it reliably?
    • What are the exact steps?
    • Does it happen every time?
    • If not reproducible → gather more data, don't guess
  3. Check Recent Changes

    • What changed that could cause this?
    • Git diff, recent commits
    • New dependencies, config changes
    • Environmental differences
  4. Gather Evidence in Multi-Component Systems

    WHEN system has multiple components (CI → build → signing, API → service → database):

    BEFORE proposing fixes, add diagnostic instrumentation:

    For EACH component boundary:
      - Log what data enters component
      - Log what data exits component
      - Verify environment/config propagation
      - Check state at each layer
    
    Run once to gather evidence showing WHERE it breaks
    THEN analyze evidence to identify failing component
    THEN investigate that specific component
    

    Example (multi-layer system):

    # Layer 1: Workflow
    echo "=== Secrets available in workflow: ==="
    echo "IDENTITY: $([ -n "${IDENTITY:-}" ] && echo SET || echo UNSET)"
    
    # Layer 2: Build script
    echo "=== Env vars in build script: ==="
    printenv IDENTITY >/dev/null 2>&1 && echo "IDENTITY in environment" || echo "IDENTITY not in environment"
    
    # Layer 3: Signing script
    echo "=== Keychain state: ==="
    security list-keychains
    security find-identity -v
    
    # Layer 4: Actual signing
    codesign --sign "$IDENTITY" --verbose=4 "$APP"
    

    This reveals: Which layer fails (secrets → workflow ✓, workflow → build ✗)

  5. Trace Data Flow

    WHEN error is deep in call stack:

    See root-cause-tracing.md in this directory for the complete backward tracing technique.

    Quick version:

    • Where does bad value originate?
    • What called this with bad value?
    • Keep tracing up until you find the source
    • Fix at source, not at symptom

Phase 2: Pattern Analysis

Find the pattern before fixing:

  1. Find Working Examples

    • Locate similar working code in same codebase
    • What works that's similar to what's broken?
  2. Compare Against References

    • If implementing pattern, read reference implementation COMPLETELY
    • Don't skim - read every line
    • Understand the pattern fully before applying
  3. Identify Differences

    • What's different between working and broken?
    • List every difference, however small
    • Don't assume "that can't matter"
  4. Understand Dependencies

    • What other components does this need?
    • What settings, config, environment?
    • What assumptions does it make?

Phase 3: Hypothesis and Testing

Scientific method:

  1. Form Single Hypothesis

    • State clearly: "I think X is the root cause because Y"
    • Write it down
    • Be specific, not vague
  2. Test Minimally

    • Make the SMALLEST possible change to test hypothesis
    • One variable at a time
    • Don't fix multiple things at once
  3. Verify Before Continuing

    • Did it work? Yes → Phase 4
    • Didn't work? Form NEW hypothesis
    • DON'T add more fixes on top
  4. When You Don't Know

    • Say "I don't understand X"
    • Don't pretend to know
    • Ask for help
    • Research more

Phase 4: Implementation

Fix the root cause, not the symptom:

  1. Create Failing Test Case

    • Simplest possible reproduction
    • Automated test if possible
    • One-off test script if no framework
    • MUST have before fixing
    • Use the superpowers:test-driven-development skill for writing proper failing tests
  2. Implement Single Fix

    • Address the root cause identified
    • ONE change at a time
    • No "while I'm here" improvements
    • No bundled refactoring
  3. Verify Fix

    • Test passes now?
    • No other tests broken?
    • Issue actually resolved?
    • Use the superpowers:verification-before-completion skill before claiming success
  4. If Fix Doesn't Work

    • STOP
    • Count: How many fixes have you tried?
    • If < 3: Return to Phase 1, re-analyze with new information
    • If ≥ 3: STOP and question the architecture (step 5 below)
    • DON'T attempt Fix #4 without architectural discussion
  5. If 3+ Fixes Failed: Question Architecture

    Pattern indicating architectural problem:

    • Each fix reveals new shared state/coupling/problem in different place
    • Fixes require "massive refactoring" to implement
    • Each fix creates new symptoms elsewhere

    STOP and question fundamentals:

    • Is this pattern fundamentally sound?
    • Are we "sticking with it through sheer inertia"?
    • Should we refactor architecture vs. continue fixing symptoms?

    Discuss with your human partner before attempting more fixes

    This is NOT a failed hypothesis - this is a wrong architecture.

Red Flags - STOP and Follow Process

If you catch yourself thinking:

  • "Quick fix for now, investigate later"
  • "Just try changing X and see if it works"
  • "Add multiple changes, run tests"
  • "Skip the test, I'll manually verify"
  • "It's probably X, let me fix that"
  • "I don't fully understand but this might work"
  • "Pattern says X but I'll adapt it differently"
  • "Here are the main problems: [lists fixes without investigation]"
  • Proposing solutions before tracing data flow
  • "One more fix attempt" (when already tried 2+)
  • Each fix reveals new problem in different place

ALL of these mean: STOP. Return to Phase 1.

If 3+ fixes failed: Question the architecture (see Phase 4.5)

your human partner's Signals You're Doing It Wrong

Watch for these redirections:

  • "Is that not happening?" - You assumed without verifying
  • "Will it show us...?" - You should have added evidence gathering
  • "Stop guessing" - You're proposing fixes without understanding
  • "Ultra-think this" - Question fundamentals, not just symptoms
  • "We're stuck?" (frustrated) - Your approach isn't working

When you see these: STOP. Return to Phase 1.

Common Rationalizations

Excuse Reality
"Issue is simple, don't need process" Simple issues have root causes too. Process is fast for simple bugs.
"Emergency, no time for process" Systematic debugging is FASTER than guess-and-check thrashing.
"Just try this first, then investigate" First fix sets the pattern. Do it right from the start.
"I'll write test after confirming fix works" Untested fixes don't stick. Test first proves it.
"Multiple fixes at once saves time" Can't isolate what worked. Causes new bugs.
"Reference too long, I'll adapt the pattern" Partial understanding guarantees bugs. Read it completely.
"I see the problem, let me fix it" Seeing symptoms ≠ understanding root cause.
"One more fix attempt" (after 2+ failures) 3+ failures = architectural problem. Question pattern, don't fix again.

Quick Reference

Phase Key Activities Success Criteria
1. Root Cause Read errors, reproduce, check changes, gather evidence Understand WHAT and WHY
2. Pattern Find working examples, compare Identify differences
3. Hypothesis Form theory, test minimally Confirmed or new hypothesis
4. Implementation Create test, fix, verify Bug resolved, tests pass

When Process Reveals "No Root Cause"

If systematic investigation reveals issue is truly environmental, timing-dependent, or external:

  1. You've completed the process
  2. Document what you investigated
  3. Implement appropriate handling (retry, timeout, error message)
  4. Add monitoring/logging for future investigation

But: 95% of "no root cause" cases are incomplete investigation.

Supporting Techniques

These techniques are part of systematic debugging and available in this directory:

  • root-cause-tracing.md - Trace bugs backward through call stack to find original trigger
  • defense-in-depth.md - Add validation at multiple layers after finding root cause
  • condition-based-waiting.md - Replace arbitrary timeouts with condition polling

Connect through AnyModel API

This skill runs on whatever model your agent uses. Point Claude Code, Codex or Cursor at AnyModel to run it on Claude, GPT, Gemini and more with one key — pay per token, no subscription.

More from the pack Superpowers

Brainstorming before you build

Finds out what you want to build and why: one question at a time, its understanding played back part by part, two or three approaches to pick from and a design doc, with no code until you approve it. Use for a new feature or change when the request leaves the essentials open.

Dispatching parallel agents

Splits independent problems between parallel subagents, each with one domain and a self-contained brief with scope, constraints and the report expected back, then checks their edits for conflicts and runs the full suite. Use when several unrelated things fail at once.

Executing plans

Runs a plan from writing-plans in the current session with no subagent per task: tasks in order through TDD, a ledger of progress and rulings that survives context compaction, and one fresh review of the whole branch at the end. Use when per-task review is not needed or there are no subagents.

Finishing a development branch

Finishes a branch: runs the tests, offers to merge locally, open a pull request or keep it as is, reruns the tests after a merge and removes its own worktree, and discards work only after you type "discard". Use when the implementation is done.

Receiving code review

Checks each review comment against the code before changing anything, clears up every unclear item first, pushes back with technical reasons and never performs agreement. Use when working through review feedback without blind fixes.

Requesting code review

Hands finished work to a fresh reviewer subagent through a template: the commit range, the plan or requirements, findings by severity and a verdict on merging. Use when a task or feature is done and a merge to main is next.

Subagent-driven development

Executes a plan through subagents: a fresh implementer and reviewer for every task, checking spec compliance and code quality, up to five fix rounds and a final review of the whole branch. Use for a plan of independent tasks when each one needs a review.

Test-driven development, Superpowers style

Drives work test-first: a test seen failing, then the least code that passes and a run of the whole suite; code written before its test is deleted. Use for features and bug fixes when the agent tends to add tests after the code.

Using git worktrees

Moves feature work into an isolated git worktree: checks whether the agent is already isolated, prefers the harness's own worktree tool, keeps the folder git-ignored, installs dependencies and runs a baseline test. Use when the work must not land on the branch you have open.

Verification before completion

Stops the agent from saying "done" or "tests pass" until it has rerun the check and read the whole output. Use whenever the agent reports success, commits or opens a pull request.

Writing implementation plans

Turns a spec into a plan for an implementer new to the codebase: files, signatures, interfaces between tasks, tests with exact values and TDD steps with each command's expected output. Use for a multi-step task when the spec is ready and no code exists yet.

Writing skills

Writes and edits skills the TDD way: a subagent first does the task without the skill and its mistakes are written down, then the smallest text that answers them goes in and the loopholes get closed. Use for a skill that other agents will load.