Adding a decision ledger to Codex Plan mode with codex-plan-ledger: recording choices and checking scope drift

When you write code with Codex's Plan mode (/plan), the most valuable part is often not the plan itself, but the handful of choices settled during planning: whether to retry on failure, whether config goes through arguments or environment variables, which files must not be touched. These choices usually live only in the conversation. Look at the PR two weeks later and reviewers see the code, not "why we chose this back then"; there's nothing to check whether the code went beyond what was agreed.

codex-plan-ledger does exactly these two things:

1. Decision ledger. It writes the decisions from Plan mode into a docs/plans/<id>/decisions.json in the repo: the questions Codex asked you and your answers, the choices still pending in the plan, the defaults the plan spells out — all of it. It sits in the same PR as the code, so it can be diffed and reviewed.
2. Scope drift check. After the code is written, plan-ledger check --base main compares the changed files against the plan: files changed outside the plan, and files listed in the decisions but never touched, both get reported.

How it works

How to use it

npm i -g github:miniLV/codex-plan-ledger
plan-ledger init --write   # 写入 Codex hooks 配置

After that, /plan in Codex as usual, and answer Codex's questions in Codex as usual. When the plan comes out, the Stop hook does three things: parse the plan, write the ledger, generate a one-page offline HTML. You get one extra line in the terminal, something like:

plan-ledger: nothing to answer; 2 answered in Codex → file://…/plan.html

On this page you can review all the decisions: ones already answered in Codex are marked "answered in Codex"; defaults spelled out in the plan are marked "default, reviewable" and don't count as pending; for items not yet settled, pick them all at once, click the button to generate the JSON to send back, and paste it into the next message. Unselected items are recorded as "unanswered, default kept" and do not count as agreement.

After the code is written:

plan-ledger check --base main          # 可读报告
plan-ledger check --base main --strict # 有偏离时 exit 1，可以放进 CI

A few design constraints:

• Rendering is pure templating throughout, no extra model calls, zero dependencies, a single HTML file, no network.
• The hook never blocks the turn, and never waits for you. If the plan format isn't recognized, it passes through as-is with just one extra notice.
• Answers on the page are stored as data only and are never executed.

What the ledger looks like

{
  "id": "n-selection",
  "title": "Selection",
  "question": "How should users select the greeting language?",
  "source": "codex-native",        // Codex 自己问的，你在 Codex 里答的
  "options": [
    { "id": "a", "label": "Named flag", "recommended": true },
    { "id": "b", "label": "Second positional argument", "recommended": false },
    { "id": "c", "label": "Environment variable", "recommended": false }
  ],
  "chosen": "a",
  "status": "answered"
}

This section comes from a real Plan mode session (codex-cli 0.156.0): Codex asked 2 questions with request_user_input, and the real Stop hook read the Q&A from the session log and wrote it into the ledger. The whole session was about 43000 tokens. One caveat: this prompt explicitly asked Codex to ask questions first, so it's a feature check, not an effectiveness measurement.

Measured: honestly, the number of turns didn't go down

The original idea was "answer all the decisions on one page at once, less hassle than Codex asking round after round." I ran a paired experiment on vercel/[email protected]: same task, same model and settings, native Plan mode and plan-ledger each run once, with questions answered by a script against a hidden answer key. Two rounds total, 2 tasks, n = 3 pairs, so it's directional data only:

• Turns didn't improve: out of 3 pairs, lost 2, tied 1. Tokens went both up and down with no consistent direction (the two rounds used different run configurations, so tokens can only be compared within the same round).
• Scope drift caught 3/3: after implementation I planted an out-of-plan new file in a copy and reverted an in-plan file to its original state, and all 3 runs reported it; when clean, 3/3 had no false positives.
• In-file changes contradicting decisions caught 0/3: writing code inside an in-plan file that contradicts the chosen approach goes undetected. It only looks at "which files changed," not the code content — a known limitation.

So the project's positioning shifted from "answer on one page, save turns" to what it is now: "decision ledger + scope drift check." The experiment also surfaced two things, now fixed: Codex Plan mode asks questions first with its built-in request_user_input even when told to list decisions, so now those Q&As are recorded directly into the ledger instead of being asked again; and the "Chosen defaults" at the end of the plan used to be treated as pending items, but now they're recorded as defaults, reviewable, not pending. These changes haven't been re-measured yet.

The full data and scripts are in the repo: round 1, round 2, measured plans.

Current status and next steps

Early prototype v0.1, with test coverage. The experimental parts: the UserPromptSubmit hook, and reading native Q&A from the session log (the session log format isn't a public contract; if it can't be read, it's skipped). Windows is untested.

Next steps:

• Check decisions by content (that's the 0/3 above).
• More tasks, at least 3 repetitions per group, to measure how well the ledger covers decisions in a session and how well drift detection works.

Try it out and file issues: github.com/miniLV/code… · homepage and live demo: minilv.github.io/codex-plan-…

Inspired by html-plan and answer-me-with-html; no code reused.