Playbook

Stale documentation: an audit-and-fix playbook

Every team has docs nobody trusts. The fix is not a heroic rewrite, it is a short audit in three passes: find the docs that lie, triage them ruthlessly, and automate freshness for the few that matter. This is the playbook, with the checklists.

Redraw the stale diagramWhy docs drift
9 min readFor whoever just inherited the docs

See it as a diagram

Everything below, as a diagram you can edit. Describe yours and see it in seconds.

133/2000• 3 credits left
Try:

No account needed · Editable canvas, not a picture

Why an audit beats a rewrite

The instinct in front of a rotten wiki is to rewrite it. Resist that. A rewrite produces a fresh snapshot that starts rotting the day it ships, because the process that rotted the last one is untouched. (The mechanism behind that rot has a name and a page: documentation drift.)

The audit takes a different position: most stale docs should not be fixed at all. They should be deleted or clearly marked as history, so the small set that remains can be kept true, mostly by machines. Three passes, roughly an afternoon for a mid-sized docs tree.

Pass 1: find the stale docs (about 30 minutes)

Do not read pages. Collect signals. Any doc that trips two of these is presumed stale:

  • Old doc, busy subject. Last-modified date far behind the change rate of the code or system it describes. In a repo this is one loop:
# Every doc, oldest first, with the date it last changed
git ls-files 'docs/**/*.md' | while read -r f; do
  printf '%s  %s\n' "$(git log -1 --format=%as -- "$f")" "$f"
done | sort
  • Broken links and dead references. Run a link checker, then grep the docs tree for the names of services, tables, or repos you have renamed or retired. One hit is a stale page found.
  • Screenshots and exported diagrams. Every pasted architecture image is stale until someone proves otherwise. Nobody re-exports on change; that is the whole problem with exporting.
  • Questions the docs supposedly answer. Search Slack for the last month of “how does X work” questions. Each one that has a doc is a doc people tried and abandoned, or never found. Both are findings.
  • Zero readers. If you have page analytics, anything with no views in ninety days goes on the list, not to fix, but to delete.

Pass 2: triage into four buckets (about an hour)

Every flagged doc gets exactly one of four labels. The order matters; try each label before falling through to the next:

Delete. No readers, or superseded, or describes something that no longer exists. A wrong doc is worse than no doc, because absence sends the reader to the code while a stale page sends them confidently in the wrong direction. This bucket should be the biggest, and deleting is the step teams skip out of sentimentality.

Date-stamp. Design docs, decision records, postmortems. These are true statements about the past, not claims about the present. Add a visible date and a one-line banner (“describes the system as of the date above”) and they stop misleading anyone without costing a minute of maintenance.

Fix by hand. Load-bearing docs whose subject changes rarely: the on-call runbook, the access request process. Hand-maintenance is fine where change is yearly.

Automate. Load-bearing docs whose subject changes often: API reference, schema docs, the architecture diagram. Fixing these by hand just schedules the next audit. They go to pass 3.

The trustworthy core

Most teams end this pass with ten to twenty docs that actually matter. That is not a failure of documentation culture, it is the honest size of what a team can keep true. Everything else is generated output or clearly labeled history.

Pass 3: automate freshness for what remains

The automate bucket gets one of three structural fixes, chosen by doc type:

Generate it from source. API reference from OpenAPI, infrastructure reference from terraform-docs, warehouse models from dbt docs. Generated docs are build artifacts, so they cannot drift; they can only fail to build, which CI notices. Data platforms have the most to gain here, and a dedicated guide on keeping data documentation in sync covers that stack in detail.

Move it into the repo. Prose that cannot be generated (the why, the guide, the caveat) lives next to the code and changes in the same commit. Add the crude CI rule: when a flagged directory changes and its doc does not, the PR fails and the author writes one line explaining why. Drift still happens; it just happens in review, where someone sees it.

Replace the export with a live embed. The stale architecture screenshot has a structural fix: a diagram that is embedded from a live canvas updates in every doc the moment the canvas changes. One edit, every copy current. The data pipeline diagram that appears in four different docs is the textbook case: keep one source, embed it four times.

And if your team codes with agents, close the last gap: through an MCP diagram server, the agent that just changed the pipeline can update the diagram in the same session, while it still has the change in context. The doc update stops being a separate task, which is the reason it was skipped in the first place.

These fixes are a subset of a larger menu; the full comparison lives in best tools to keep docs and code in sync, and the target state they add up to is described in living documentation.

The loop that keeps it fixed

Staying fixed costs three small habits, not a documentation culture:

  • Per PR: the CI drift rule runs; authors either update the paired doc or say in one line why not.
  • Monthly: regenerate the doc-age listing from pass 1 and skim the top. It takes five minutes and catches what the CI rule cannot see.
  • Quarterly: rerun the audit light: check the trustworthy core is still true, delete anything that snuck back in, confirm every doc still has an owner.

If the quarterly pass keeps finding fresh rot in the same doc, that doc belongs in the automate bucket, and the cycle shrinks it out of the manual set.

FAQ

How do I find stale documentation?

Use signals that do not require reading every page: last-modified dates compared against the change rate of the system described, broken links, references to renamed or retired services, screenshots and exported diagrams (assume stale until proven otherwise), and the questions people still ask in chat even though a doc supposedly answers them. Reading everything is the method that never finishes.

How do I keep documentation in sync with code?

Three mechanisms, in order of reliability: generate docs from source so they cannot drift (OpenAPI, dbt docs, terraform-docs); keep hand-written docs in the repository so they change in the same commit as the code; and use live artifacts, like embedded diagrams, that update at the source instead of being re-exported. Process fixes such as PR checklists and doc owners help, but they degrade under deadline pressure. The structural fixes do not.

Should I delete old documentation?

Usually, yes. A wrong doc is worse than no doc: absence sends the reader to the code or a teammate, while a stale page sends them confidently in the wrong direction. Delete pages nobody reads, date-stamp decision records so they read as history rather than current state, and keep only what you are prepared to keep true.

How often should documentation be reviewed?

Pin the cadence to change rate, not the calendar. High-churn docs (architecture overviews, setup guides) deserve a check whenever their subject changes, ideally enforced by a CI rule. Low-churn docs are fine with a quarterly pass. A single yearly review of everything is the cadence that produced the mess in the first place.

Can AI fix stale documentation?

It helps more with prevention than with cleanup. A coding agent that just made a change can update the affected doc or diagram in the same session, while the context is loaded, and that reliably prevents new staleness. Asking an LLM to repair an already-stale wiki is weaker: a human who knows the current truth still has to verify every claim, and that verification is most of the original work.