Use Case Guide

Incident diagram: mapping the system that keeps paging you

By the engineer who builds Datadef, from client work on data platforms · Reviewed August 21, 2026

Some systems generate a disproportionate share of the incidents, and everyone on the rotation knows which ones they are. The artefact that helps is not a full topology map, it is a narrow diagram of the failure path: what the alert means, what sits upstream of it, what breaks downstream when it goes, and who to wake. The hard part is not drawing it. It is putting it where the page routes to, and keeping it true while the team actively works to remove the failure mode it describes.

6 min readFor SREs documenting a system with a recurring incident

See it as a diagram

Everything below, as a diagram you can edit. Describe yours and see it in seconds.

183/20003 credits left
Try:

No account needed · Editable canvas, not a picture

The map on-call actually needs at three in the morning

A responder woken by a page needs three answers fast. What does this alert actually mean in terms of components, what could have caused it, and what is already broken for users that nobody has reported yet. A complete architecture diagram answers none of those quickly, because the responder has to find the relevant subgraph first.

Draw the subgraph instead. Start at the alerting component, walk one or two hops upstream to the things that can cause it, walk downstream to what degrades when it fails, and stop. Everything else on the estate is noise at three in the morning, and noise is measured in minutes of mean time to recovery.

Put the owning team on every node. Escalation is the slowest part of most incidents that involve more than one team, and it is slow because the responder is searching a wiki for who owns a queue rather than because anybody is unreachable.

Draw it during the review, while it is still true

The best moment to produce this diagram is the incident review, when the whole failure path has just been reconstructed by people who were there. The reconstruction happens whether or not anyone writes it down; the only question is whether it survives past the meeting.

One diagram attached to the review, showing the path the failure actually took, is worth more than the timeline section that most reviews spend an hour on. It is also the artefact most likely to be read by the next responder, who will not read a document but will look at a picture.

Annotate the diagram with the incident specifics: which edge was saturated, which retry storm amplified, which cache was cold. Those annotations are what turn a topology into a story, and stories are what people remember at three in the morning.

Attach it, do not file it

A diagram in a folder nobody opens is not documentation. Put the embed in the runbook page and the alert description, so it arrives with the page rather than being looked for.

Keep it attached to the runbook

The failure of incident documentation is almost never that it was not written. It is that it was written in one place and the responder is looking in another. Whatever the page routes to, the runbook page, the alert description, the service catalogue entry, that is where the diagram belongs.

An embed makes that practical, because it is a permanent URL rendering the current diagram. One markdown image line in the runbook and the picture stays right without anyone re-exporting it after the fix ships. Confluence, Notion, a docs site, or the repository all take the same line. See live embeds for the mechanics.

One operational detail worth knowing before a wiki migration: embeds depend on the project being publicly viewable, and turning that off stops every embed of it at once. If you are tightening sharing across a workspace, check which runbooks would go blank first.

When the fix lands, the diagram should change on its own

The reason incident diagrams rot is that they document a shape somebody is actively working to remove. The retry logic gets fixed, the queue gets a dead letter, the service gets split, and the diagram in the runbook still shows the old failure path, which makes the next responder chase something that no longer exists.

A repository connection removes most of that lag. The branch is re-read about once a day, the diagram and the architecture document regenerate, and commits that change nothing structural leave the picture alone, so it does not churn every time somebody bumps a dependency. Nodes you positioned by hand during the review keep their position, which matters here more than elsewhere: the responder learned that layout under stress.

The pin is measured rather than inferred. The sync records where it left every node, and anything more than 12 pixels from that spot at the next run counts as a deliberate move and is put back where you put it, while a node still sitting on its computed coordinates stays free to re-lay out as the system grows. The regeneration is also told not to rename zones, re-split them, or change the reading direction, which is the part that would otherwise force a responder to re-learn the picture at three in the morning.

For the parts of the failure path that are not in code, the alert thresholds, the manual failover step, the third-party dependency with no status page worth reading, keep them as annotations on the canvas. Those are the parts a sync will never know about and the parts that cause the next incident.

FAQ

What should an incident diagram show that a normal architecture diagram does not?

The failure path rather than the topology. Start at the component that alerts, include one or two hops upstream that can cause it and the downstream services that degrade when it fails, and stop there. Add the owning team on each node, because escalation is usually the slowest part of a multi-team incident.

When is the best time to draw it?

During the incident review, while the failure path has just been reconstructed by the people who were there. The reconstruction happens anyway; drawing it is what makes it survive the meeting and reach the next responder. Annotate it with the specifics while they are fresh: which edge saturated, which retry storm amplified, which cache was cold when the page fired.

Where should the diagram live so people actually see it?

Wherever the page routes to: the runbook page, the alert description, or the service catalogue entry. An embed is a permanent URL that renders the current diagram, so one markdown image line in the runbook keeps the picture correct without anyone re-exporting after a fix ships.

How do we stop the diagram showing a failure mode that has been fixed?

Ground the structural part in the repository. A read-only connection re-reads the branch about once a day and regenerates the diagram, while commits that change nothing structural leave it untouched. Nodes moved by hand keep their position, so the layout responders learned does not shift under them.

What about the parts of the failure path that are not in the code?

Keep them as annotations on the canvas: alert thresholds, manual failover steps, third-party dependencies with unreliable status reporting. A repository sync cannot know about any of those, because none of them are declared in the code it reads, and they are frequently the parts that cause the next incident rather than the topology that is drawn correctly.