Data Lineage Guide

How to document data lineage without a data catalog

By the engineer who builds Datadef, from client work on data platforms · Reviewed August 21, 2026

Catalog procurement takes quarters, and the audit, the migration, or the argument about a KPI is happening this month. Documenting lineage by hand is a legitimate answer to that, as long as you accept the two rules that make it survive: scope it by question rather than by inventory, and put the artifact where the people who change the pipelines already work.

Try it free
7 min readFor data teams who need lineage documented before any platform is bought

See it as a diagram

Everything below, as a diagram you can edit. Describe yours and see it in seconds.

156/2000• 3 credits left
Try:

No account needed · Editable canvas, not a picture

Start from the question, not the inventory

The failure mode of manual lineage is trying to cover everything. A warehouse with 400 tables produces a documentation project that never finishes and a spreadsheet nobody trusts by month three. Inventory is what a catalog is for, and a catalog is what you do not have yet.

Instead, write down the questions that made someone ask for lineage. In most teams there are two or three: prove where the numbers in this regulated report come from, work out what breaks when we migrate this source, settle why two dashboards disagree about the same metric. Each question implies a small number of flows, usually between five and fifteen, and those flows are your scope.

The nice property of question-first scoping is that it has a finish line. When someone outside the data team can answer the question without asking you, that flow is documented. Everything else can wait for automation.

Document flows end to end, not hop by hop

A per-object description of upstream and downstream neighbours is easy to write and nearly useless to read, because the reader has to reassemble the journey themselves. The artifact that works is one picture per flow, running from the system where the data is created to the surface where a human sees it, including the hops that are not in the warehouse.

Four facts per hop are enough: the system, the object inside it, what the transformation does in one line, and who owns it. Anything more detailed belongs in the code. Anything less detailed and the reader still has to ask a person, which is the situation you were trying to end.

Include the boundaries explicitly. The hops where data crosses from one system or one team to another are where breaks happen, where automated lineage gives up, and where the owner question has a non-obvious answer. Those are the rows people come back to read.

Flow: monthly revenue report

  hop  system        object              transformation            owner
  1    Stripe        charges (API)       raw ingest, hourly        Data Eng
  2    Snowflake     raw_charges         append only               Data Eng
  3    dbt           stg_charges         currency normalised       Analytics
  4    dbt           fct_revenue_daily   sum, refunds excluded     Analytics
  5    Looker        Revenue dashboard   MTD calc in the tile      Finance

Boundary hops: 1 (external API), 5 (logic outside the warehouse)

Put the artifact where it will actually be maintained

A document in a shared drive is stale by the next merge because nothing in the change process points at it. Two placements work: next to the code that defines the flow, so a pull request touching the model touches the map, or embedded in the page the readers already open, so its age is visible to the people who would notice it is wrong.

For teams with nothing bought yet, the free scratch canvas is the fastest start: it is the full editor, no account needed, and pasting the SQL behind a model gets the tables and columns drawn instead of retyped. Work saves in the browser and can be claimed into an account later. Diagrams export as PNG or JPEG for a slide deck, and on paid plans a live embed puts the current canvas straight into a Confluence page, a Notion doc, or a README with one markdown line, so the wiki stops carrying a screenshot from last spring.

Owner and team belong on the nodes, not in a legend. A node that carries its owner, a link to the repo that defines it, and a link to the runbook turns the map into a routing table, which is the feature people keep coming back for long after the audit is closed.

Manual today does not mean manual forever

The flows whose definitions live in git can be regenerated instead of maintained. See how repository sync works for the parts you can stop editing by hand.

Fill the fields that turn a picture into a routing table

A hand-drawn map earns its maintenance only if it answers who and what, not only where. Each node on a Datadef canvas carries a label, which should be the real object name rather than a friendly one, a one-line description of what it is and why it exists, and three governance fields: dataOwner, dataSteward, and teamInCharge. Table nodes carry their columns with a type and primary or foreign key flags. Pipeline nodes carry a technology and a schedule, so the job between two tables reads as dbt, 0 2 * * * instead of as a bare arrow.

validate_canvas is the cheap review pass over that work. It reports edges with no label, table nodes with no columns, and a missing-descriptions warning once more than half the nodes have none, which is the state nearly every first draft is in. Clearing those three findings is most of the distance between a picture and a document.

Skip the rest deliberately. Column detail on every table, KPI formulas on every metric, and a steward on nodes nobody disputes are maintenance you have signed up for and will not do, and a field that is wrong reads worse than a field that is empty.

FAQ

Is a spreadsheet good enough for documenting data lineage?

It is good enough to capture the facts and poor at communicating them, because readers cannot see a flow in a table of rows. A workable compromise is to keep the rows as the source of record and maintain one diagram per critical flow for the people who have to read it.

How many flows should I document by hand?

As many as your open questions require, which is usually five to fifteen. Manual coverage of an entire warehouse does not finish and is not trusted by the time it would. Document the flows behind regulated reports, contested metrics, and imminent migrations, and leave the rest to automation later.

Should manual lineage be column level?

Only where a specific field is the subject of the question, such as an audited amount or a customer identifier. Recording every column by hand across a full flow is the fastest way to produce documentation that is wrong within a month.

How do I stop hand-written lineage from going stale?

Give each flow an owner and a review date, show the date on the artifact itself, and place the artifact where the change happens: in the repository next to the code, or embedded in the page people read. Freshness comes from placement and ownership, not from good intentions.

Will this work still count once we buy a catalog?

Yes. The scoping decisions, the owner assignments, and the boundary hops you identified are exactly the inputs a catalog rollout needs, and they are the parts automated scanning cannot infer. Teams that do the manual pass first reach a usable graph faster after the purchase, because the connector work is aimed at flows somebody already judged worth the money.