Data Pipeline Guide

Airflow DAG diagram conventions: the graph view is not an architecture diagram

By the engineer who builds Datadef, from client work on data platforms · Reviewed August 21, 2026

The Airflow graph view is excellent at what it was built for, which is showing task instances and their dependencies for a single run. It is a poor architecture diagram for the same reason terraform graph is: it renders the execution structure rather than the systems the execution touches, and it gets unreadable exactly when the DAG gets interesting. A diagram meant for humans needs a different set of conventions, and most of them are about deciding what not to draw.

Try it free
6 min readFor teams whose Airflow deployment has outgrown the graph view

See it as a diagram

Everything below, as a diagram you can edit. Describe yours and see it in seconds.

234/2000• 3 credits left
Try:

No account needed · Editable canvas, not a picture

What the graph view cannot tell you

The graph view answers what ran and in what order. It cannot answer which systems this DAG touches, because an operator name like PythonOperator or KubernetesPodOperator says nothing about the database it queries. It cannot answer what happens if a source is late, because sensors and timeouts are properties rather than shapes. And it cannot show cross DAG relationships at all in the classic trigger model, which is where most real dependencies hide.

It also degrades badly at scale in a specific way: dynamic task mapping renders every expanded instance, so a DAG that maps over 200 partitions shows 200 nodes that are all the same node. That is correct for monitoring a run and useless for understanding the pipeline.

So the architecture diagram is a second artifact with a different subject. The graph view is about the run; the diagram is about the systems and the data.

The conventions

One zone per DAG, with the DAG id and the schedule on the zone label. Writing the schedule the way the code writes it, for example 0 2 * * * or @daily, means a reader can grep for it. If the DAG is dataset triggered rather than time triggered, say so on the label, because that is a different operational model.

Nodes are external systems, not tasks. A DAG with fourteen tasks that read one Postgres database, call one API, and write one warehouse schema should show three nodes plus the DAG zone, not fourteen. The unit that matters to a reader is the system, because that is what can be down.

TaskGroups become sub zones when they represent a meaningful stage, and get dropped when they exist only for UI tidiness. The test is whether the group has a name someone would say out loud during an incident.

Dynamically mapped tasks collapse to one node with the expansion count, for example extract_partition, mapped over roughly 200 partitions. The count belongs on the node because it drives concurrency and cost conversations, and the individual instances belong nowhere on a diagram. Where the count is load-bearing, put the ceiling beside it: the core.max_map_length setting caps how many instances an expand may create, defaults to 1024, and makes the source task fail rather than truncate when its list is longer.

Sensors are drawn only when they express a real cross boundary dependency, such as waiting on a file from a partner or a table from another team. A sensor waiting on something inside the same DAG is scheduling detail and should not survive onto the diagram.

ZONE dag: ingest_salesforce   schedule: 0 1 * * *   owner: data-platform
  Salesforce API  -->  s3://raw/salesforce/  -->  RAW.SALESFORCE
  mapped: extract_object x 14 objects

ZONE dag: transform_core     trigger: dataset RAW.SALESFORCE, RAW.APP
  RAW.*  -->  dbt build (tag: core)  -->  ANALYTICS.MARTS

ZONE dag: reverse_sync       schedule: @hourly
  ANALYTICS.MARTS.customer_360  -->  Hightouch  -->  Salesforce

Datasets changed what is worth drawing

Under the classic model, cross DAG dependencies were expressed with triggers and external task sensors, which are control flow and awkward to draw honestly. Data-aware scheduling changed that: a DAG declares the data it produces and the data it consumes, and those declarations are genuine data edges between DAGs. Draw them. They are the most valuable arrows on the whole diagram, because cross DAG dependencies are where the surprises come from.

Use the vocabulary of the version you actually run, because it moved. Airflow 3.0 renamed Datasets to Assets, added an @asset decorator, folded time-based and event-based triggering into a single schedule field, and renamed the matching setting from core.dataset_manager_kwargs to core.asset_manager_kwargs. A zone label reading "dataset dependency" on an Airflow 3 deployment sends a reader looking for a concept the UI no longer calls that.

The practical result is a diagram with two edge types: inside a DAG zone, arrows are the flow of data through external systems; between DAG zones, arrows are dataset dependencies. Two meanings, clearly separated by whether the arrow crosses a zone boundary, and no legend required.

Keep the diagram close to the DAG code so it does not become fiction. Connecting the repository read only lets a daily sync regenerate the diagram and an architecture.md from the branch you nominate, and node identity stays stable across syncs so a re-sync updates the picture rather than reshuffling it. An agent in your editor can check freshness and trigger a refresh after a DAG change merges, through repo_status and repo_refresh, two of the nine outcome-level tools on the MCP server that sit alongside 27 canvas_* tools for node-level edits.

Draw the systems, count the tasks

The rule that fixes most Airflow diagrams: nodes are things that can be down, tasks are things that can be retried. Only the first kind belongs on the canvas.

FAQ

Why is the Airflow graph view not enough as documentation?

It renders task instances and their dependencies for a run, so it shows execution structure rather than the systems involved. An operator name does not identify the database it queries, sensors and timeouts have no visual shape, and dynamic task mapping expands into hundreds of identical nodes. It is built for monitoring a run, not for explaining a pipeline.

What should the nodes be on an Airflow architecture diagram?

External systems rather than tasks. A DAG with fourteen tasks that read one Postgres database, call one API, and write one warehouse schema should be drawn as three system nodes inside one DAG zone. The useful unit is the thing that can be down, not the thing that can be retried.

How do I draw dynamically mapped tasks?

As a single node carrying the expansion count, for example extract_partition mapped over roughly 200 partitions. The count matters because it drives concurrency limits and cost, and the expanded instances carry nothing a reader of an architecture diagram can use. The core.max_map_length setting caps that expansion and defaults to 1024, so write the count on the node when it runs anywhere near that.

How should cross DAG dependencies appear?

As arrows between DAG zones. With data-aware scheduling each DAG declares what it produces and consumes, and those declarations are genuine data dependencies worth drawing. They are usually the highest value arrows on the diagram, since cross DAG dependencies are where the unexpected failures come from. Airflow 3.0 renamed this concept from Datasets to Assets, so match the label to the version you run.

Should TaskGroups be drawn as sub zones?

Only when the group represents a stage someone would name out loud during an incident, such as extract, validate, or publish. TaskGroups that exist purely to tidy the Airflow UI carry no architectural meaning and should be left off.