Airflow Guide

Diagram an Airflow repository: the platform around the DAGs, not the task graph

By the engineer who builds Datadef, from client work on data platforms · Reviewed August 21, 2026

Airflow already draws the task graph. The Graph view does it, and airflow dags show renders it to a file. What no Airflow surface draws is the platform the DAGs run on: which executor, which broker, which metadata database, which object store the tasks read and write, and how the DAG folder gets to the workers. That picture lives in the repository, in the compose file, the chart values, and the CI pipeline.

7 min readFor data platform engineers who own the Airflow deployment, not just the DAGs

See it as a diagram

Everything below, as a diagram you can edit. Describe yours and see it in seconds.

186/20003 credits left
Try:

No account needed · Editable canvas, not a picture

What a sync finds in an Airflow repo, and what it does not

The file selector classifies by what a file declares, not by what language it is written in. In an Airflow repository that means the docker-compose file, every Dockerfile, the YAML under a chart or manifest directory, the CI workflow, pyproject.toml, any .sql the tasks execute, and the README all land in the corpus. The ceilings are per class: 6 compose files, 6 Dockerfiles, 12 chart or manifest files, 8 CI pipelines, 8 dependency manifests, and they rise to 8, 8, 14 and 10 when the diagram is set to the architecture focus.

The DAG modules themselves do not. Python files are classified only when they sit under a migrations directory, so dags/etl_daily.py is not fetched. What the generator sees of that folder is the tree summary line, dags/ (43 files), alongside every other directory to three levels with its count. The DAG folder registers as a large body of work whose internals were not read.

That boundary is worth stating rather than hiding, because it decides which question this diagram answers. Task dependencies inside one DAG are an Airflow concern with good native tooling. What runs Airflow, what it depends on, and where it deploys is a repository concern with almost no tooling, and that is the gap the sync fills.

Airflow repo, what the classifier takes
  docker-compose.yaml            containers class
  Dockerfile, Dockerfile.worker  containers class
  charts/airflow/values.yaml     manifests class
  .github/workflows/deploy.yml   CI class
  pyproject.toml                 dependency manifest class
  include/sql/*.sql              SQL class
  README.md                      prose class

Seen only as a directory count
  dags/  plugins/  tests/        (Python modules are not classified)

The diagram that comes out

From the compose file and the chart values, the generator has the real component list: scheduler, API server or webserver, triggerer, workers, the executor in use, the Postgres metadata database, and Redis when the deployment uses Celery. Those become nodes with their real icons rather than a generic box labelled Airflow.

From the service environment and the SQL, it has the edges that leave the cluster: the object storage bucket for logs and staged files, the warehouse the operators query, the secrets backend, the API the sensors poll. Those are the connections people actually ask about during an incident, and they are declared in exactly the files the selector prioritized.

From the CI workflow, it has the delivery path: how the DAG folder reaches the deployment, whether through an image build, a git-sync sidecar, or an object storage upload. A synced DAG folder and a baked image are very different operational stories, and the pipeline file is where the difference is written down.

Which focus to pick on the first sync

The first sync counts what it classified and proposes a focus in one deterministic sentence, composed without a model call. An Airflow platform repo typically reads as: Mostly Docker Compose files and CI pipelines (11 files) with 4 data model and API files. That leans toward the architecture focus, which promotes infrastructure, containers, CI, and manifests to the front of the fetch order and samples SQL only lightly.

Pick the content focus instead when the repository is mostly transformation SQL that Airflow happens to orchestrate. It reverses the ceilings, pulling many more SQL files into the corpus and drawing the data path rather than the runtime. The choice is stored on the connection and every later sync keeps it.

Poetry and uv lock files are skipped as lockfiles, and __pycache__ and .venv never enter the walk, so a repo with a committed virtual environment does not waste any of the 40-file budget.

For the DAG-shaped diagram

When what you need is the task graph as a presentable picture rather than the platform, describe it and generate it directly with the Airflow DAG diagram generator.

After the first draw

The connection re-syncs daily and skips commits that did not change the branch head, so a repository where only DAG code moved gets redrawn only when the platform files move with it. The diagram stays editable on the canvas, exports as PNG or JPEG, and can be embedded as a live image in the runbook or the README so on-call reads the current version.

For an orchestration platform, that last part is the point. A deployment diagram in a wiki page is trusted exactly as long as nobody has changed the executor since it was drawn.

FAQ

Does the sync read my DAG Python files?

No. The file classifier picks up Python only under a migrations directory, so DAG modules are not fetched. The generator sees the dags folder as a directory line with its file count in the repository tree summary, and builds the diagram from the compose file, chart values, CI pipeline, SQL, and dependency manifest instead.

So what does the resulting diagram show?

The Airflow platform and its surroundings: scheduler, workers and the executor in use, the metadata database, the broker where one exists, the object storage and warehouse the tasks connect to, and the pipeline that delivers the DAG folder to the deployment.

Where do the Helm chart files come from?

YAML is classified as a deployment manifest when it sits under a directory named k8s, kube, kubernetes, manifests, charts, or helm. An Airflow chart under charts/airflow, including its values file and templates, matches that rule and is fetched at container priority.

Can I get the task graph of one DAG instead?

Not from the repository sync, and Airflow already does it well through the Graph view and the airflow dags show command. For a presentation-quality DAG picture, describe the pipeline to the generator and edit the result on the canvas.

Does it need access to a running Airflow instance?

No. The connection is read-only access to the repository on GitHub, GitLab, or Azure DevOps. Nothing is executed, no Airflow REST API is called, no scheduler or metadata database is contacted, and no credentials to the deployment are involved. That is what lets the diagram be produced for a branch under review, or for a deployment you have the code for but no access to.