Databricks Unity Catalog diagram: system tables hold the object graph and the lineage

Each catalog carries an information_schema scoped to itself, and the system catalog carries one that spans the metastore: system.information_schema.tables lists every table and view Unity Catalog governs, across all catalogs, with type, owner, and timestamps. That single query is the node list, no per-catalog looping required, which is precisely what the per-dataset scoping in BigQuery makes you script around.

-- Nodes: everything Unity Catalog governs, all catalogs at once
SELECT table_catalog, table_schema, table_name, table_type
FROM system.information_schema.tables
WHERE table_catalog <> 'system';

-- Edges: reads and writes recorded from actual runs
SELECT DISTINCT source_table_full_name, target_table_full_name, entity_type
FROM system.access.table_lineage
WHERE event_date >= current_date() - INTERVAL 30 DAYS
  AND source_table_full_name IS NOT NULL
  AND target_table_full_name IS NOT NULL;

system.access.table_lineage is not parsed from view definitions: rows are written when queries actually execute, whether from a notebook, a job, a pipeline, or a SQL warehouse, and the entity_type column says which. Column-level detail lives beside it in system.access.column_lineage. Edges earned from execution are more truthful than edges parsed from SQL text, because they capture notebooks and jobs that no view definition mentions.

The same property is the caveat: a path that has not run recently has no recent rows. Window the query too tightly and a monthly job's edges vanish from the diagram; window it too wide and long-deleted flows linger. Pick a window that covers your slowest real cadence, and treat an edge that ages out as a prompt to ask whether the flow still exists.

Tables alone make a warehouse map, not a platform map. The Jobs API, GET /api/2.1/jobs/list, returns every job with its schedule and tasks, and those are the moving parts readers ask about: what runs nightly, what writes gold, what breaks when a cluster is resized. Because table_lineage rows carry the entity that performed each read and write, you can join jobs to the tables they touch and draw each job as a node with its true inputs and outputs, rather than guessing from naming conventions.

The catalog and schema hierarchy is the zoning: medallion layers as schemas within a catalog, or per-domain catalogs with bronze, silver, and gold inside each. Map the hierarchy onto nested zones and the diagram matches the access model your grants already enforce. The broader platform picture around it is covered in Databricks data platform diagrams.

The redraw is a scheduled job like any other on the platform: after your nightly pipelines complete, a job runs the queries above and hands the result to an agent connected to Datadef's MCP server, registry name io.datadef/mcp, which updates the diagram in place using an API key created in settings, on paid plans. Datadef does not connect to Databricks and does not poll the metastore; your job is the trigger and the agent call is the redraw. First diagram from a description instead: the Databricks diagram generator.

System tables are not on by default: an account admin has to enable the relevant schemas on the metastore before system.access.table_lineage returns anything. And lineage only covers what Unity Catalog governs, so a job writing to an external service or reading over JDBC from an outside database leaves no lineage row; those edges come from the job code, which an agent can read, not from the system tables.

The surface limits apply as everywhere: a live-embedded diagram requires the project shared public, which a governed lakehouse team may reasonably decline, and the loop maintains the diagram rather than the documentation prose around it.

• Databricks diagram generator
• Databricks data platform diagrams
• Keep a data catalog in sync
• The Datadef MCP diagram server