AI Agents Guide

What an agent can and cannot infer from your code

By the engineer who builds Datadef, from client work on data platforms · Reviewed August 21, 2026

An agent reading a repository is doing static analysis with a language model attached. That combination is much better than either half at recognising structure, and it inherits the hard limit of the static half: code declares what a system is made of, never what it does at three in the morning. Knowing which side of that line a claim falls on is most of what diagram review is.

Try it free
7 min readFor anyone about to publish a diagram an agent produced

See it as a diagram

Everything below, as a diagram you can edit. Describe yours and see it in seconds.

136/2000• 3 credits left
Try:

No account needed · Editable canvas, not a picture

Reliable: what the source actually declares

Infrastructure as code is the strongest signal there is, because it is a complete declaration by construction: what apply will create is what the files say. A Terraform parse reads resources, modules, count and for_each, and references resolved through outputs, with variables followed through call sites, defaults, and tfvars to literal values where they exist. Nothing there is inferred.

Application code is nearly as good for structure. Imports and module boundaries, HTTP route definitions, database schemas and migrations, queue and topic names, client SDK instantiations, scheduled jobs in configuration. These are declarations too, just spread across more files and more idioms.

The dependency manifest is the boundary of the system as the code sees it: every third-party service with a client library is listed there, whether or not anyone remembers it.

Invisible: what no amount of reading will produce

Runtime behaviour, first. Which of the three paths carries the traffic, whether the fallback has ever fired, what the actual latency between two services is, whether the batch job still runs. Static analysis cannot decide these questions even in principle, and a model reading the same files inherits the limitation while sounding more confident about it.

Deployment truth, second. The code declares what could be deployed; it does not know which version is running where, which feature flag is on, which service was decommissioned last quarter with its module left in the repository.

Everything outside the repository, third. The vendor system somebody integrates by uploading a file. The team that owns the downstream warehouse. The manual approval step in the middle of the pipeline. The reason the boundary is drawn where it is. None of that has a file to read.

The guessing zone, and how it looks

Between the two lies the region where models produce plausible content. Four shapes recur. A node that matches the pattern of the architecture but has no file behind it, often a cache or a load balancer that the diagram feels like it should have. An edge inferred from naming rather than from a call, where user-service and user-db are drawn as connected because the names rhyme. A label taken from an old comment after the thing was renamed. And a whole subsystem missing because it lives in a repository the agent was not shown.

All four are cheap to catch with one question, which is worth asking as a routine step rather than a suspicion.

A fifth cause of silent omission has nothing to do with the model. Whatever reads the repository reads a subset of it. A Datadef corpus is at most 40 files and 250KB, 30KB per file with a marker where a longer file was cut, taken class by class in a fixed order: infrastructure code, then containers and orchestration, then the data layer, then API definitions, then CI, then dependency manifests, then the README. Lockfiles, binaries, and anything under node_modules, vendor, dist, target or .terraform never enter, and a single file over 400KB is skipped outright.

Per-class caps decide the rest, and they move with what you asked for. Under an architecture focus, sixteen .tf files can enter and only three dbt models; under a content focus that inverts to three .tf files and eighteen models. A compact summary of the top three directory levels with their file counts rides along with the corpus, so the model can at least see the shape of what it was not given. Knowing those numbers is the difference between "the agent missed the billing service" and "the billing service is two hundred SQL files under a focus that samples three of them".

For every node and every edge in the diagram, give me:
  - the file path that justifies it
  - or the word ASSUMPTION

Then list what you could not determine from the repository.

What to do with the line once you can see it

Treat the declared layer as the base and generate it deterministically where you can. Structure that comes from parsed source does not drift and does not need reviewing for invention: it is what the files say.

Then add the human layer explicitly on the canvas rather than in a paragraph nobody reads: a note on the deprecated path, a zone label naming the owning team, an annotation where the manual step sits, a marked boundary for the vendor system. Those are the facts that make a diagram worth more than the repository it came from, and they are exactly the facts no agent can recover.

For the parts that are declared in infrastructure code, repository sync keeps the base layer current on its own, so the human layer is the only thing anyone maintains by hand.

A useful phrasing for reviews

Ask not is this diagram right, but which parts of this diagram could the agent not have known. The answers are where the errors are. See reviewing what the agent drew.

FAQ

What can an AI agent reliably determine from a repository?

Declared structure: infrastructure resources and their references, module and import boundaries, HTTP routes, database schemas and migrations, queue and topic names, scheduled jobs in configuration, and third-party clients listed in the dependency manifest. These are statements the code makes, not inferences about it.

What can it not determine, no matter how good the model is?

Anything that only exists at runtime or outside the repository: which path carries the traffic, what is deployed where, whether a component is dead code still checked in, the manual step in the middle of a process, who owns a downstream system, and why a boundary was drawn where it was.

How do I spot an invented component in a generated diagram?

Ask the agent to justify every node and edge with a file path or mark it as an assumption. Invented elements tend to be architecturally plausible pieces such as a cache or a gateway, or edges inferred from similar names rather than from an actual call, and both collapse immediately under that question.

Is a parser more trustworthy than a model for this?

For what it covers, yes. A parser that reads infrastructure code only draws what a file declares, so it cannot invent a component, while a model can. The trade is coverage: a parser understands one language or format, and a model can read application code, configuration, and comments that no parser models.

Does the tool read the whole repository?

No, and nothing that fits a repository into a model context does. A Datadef corpus is capped at 40 files and 250KB, 30KB per file, taken class by class in a fixed order with per-class caps that shift with the chosen focus, plus a summary of the top three directory levels so the shape of what was excluded stays visible. A subsystem living entirely in files the budget never reached is absent with nothing saying so.

Does the agent need to run the code to understand the architecture?

No, and for architecture questions execution adds little. What exists and how it wires is declared in source. Execution answers different questions, such as which path is hot and whether the fallback ever fires, which belong to monitoring rather than to a diagram.