/llms.txt is not enough: a discovery suite for agent-facing APIs

The short version

The principle: predictable availability

Agents fail differently from humans. A developer who hits a 401 on /openapi.json walks the URL up the tree, opens a browser, asks a colleague, and recovers in minutes. An agent has none of those affordances. A 401 on a discovery surface looks identical to a 503, and the planning loop has no signal whether to retry, back off, or give up.

Discovery surfaces must be always-on and never gated. Not "usually up," not "behind a feature flag," not "rate-limited at 5 req/min." The moment one of these surfaces is variable, the agent's first failure becomes silent and the operator's debug session begins. Our own version of this lesson came from F-440: a /llms.txt that advertised /docs and /docs/json shipped 404s to any agent that read the link index as ground truth. Closed in v0.20.3 by removing the broken paths and pointing at /openapi.json instead.

Each surface below earns its place by serving a question the agent has at session start. None is interchangeable; none is optional once you decide LLM agents are first-class consumers of your API.

/llms.txt is a curated link index, not a spec dump

The 2026 /llms.txt spec is exactly that: a link index. Agents follow links; they do not ground from prose dumps. The structure that holds up under real agent traffic is short section headers (Onboarding, Authentication, Common workflows, Reference), each with a few lines of prose plus links, never paragraphs of static narrative.

The detail that matters more than the structure is what we now call the Agent Grounding Directive: a small block at the top of the file that tells the agent which response-body fields to consult on each call. It is the difference between a doc that goes stale and a doc that points the agent at fields that cannot. Static enums in a Markdown file go stale every time the API moves a transition or renames a status; response-body fields update with the engine, by construction.

We learned that the hard way. F-428: our /llms.txt had advertised a transition action that we had renamed in the engine. The reporter framed it as a prose-edit bug; we reframed the fix as an architectural change. Static enumeration in a doc file will always lag the engine. The Grounding Directive says "read nextActions, validTransitions, nextSteps, recoveryHint, and refreshUrl on the response body as ground truth" — closed v0.20.2.

The directive only steers behavior if the data it points to is correct. F-431, found during the same sweep: nextActions was returning entries that contradicted the directive's own prose. Fixed v0.20.3 by deriving the field from a single in-engine source. Both findings argued the same point: the directive and the response data have to land together.

Customizable matters too. Operators will need to override sections their auditors rejected, omit irrelevant ones, prepend org-specific context. The pattern that works for us is environment-driven: _OVERRIDE_PATH for full replacement, _PREPEND_PATH and _APPEND_PATH for wrapping, _OMIT_SECTIONS for trimming the token footprint of surfaces a given install does not use. The doc is content, not code.

/openapi.json is the machine-readable contract

The agent will codegen, validate, or introspect against the OpenAPI 3.1 document. Three properties matter more than the rest of what the spec lets you encode.

Byte-stable when nothing has changed. Random key ordering breaks every diff-based caching layer downstream. A stable serializer is non-negotiable. If your JSON output rearranges fields between calls, every consumer who hashes the document for cache keys is forced to refetch on every request.

Every route, including admin and discovery routes. Agents do not know which routes are admin-only until they fetch and read. Hiding admin routes from /openapi.json forces agents to probe blind. The right answer is to publish every route and let the scope model (next section) tell the agent which it can call.

Response schemas declare every field the runtime returns. Most JSON-Schema-validating frameworks silently strip undeclared response fields at serialization time. A schema that is one field behind the runtime is silently lossy in production. The discipline is removeAdditional: false on the request side and explicit response schemas with every field declared on the response side.

One more failure mode worth naming. F-403: /openapi.json declared servers.url=/v1 while the engine was actually mounting routes at the root. Agents codegenned clients that pointed at the wrong base path. Fixed v0.19.16 by removing servers entirely and letting clients default to the request base URL. The lesson: every field in the OpenAPI document is a contract, including the ones that look incidental.

/.well-known/agent-card.json for A2A protocol

The Google A2A AgentCard is the capability advertisement an agent-to-agent client reads before sending a single message. It declares four things: the skills the responder can perform (with tags so peers can filter), the spec-mandated capabilities (streaming, push notifications, state-transition history) plus any implementation-specific flags namespaced under your vendor prefix, the transport binding (JSON-RPC or HTTP base URL, protocol version), and the provider identity (organization name and URL).

The .well-known prefix is not cosmetic. Per RFC 8615, it is the conventional discovery path for capability advertisement; clients can probe it without prior knowledge of the host's URL conventions. Adopting the convention is what makes your endpoint discoverable by an A2A client written by a third party.

A practical detail: the AgentCard is a good place to surface the supported intents your service accepts (the verbs a peer can call against you). Forcing peers to probe for unknown methods is the same anti-pattern as hiding admin routes from OpenAPI. The card declares it, the routes enforce it, and the agent never has to guess.

/v1/scope-profiles for permission self-introspection

Most APIs make agents guess at scope strings. A discoverable scope-profile catalogue lets the agent answer three questions at session start: which scopes does my key have, which named profiles are available so I can request one by name rather than enumerate scopes, and which roles each profile is allowed for so a scope-escalation attempt fails fast with a meaningful error.

We have hit the failure mode of mismatched advertisements twice. F-438: an admin-default profile advertised one scope but the route enforced a stricter "platform role required" check. Admin keys went into a 403 retry loop they could not escape without out-of-band guidance. Fixed v0.20.3 by aligning the enforcement to the advertised profile. F-426: agent-role keys could not read the trail on records they were a party to, because the profile was missing the audit:read scope. Fixed v0.20.1 by re-granting the scope and enforcing party-scope at the handler layer.

Both findings argue the same point: the catalogue is the contract. A 403 the agent cannot self-diagnose is a product bug, not a security feature. If the profile says you have the scope, the route must accept the call.

The pattern that worked for us: a single declaration is the source of truth, the discovery endpoint projects from that declaration at module load, and route-level scope assertions read from the same declaration. The agent calls GET /v1/scope-profiles once at session start, picks a profile, and the rest of the session is ground truth.

/v1/verification-keys for offline verification

For APIs that emit signed payloads, including audit chains, federation messages, and signed receipts, a verification-keys endpoint lets a verifier check signatures without trusting the running server. The keys are fetched once, cached, and verification proceeds offline against a static dump of the signed data.

Be honest about scope. Most APIs do not need this. If your service is not emitting signed artifacts that a third party will verify offline, you do not need a /v1/verification-keys endpoint. We list it because the post is laying out the full discovery suite for agents that need it. We expose it because we ship a hash-chained, Ed25519-signed audit trail and the verifier has to be able to check signatures against the engine offline.

Never gate discovery

Every surface above is unauthenticated and always-on. This is non-negotiable. The moment one of them returns 401 or 5xx behind a flaky front door, the agent's planning loop breaks and the failure is impossible to diagnose because every other surface still works. The agent has no model of "discovery is sometimes up." It assumes discovery is the floor.

One defensive note. A Swagger UI mounted at /docs is gateable; that is fine. Swagger UI is a human tool. A developer browses it, an agent does not. Gating Swagger UI behind an env flag in production is reasonable. Gating /openapi.json is a bug. The discovery suite is not Swagger.

When this matters

A discovery suite earns its keep when LLM agents are first-class consumers of the API. For human-only APIs, most of these surfaces are noise: developers will read your README, not your scope-profile catalogue, and they will accept a Swagger UI that is occasionally down for maintenance.

The bar shifts when planning loops are running against your endpoints. Agents need to ground without prior context, read the contract without guessing, choose a permission model without enumerating, and verify outputs without trusting the server. Each surface answers one of those questions. The principle that ties them together is predictable availability. None of this works if any one of them is sometimes there and sometimes not.

Related

For the AGLedger-specific projection of these surfaces, see our customer documentation: Discovery Surfaces. Two adjacent topics live in the same docs section: Response Envelope (the response-body fields the Grounding Directive points at), and Thin-SDK Pattern (why the SDK depends on the API for live docs rather than embedding them).