A seven-layer reference architecture for enterprise agent platforms

AI-generated content: this article was created with AI.

When a large organisation moves from one AI pilot to dozens of agents, the conversation in the architecture board changes. The question is no longer which model performs best on a benchmark, but how many teams can build agents on a shared foundation without each of them solving identity, retrieval, cost allocation and audit on their own. A reference architecture helps to structure that conversation. It names the capabilities, separates them into layers and makes the interfaces between them explicit.

The following frame is vendor-neutral and deliberately coarse. It describes seven layers that we see in every enterprise agent platform that scales and stays auditable.

Layer 1, identity and access: every agent receives its own workload identity, and user context is passed through OAuth 2.0 Token Exchange so that both the person and the agent remain visible. Policies are evaluated centrally rather than coded into each application. This layer comes first because every other layer depends on knowing who is acting and on whose behalf.

Layer 2, model gateway: a single entry point for all models handles routing, fallback, quotas and data classification per request. Applications know endpoints, not vendors. This is what allows a model to be replaced, or a second region to be added, without touching application code.

Layer 3, knowledge and retrieval: ingestion, chunking, embeddings, a vector index and re-ranking. The decisive property is that permissions are checked at retrieval time, for the requesting person, and not only when documents are imported. Otherwise the index slowly becomes a copy of the organisation's data without its access model.

Layer 4, tools and integrations: agents act through tool servers, for example via the Model Context Protocol (MCP). A registry lists every available tool together with an owner, a scope and a rate limit, and new tools pass through an approval process. Without such a registry, the set of actions agents can take grows without anyone having an overview.

Layer 5, agent runtime: orchestration, durable state, retries with idempotency, timeouts and human-in-the-loop steps for consequential actions. Agent logic runs isolated per tenant. This is the layer that turns a script calling a model into a service that can be operated.

Layer 6, evaluation and observability: a golden dataset, evaluations as a gate in the delivery pipeline and traces based on OpenTelemetry. Quality, latency and cost become visible per call and per step. The OpenTelemetry semantic conventions for generative AI are still in development, so it is sensible to plan for attribute names to change.

Layer 7, governance, cost and audit trails: usage metering per cost centre, model allowlists and immutable logs. In practice, this is the layer that internal audit and finance ask about first, and it is often the one that was planned last.

The real value of the frame lies in the interfaces between the layers. If the runtime talks to models only through the gateway, a model change is a configuration change. If retrieval accepts identity only from the token and never from the prompt, permission enforcement does not depend on every agent developer getting it right. If the gateway emits metering data in one format, chargeback becomes a query. Stable interfaces are what allow components such as a model or a vector database to be replaced without rebuilding agents.

There are limits. This is a thinking frame, not an organisational chart. Seven layers do not mean seven teams, and in smaller platforms several layers are often run by one group. The layers also differ in maturity. Identity and gateways rest on established patterns, while evaluation practice and observability conventions for generative AI are still evolving. Finally, the frame says nothing about which use cases are worth building. That remains a business decision.

For an architecture team, a practical starting point is a one-page map. List the seven layers and, for each, note what exists today, who owns it and which interface applications use. Mark where projects bypass a layer, for example by calling a model directly or reading from an index without a permission filter. Then decide which interfaces you want to fix first. In our experience, the gateway interface and the identity propagation pattern create the most leverage, because almost every other layer depends on them.

cloudstrata builds and operates AI platforms for enterprises. AI Foundry is our managed runtime for AI agents and covers multi-tenant workspaces, routing of models and tools, knowledge retrieval, evaluation hooks, usage and cost metering, audit trails and central governance. To compare this reference architecture with your own landscape, you can request a technical introductory conversation via cloudstrata.io/contact.