Where Should Your Agents Live?
The default pattern I see in enterprise agent projects goes like this:
- Spin up an agent runtime in one place — usually wherever the AI team has admin rights.
- Wire in data sources via API, MCP, or a nightly RAG pipeline.
- Run every user query through that central agent.
- Watch the latency, the egress bill, and the governance review queue all light up at once.
It works. It also leaves a lot on the table — and in some industries it isn't actually the safe choice it looks like.
There's an older idea from cloud architecture that explains why, and it's worth dragging back into the agent conversation: data gravity.
Data Has Weight
Dave McCrory coined data gravity back in 2010. The intuition is simple: large datasets are expensive to move, so applications, services, and now agents tend to settle in orbit around them. The bigger the dataset, the stronger the pull.
For agents, gravity comes from three places, not one:
Bytes. Pulling a million records over the network for a single agent reasoning step is slow and expensive. Doing it ten thousand times a day to support an internal tool is worse.
Governance. Data platforms enforce row-level security, masking, lineage, and audit. Once you pull data out, you're rebuilding those guardrails in the agent layer — usually badly.
Recency. Data that's good enough at 8am may be wrong by 9am. The further your agent sits from the source of truth, the more snapshot-driven its reasoning becomes.
A useful way to think about it: any time your agent needs data, something commutes. Either the model commutes to the data, or the data commutes to the model. Pick whoever's lighter.
In enterprise systems, the data is almost always heavier.
The Commute Model
In the far-from-data pattern, the agent makes copies. Every reasoning step pulls data across a network boundary, into a runtime that doesn't share the source's access model.
In the near-data pattern, the heavy work happens where the data already lives, under the data platform's existing trust boundary. The orchestrator only sees what each domain agent chooses to expose.
It's the same pattern that won in microservices: don't send raw data over the wire when you can send a request and get back a result.
Every Hop Is a Copy
Here's the thing nobody puts on the architecture diagram: every network hop between your data and your agent creates another place for data to be cached, logged, or leaked.
A real production trace looks like this. The customer record lives authoritatively in one place. By the time the agent answers, fragments of it exist in a vector store, a tool-call log, the LLM provider's request audit, and probably a tracing system. Each of those needs its own access policy, its own retention rule, its own breach-notification plan.
Putting the agent next to the data doesn't eliminate this problem. It does dramatically shrink the surface area you have to govern.
A Composite Story
A manufacturing company — call them Northwind — ran an internal customer-360 agent for their sales team. Data lived in four places: Salesforce for accounts and opportunities, SAP for order history, Snowflake for product telemetry, and Confluence for spec sheets.
The first build was centralized. A hyperscaler-hosted agent runtime, with a nightly RAG pipeline that synced everything into a vector store next to the agent. Reps would ask "tell me everything about Acme's account before this call."
It worked technically. It failed operationally:
- Latency: 12–18 seconds per query. Reps gave up mid-call.
- Freshness: Order data was up to 24 hours stale because the SAP sync was nightly.
- Governance: Salesforce field-level security wasn't reflected in the vector store. A handful of reps could see commission data they weren't supposed to. Legal noticed.
- Cost: Egress and embedding regeneration were line items nobody had budgeted for.
The rebuild moved the agent topology, not the model. They put a domain agent next to each data source — one inside the Salesforce/Data Cloud boundary, one inside Snowflake, a small service near SAP for orders. A thin orchestrator at the edge composed answers across them.
Latency dropped to 2–4 seconds. Freshness became real-time, because each domain agent queried its own live data. Governance simplified, because each domain agent inherited the source platform's existing access controls. The orchestrator only saw what each domain chose to return, not the underlying tables.
The interesting part: they kept the central runtime for things that genuinely needed cross-source synthesis. They just stopped using it as the default.
Where Each Platform Family Stands
There's no clean winner here. Each family makes a different bet on where gravity lives, and each is honestly best at a different shape of problem.
The big-3 hyperscalers — AWS Bedrock Agents, Azure AI Foundry, GCP Vertex AI Agents. Strongest when your data already lives in their object stores and warehouses (S3 + Redshift, ADLS + Fabric, GCS + BigQuery). The agent runtime sits genuinely close to the data, and the IAM model is consistent end-to-end. Weakest when your data of record lives outside the cloud — in a SaaS like Salesforce or SAP, in on-prem Oracle, in a partner's system. Then "close to the data" quietly becomes "close to a copy of the data," which is a different thing entirely.
Data-platform-native — Snowflake Cortex Agents, Databricks Agent Bricks / Mosaic. Strongest for analytical workloads where the warehouse or lakehouse is already the source of truth. Agents run inside the data platform's trust boundary, queries hit live tables, governance is unified. Weakest for operational systems — they're not where your CRM, your ERP, or your support tickets actually live, so any cross-source agent still has to reach out.
SaaS-data-native — Salesforce Agentforce + Data Cloud, SAP Joule, Oracle's agent stack. Strongest when the system of record is the SaaS itself. The agent inherits the application's permissions, business logic, and audit. Zero-copy federation reduces (but doesn't eliminate) the data-movement problem. Weakest when you need to reason across many domains — a CRM-native agent has to reach out for ERP and warehouse data, with the same trade-offs as anyone else.
The honest summary: whichever platform owns the heaviest dataset in your enterprise has a structural advantage for agents that touch that dataset. Pick based on where gravity actually lives, not on where your AI team is most comfortable.
When Far-From-Data Is Actually Fine
The principle isn't a rule. There are real cases where centralizing the agent is the right call:
- Small, low-sensitivity, low-velocity data. A product catalog of 10K SKUs doesn't have meaningful gravity. Cache it anywhere.
- Truly cross-source synthesis. If the agent's job is to combine three systems that no single platform owns, you need an orchestrator somewhere — and that "somewhere" probably isn't inside any one source.
- Read-mostly, snapshot-tolerant use cases. Internal search, FAQ answering, basic summarization. If yesterday's data is fine, gravity is weaker.
- Building speed. A centralized prototype unlocks the question "is this useful?" faster than a federated one. Just don't accidentally ship the prototype.
The point isn't "always run agents on the database." It's "be honest about where the gravity is, and let that drive the topology."
The Three-Question Test
Before locking in an agent topology, I run three questions past it.
Where does the data of record live?
Not the warehouse copy. Not the vector store. The actual source of truth that, if it disagreed with everything else, would win.
What governance does that source already enforce?
Row-level security, masking, lineage, audit. Whatever the agent doesn't inherit, you're rebuilding — usually with less rigor than the source had.
What's the cost of the data being one step stale?
If the answer is "nothing," gravity is weak and you have flexibility. If the answer is "we mis-quote a customer," gravity is strong and you don't.
If two or three questions point at the same platform, that's where the agent — or at least that agent — should live.
Bottom Line
Agentic architecture is having its "monolith vs microservices" moment. The default — one big agent runtime, data piped in from everywhere — works in demos and falls over in production for the same reasons centralized data architectures fell over: latency, governance, freshness, cost.
The fix isn't a vendor choice. It's a topology choice. Run your agents where the data they reason over already lives. Let the data platform's existing trust boundary do the work it was already doing. Use the central orchestrator for what actually needs orchestration — composing across sources — not for everything.
Data has gravity. Agents do too. They settle most stably when those two pulls point in the same direction.