Every enterprise building AI and data infrastructure today is fighting the same silent budget leak: workloads run on the wrong engine, prompts run on the wrong model, and both decisions are made once, statically, and never revisited. As AI moves from experimentation to mission-critical, physical, and edge deployment, that static approach becomes a structural cost and performance liability.
Zetaris has built what we call The TCO Machine — a real-time control plane that continuously matches every query to its most cost-effective processing engine and every prompt to its most cost-effective LLM, while pushing as much computation as possible back to the systems where the data already lives. It does this across on-premises, cloud, and edge infrastructure simultaneously, and it learns and documents the optimal processing pattern for each customer’s unique workloads and operating environment rather than applying a generic and inefficient template.
This paper explains the problem the TCO Machine solves, how its two core mechanisms — Query Director and multi-LLM routing — work together, why pushing processing to the source is the single highest-leverage lever for cost reduction, and how the system compounds its advantage over time by discovering and documenting the lowest-TCO patterns unique to each customer.
Enterprises did not get an AI and data cost problem because they chose bad platforms. They got a cost problem because every platform they chose is good at some things and expensive at others, and nothing in their stack decides, query by query and prompt by prompt, which engine or model should actually do the work.
On the data side, a modern enterprise typically runs Spark, Trino, Presto, DuckDB, a cloud data warehouse, and several operational databases, each with distinct strengths for trillion-row joins, low-latency lookups, streaming aggregation, or ad-hoc analytics. When a query lands on the wrong engine, that engine does not fail gracefully — it over-provisions compute and memory to force the job through, and the enterprise pays for it. Zetaris’ research shows that running the wrong query on the wrong engine, or centralizing every query on a single warehouse “just in case,” is responsible for 30–60% of avoidable compute cost in large data estates, and 30–40% of data spend overall is pure waste created by duplication, migration, and one-size-fits-all engine selection (Zetaris — Why Data Sovereignty Matters for AI Innovation; Zetaris).
On the AI side, the same mismatch is now happening with LLMs, and it is compounding faster. A single enterprise prompt often contains multiple distinct sub-tasks — a data retrieval instruction, a narrative explanation, a code generation request, a compliance check — each with a different optimal model. Sending the whole prompt to one large, expensive, general-purpose model wastes tokens on tasks a smaller or cheaper specialized model could handle just as well, while a jurisdiction-restricted or sensitive task may need to avoid that model entirely for compliance reasons. Static, whole-prompt routing to a single LLM instance is the AI-era equivalent of forcing every SQL query onto one warehouse.
Doing this in real time is what actually matters, and the stakes are rising quickly for three structural reasons:
· AI is becoming mission-critical. Decisions that used to be advisory — fraud flags, clinical triage, network routing — are now operational, which means routing errors and cost blowouts are no longer back-office problems; they show up in SLAs and safety margins.
· AI is becoming physical. As inference moves into robotics, industrial control, and autonomous systems, the cost and latency budget for a decision is measured in milliseconds and watts, not just dollars — you cannot batch-process a decision that a machine needs right now.
· AI is moving to the edge. When inference has to happen close to the data source to avoid network latency, there is no single “right” infrastructure choice anymore — the right engine or model might be on-prem, in the cloud, or on a device at the edge, and it can change from one moment to the next depending on where the data and the demand are.
As Zetaris CEO Vinay Samuel put it in a customer deployment context: “To create edge AI experiences that mimic human experiences, you need to beat physics. You need to have the data close to the experience”. A static architecture decided once at design time cannot beat physics. Only a system that makes the workload-to-engine and prompt-to-model decision in-flight, every time, can.
Zetaris (the Zetaris AI Data Harness) solves this with a control-plane architecture built around two coordinated routing mechanisms that together form The TCO Machine.
Query Director is Zetaris’ real-time, multi-engine query optimizer. It sits above an organization’s existing platforms — Spark, Trino, Presto, DuckDB, Databricks, Snowflake, Teradata, and more — and evaluates every query before it runs: how large the join is, how complex the plan is, whether it needs streaming or parallel execution, and what each candidate engine would cost to execute it. It then routes the query, or sub-queries within it, to the engine best suited to that specific workload (Query Director — Zetaris Knowledge Base; Zetaris Cloud).
As Samuel explains it: “You look at the query in flight and you can work out the cost of that query before you run it and put it on the right platform from a cost and performance point of view” (The Hidden Cost of Data Platforms). At one large media services customer, this real-time matching alone delivered a 40% reduction in total cost of query by eliminating the over-provisioning that occurs whenever a workload lands on a platform that is not built for it (Why Data Sovereignty Matters for AI Innovation).
Critically, Query Director does not require replacing anything. It is a coexistence layer that sits alongside Databricks, Snowflake, Dremio, Starburst, and legacy warehouses, extending the value of investments customers have already made rather than asking them to rip anything out (Zetaris).
Zetaris applies the identical philosophy to inference. Its multi-LLM harness treats a user or agent prompt the way Query Director treats a SQL statement: it decomposes the prompt into constituent micro-prompts — a data-retrieval component, a narrative-explanation component, a code-generation component, a compliance-check component — and evaluates each one against a live registry of available LLMs, scored on token cost, latency, quality for that task category, and compliance or jurisdictional constraints.
Each micro-prompt is then routed to whichever model — small or large, on-prem, cloud-hosted, or edge-deployed — is the best fit for that specific sub-task, rather than sending the entire prompt to one expensive, general-purpose model by default. The system tracks token usage, latency, and quality outcomes per model and per micro-prompt type, and uses that feedback to continually refine future routing decisions, so the same category of prompt gets progressively cheaper and faster to serve over time. This routing intelligence is the subject of a Zetaris patent-pending innovation, the Multi-LLM Micro-Prompt Routing System for Performance, Cost, and Inference Management.
The reason Query Director and multi-LLM routing matter together, rather than separately, is that Zetaris treats infrastructure location as just another routing variable. The same real-time decision engine that chooses between Spark and Trino for a query chooses between a cloud-hosted frontier model and a small on-device model for a prompt — and it can mix on-premises, cloud, and edge infrastructure within a single request. Zetaris’ AI Data Harness is explicitly built as a “hub and spoke” model where data and processing can be distributed across cloud, data centers, and edge locations and still be queried in a single, real-time federated request (Modern Lakehouse for AI; Equinix case study). That is The TCO Machine: a single control plane that keeps every workload and every prompt on its lowest-cost, best-performing home, wherever that home physically sits, updated continuously as conditions change.
Real-time routing decides which engine or model should do the work. The second, complementary lever in the TCO Machine decides where the work should physically happen — and Zetaris’ answer is: as close to the source system as possible.
Rather than centralizing data into a warehouse or lake before it can be queried, Zetaris pushes query computation directly to the source system through its native query API. Only the result set travels back — the raw data itself never has to move (Zetaris Metadata Lakehouse). This is query-in-place, zero-copy federation, and it attacks TCO on several fronts at once:
· No ETL, no duplication, no egress fees. Eliminating the pipelines that copy and re-copy data between systems removes both the infrastructure cost of those pipelines and the storage cost of the duplicate data they create. Zetaris customers see this reflected in a documented 40% reduction in data infrastructure cost (Zetaris).
· Governance and quality enforcement happen at the source, not after the fact in a downstream copy, which removes an entire category of reconciliation and rework cost (Zetaris Technical Brochure).
· The same principle collapses AI inference cost. For agentic AI and retrieval-augmented generation, the most expensive part of a request is often not the LLM call itself but everything done to prepare context for it. By resolving as much of the query as possible in the federated data engine — filtering, joining, and aggregating before anything is handed to a model — Zetaris minimizes the volume of data and the number of tokens that ever need to reach an LLM in the first place. Less context sent means fewer tokens billed, on every single inference call, at scale.
· An Intelligent Adaptive Cache compounds the effect. Zetaris automatically detects frequently accessed dimension and fact tables from observed query patterns and caches them locally, so repeated queries stop consuming full compute on the source system at all (Crush Your Cloud Costs with Zetaris).
Combined with Query Director, this push-down architecture is what allows Zetaris to advertise compute reductions of up to 60%, query performance improvements of up to 10x, and — in Samuel’s words — a total cost of ownership that runs at roughly a third of comparable competitor platforms (Zetaris; AWS Marketplace listing; Why Data Sovereignty Matters for AI Innovation).
The final, and in many ways most durable, part of the TCO Machine is that it does not converge on one universal “best practice” and stop. Every customer has a different mix of source systems, workload shapes, compliance boundaries, and physical topology, so the lowest-TCO processing pattern is different for every environment — and it changes as that environment changes. The TCO Machine is built to discover that pattern automatically and to document it as a durable, governed asset rather than tribal knowledge trapped in a routing black box.
This happens through several coordinated mechanisms:
· Automatic pattern detection from live query behavior. The Intelligent Adaptive Cache does not require an administrator to specify which tables matter — it determines dimension and fact tables directly from the customer’s own query patterns and adapts caching accordingly, meaning the “shape” of the optimization is derived from that specific customer’s actual workload, not a generic default (Crush Your Cloud Costs with Zetaris).
· The AI Data Steward turns discovered structure into a governed, documented asset. Rather than a team hand-building a semantic layer over months, the AI Data Steward stands up a logical semantic layer in minutes, auto-generates business rules, data-quality checks, and KPIs mapped to the exact columns they govern, and builds column-level lineage from raw source to semantic layer. It captures the why behind those decisions in a business-context LLM wiki that is linked and citation-backed, so the reasoning behind the customer’s specific processing pattern is preserved and auditable, not just its output (AI Data Steward — Zetaris; LinkedIn — AI Data Steward capabilities).
· The inference management layer closes the loop on the LLM side. For every micro-prompt routed to an LLM, the system records token usage, latency, and quality outcomes, and feeds that history back into future routing weights — meaning the platform’s model-selection policy for a given customer’s prompt categories is itself a learned, documented artifact specific to that customer’s actual usage, not a static configuration set once at implementation.
· The result compounds instead of decaying. Because both the engine-routing pattern and the model-routing pattern are continuously re-learned from live behavior rather than fixed at go-live, the system’s TCO advantage does not erode as the customer’s data estate, workload mix, or model landscape evolves — it re-discovers the optimum and re-documents it automatically.
In effect, every Zetaris deployment produces its own operating manual for the lowest possible TCO in that customer’s specific problem space — discovered from real workloads, enforced through governed automation, and kept current without manual re-tuning.
The cost problem in enterprise data and AI was never really about which platform to buy. It was about the absence of a system that constantly, automatically matches the right workload to the right engine and the right prompt to the right model — in real time, across on-prem, cloud, and edge infrastructure — as AI shifts from experimental to mission-critical, physical, and edge-deployed.
The TCO Machine is Zetaris’ answer to that problem: Query Director and multi-LLM routing acting as one real-time control plane, a push-down architecture that keeps processing as close to the source as physics allows, and a self-documenting learning loop that discovers the lowest-TCO pattern unique to each customer’s environment and keeps it current. Layered on top of the platforms customers already run — not in place of them — it is how Zetaris delivers measured outcomes of up to 60% lower compute cost, 40% lower data infrastructure cost, and total cost of ownership at roughly a third of comparable alternatives, while giving customers a governed, documented record of exactly why their environment runs the way it does.
Sources
· Zetaris — The Open Agentic Lakehouse for Enterprise AI
· Zetaris Cloud — Deploy a Live Data Lakehouse in Your Cloud Account
· Zetaris Metadata Lakehouse — Query Enterprise Data Without Migration or Egress Costs
· Query Director — Zetaris Knowledge Base
· Why Data Sovereignty Matters for AI Innovation — Interview with Vinay Samuel
· The Hidden Cost of Data Platforms: How Zetaris Makes Every Query Cost-Efficient
· Modern Lakehouse for AI — Zetaris
· Zetaris fast tracks the development of Agentic AI — Equinix case study
· Delivering digital agents and other GenAI solutions — Dell Technologies case study
· Crush Your Cloud Costs with Zetaris
· General Zetaris Technical Brochure
· AI Data Steward demonstration — LinkedIn
· Zetaris LakeHouse — AWS Marketplace