[back to blog]

FinOps for AI Agents: Teaching Sidekick About Money

aifinopscost optimizationagents

Last month I wrote about Uber capping its AI coding spend after it burned a year’s budget in four months, and the routing levers that make that kind of cap unnecessary. Sidekick, the AI review and incident-analysis harness we run at work, does the thing I criticized. Every run goes to the biggest model at the highest effort setting. Incident triage, review on a two-line diff, all of it gets the same ceiling, because when we built the pipeline we picked the model that made the demo work and never came back to the decision. That pipeline fires every ten minutes across multiple teams’ queues, so the lazy decision compounds in the background all day.

So we’re coming back to it. The routing itself is the easy half. The hard half is everything around it: who sees the spend, who sets the policy, what enforces it, and how you know the cheap model was good enough. Cloud engineering already has a name for that job: FinOps.

Nobody owns the AI bill yet

The FinOps Foundation tracks AI as its own spend scope now. In their 2024 survey, 31 percent of FinOps teams said they managed AI spend. In 2025 it was 63. This year it’s 98. Cloud cost control took about a decade to grow into a real practice with tools behind it. AI spend made the same jump in two years, and the tooling is nowhere near the adoption numbers.

The cloud methods don’t transfer cleanly either. Cloud FinOps grew up on infrastructure that idles: reserved instances, rightsizing, shutting dev environments off at night. An agent workload has no idle to trim. One task fans out into ten or twenty model calls, and the money follows output tokens, which cost five times input and scale with how hard the model thinks. The only place to cut cost is the moment each request picks a model, and that moment lives inside the pipeline code, months upstream of any finance review.

Two kinds of tools, and neither is enough

The AI cost tooling that exists today splits into two camps.

The FinOps camp watches spend and can’t change it. Token attribution, chargeback dashboards, anomaly alerts. Useful, and all of it sits at the billing layer, so by the time the dashboard shows a spike the tokens are gone. A dashboard also carries no quality signal. It shows the 60 percent saving from switching to a cheap model and says nothing about whether the cheap model’s reviews hold up.

The router camp has the opposite problem. Learned routers pick a cheaper model per prompt, but they can’t show you why: the decisions are opaque by construction, and the savings claims ride on generic benchmarks rather than your workload. We looked at the managed routing option on our cloud provider and rejected it for Sidekick. We’d been through this once before: security killed our first choice of agent framework because it phoned home to its own hosted service, and we couldn’t prove what left the box. Same verdict here. The capability worked; the black box was the problem.

That rejection sets a rule for whatever we build instead. Anything that picks a model at request time has to be a lookup a reviewer can predict from the config. Analysis of the telemetry can recommend changes, and a human merges them. The moment you need a model to explain another model’s choice, the audit trail is gone, and the audit trail is the reason to build this yourself at all.

The gap between the camps is the thing worth building: routing that’s cheap to run and readable after the fact, scored against whether your own outputs were any good. Nobody sells that combination today. For one team’s workload it comes out to about a hundred lines of code.

What we’re designing into Sidekick

Sidekick’s cost layer is still mostly design, so read this as what we’re building, not something we’ve proved out. Three parts, and they land on the FinOps framework’s own phases.

Inform. The design logs a model decision for every agent run: which model, which effort level, and the rationale, sitting next to the token usage the API reports back. Cost-allocation tags on the provider side would break the invoice down by task type, so the cloud bill and the logs can check each other. The metric this feeds is dollars per reviewed PR and dollars per incident analysis, the number a budget conversation runs on and the one Uber’s COO couldn’t produce when a reporter asked what the spend bought.

Optimize. Task types map to tiers in a config file: triage on the small model, standard review on the mid tier, incident analysis and the future fix pipeline on the big one, with deterministic overrides for oversized diffs, sensitive paths, and P1 tickets. The table goes in the repo, so changing what a P4 ticket costs becomes a pull request with a diff and a reviewer, like any other change to the harness.

Operate. The router reads month-to-date spend before it picks a tier. Past 80 percent of a task type’s monthly budget the design drops non-critical runs down a tier and logs it; at 100 percent, cheap task types stop, and incident analysis pages a human rather than silently skipping. Enforcement doesn’t depend on the router behaving: each pipeline’s cloud role allowlists only the models up to its ceiling, the most expensive model’s ARN sits in no role at all until a real need shows up, and a routing bug that requests it gets access-denied instead of a surprise invoice. If pipelines held raw provider keys instead, every rule in that config file would be optional.

Quality is the part FinOps can’t see

Cheap defaults are safe in Sidekick and risky in a generic gateway for one reason: the harness produces its own ground truth. We’ve already had a PR author commit the fixes Sidekick’s review flagged. That acceptance is a quality verdict on the exact model and effort that produced the finding, and it costs nothing to collect. The autofix pipeline we’re building next adds a harder signal: it runs the repo’s build and tests before delivering any fix, and that pass or fail says whether the cheap tier was good enough for that specific finding. Fail verification, escalate that one item to the next tier up, once, and log it.

Once those signals feed the routing table, tier decisions get graded on our own workload instead of a vendor benchmark. A rising escalation rate on a task type would mean its default tier is set too low. That loop is the part I’d steal for any AI system: find the places your pipeline already knows whether the output was good, a build that passes, a suggested fix the author commits, and wire them back into the table. Ours come from review acceptance today and the verification gate once autofix ships.

The boring part comes first

The build order starts with usage logging, about half a day of work, because everything after it gets tuned against real numbers, and a month of telemetry decides whether anything smarter earns its complexity. Then the static table, the IAM ceilings, the signal overrides, and the escalation hook, in that order.

The agents work and the bills are real. The layer connecting them is small: a routing table, a budget check, and some billing tags, sitting in a repo where anyone can read them. FinOps for AI is turning into an industry. The version one team needs is a config file somebody reviews.

Resources