AI spend crept up on a lot of engineering‑led teams this year. It didn’t look like classic cloud growth: a few “mystery” line items from Bedrock, a big charge from Azure Cognitive Services, a handful of Vertex AI services — and suddenly the totals got uncomfortable.
If you’re a small or mid‑size team moving fast, you don’t have time to become an expert in every provider’s billing quirks. You need two things:
- A clear picture of who’s using which models, where, and why.
- Tight but lightweight guardrails that keep costs in bounds without slowing delivery.
This post is a concise playbook to get both. It’s based on dozens of conversations with teams like yours and what we’ve learned building Finout for AI.
Why AI Costs Feel Opaque (Especially in Multi‑Cloud)
Three patterns make AI spend uniquely slippery:
- Token billing across different wrappers. The same prompts show up as marketplace “units” in AWS, token bundles in Azure, and per‑call plus per‑character mixes in GCP. Apples‑to‑apples comparisons vanish.
- Weak default attribution. Most LLM calls don’t carry a team, service, or user unless you pass it. CURs and invoices will tell you “what,” not “who/why.”
- Mixed commercial models. Seats (Copilot, Chat Enterprise), drawdowns (e.g., Snowflake, Databricks), and on‑demand API use live side by side. Some costs are prepaid, some variable, all appear together.
The result: a growing total with low explainability. You can’t control what you can’t see.
Step 1: Normalize Everything to Tokens
Before you optimize, standardize. Pick a single unit of measure — tokens — and convert each provider’s AI usage to it. Once everything is in tokens, trends become obvious:
- Cost per 1M tokens by model (and over time)
- Token mix by environment (prod vs. sandbox)
- Token share by team/feature
In Finout: we normalize AI services across AWS, Azure, and GCP into token‑equivalent views so you can compare models and teams side‑by‑side and track cost per token over time.
Step 2: Attach Identity at the Source
If a call can’t be tied to a team, service, environment, and (when appropriate) user, it’s a ghost cost. Add metadata at the edge:
- OpenAI: use Project to represent service or feature; consider passing user context via your proxy if you control the client.
- AWS Bedrock: prefer Inference Profiles over direct model ARNs and tag profiles with team, service, env. (Profiles are interchangeable with model IDs in clients and give you a place to keep tags.)
- GCP Vertex AI: use labels on endpoints/batch jobs (team, service, env, model).
- Azure AI (Cognitive Services / Azure OpenAI): tag the Azure resource, standardize deployment names by team/service, and forward user context through your proxy or function app.
- Gateways/Proxies: if you run an AI gateway, inject the same keys on every call and log them with token counts.
In Finout: map these keys into virtual tags and business mapping so every AI dollar is allocated to a real owner (team, product, BU), not a catch‑all account.
Step 3: Build One AI Cost View (Across Clouds and SaaS)
Create a single dashboard with:
- Spend & tokens by model (normalized)
- Top teams/features by tokens & cost
- Environment split (prod vs. non‑prod)
- Seat vs. API spend: dev tools (e.g., Copilot) vs. runtime usage
- Trend lines: 7/30/90‑day to spot slope changes
This isn’t about pretty charts. It’s about answering, in one page: what grew, who caused it, and what knob we can turn.
In Finout: the AI overview pulls AWS, Azure, and GCP together, normalizes to tokens, and lets you drill into team/model/environment in a few clicks. You can also bring in CSV/API telemetry from proxies or usage logs if you have them.
Step 4: Turn Visibility into Accountability (Showback → Chargeback)
Start with showback: monthly summaries to each team of tokens, cost, and cost per 1M tokens. When the numbers settle and the data is trusted, move to chargeback for variable API usage. (Seats can remain centrally funded.)
Two simple rules that work:
- Budgets by environment. Non‑prod gets a firm monthly cap; prod gets a threshold and alerting.
- Cost follows usage. Teams pay for their own tokens; they also reap the savings from efficiency.
In Finout: send scheduled reports to Slack/Email by team; track budgets by tag or project and alert when burn rates spike.
Lean Guardrails for Small Teams
You don’t need a platform group of 20 to keep AI bills sane. These four controls deliver most of the value with minimal overhead.
1) Budget Caps & Alerts (Per Team, Per Env)
- Hard caps for sandboxes and notebooks.
- Soft thresholds for prod with Slack alerts at 50/80/100% of monthly budget.
- Daily “delta” alerts (e.g., +30% day‑over‑day) to catch runaway loops.
In Finout: budgets & anomaly detection work across providers and roll up by tags (team/service/env).
2) Policy at the Edge (Cheap, Enforceable)
- Require tags in clients or gateway (team, service, env) — reject untagged calls.
- Cache by default for repeatable prompts; prohibit cache‑disabled in prod unless whitelisted.
- Model guardrails: default to a “good enough” model; allow upsizing by exception.
- Context window limits: cap max tokens unless explicitly approved.
These are simple checks you can implement in a gateway, SDK wrapper, or middleware.
3) Sandbox Safety Valves
- TTL on API keys for experiments.
- Auto‑shutdown for idle GPUs and notebooks.
- Nightly teardown of ephemeral stacks in non‑prod.
These save real money without touching production.
4) Cost Gates in CI/CD (Not Just Performance Gates)
- Block deploys that:
- Remove caching on high‑volume paths
- Increase default context windows beyond policy
- Switch to premium models without tag/approval
- Let engineers override with a reason — but require conscious choice.
In Finout: expose model, token, and unit‑cost KPIs via API/webhooks to your pipeline to power cost checks next to perf tests.
A 30‑60‑90 Day Plan
Days 1–30: Make It Visible
- Normalize to tokens across AWS/Azure/GCP.
- Add team/service/env tagging (profiles/labels or via gateway).
- Stand up one AI cost dashboard; start monthly showback.
Days 31–60: Make It Actionable
- Define budgets for prod/non‑prod by team; enable anomaly alerts.
- Enforce tagging & caching at the client/gateway.
- Track cost per 1M tokens by model; pick a default “good enough” model.
Days 61–90: Make It Durable
- Add sandbox safety valves (TTL, idle shutdowns).
- Introduce cost gates in CI/CD for cache/context/model changes.
- Move variable API usage to chargeback; keep seats centralized if it reduces friction.
What “Good” Looks Like (Simple North‑Star Metrics)
- ≥95% of AI spend attributed to team/service/env
- <24h time‑to‑detect unusual token or cost spikes
- ≥80% of prod traffic cache‑eligible, and cache hit rate rising
- Cost per 1M tokens trending down for the same user experience
- Sandbox spend <15% of total tokens (or trending down)
- No‑surprise month‑end (actuals within ±10% of forecast)
If you can hit those, you’ve moved from “mystery bill” to “managed utility.”