Our AI bill arrived with a single line on it: $245.31, attributed to a key named “pulse.” The problem was that Pulse, our local-SEO monitor, was not the only thing using that key. eighteen products are, and back then fourteen of them shared one key. We had no idea which one had spent the money, and no way to find out.
Then the key ran out of credit, and every AI feature we shipped went dark at the same moment. No alarm fired. Scheduled jobs kept running and kept failing silently. One product carried on writing placeholder text into its database as though it were real analysis.
This is the story of what we built instead, and the architecture decision underneath it that any company running AI in production should copy: never let a product know which AI provider it is talking to.
The real problem was not the bill, it was the coupling
Every one of our products had the same shape of code in it. Somewhere inside, a hardcoded address for Anthropic’s API, an API key in an environment file, and a model name. That looks harmless. It is actually three separate forms of lock-in stacked on top of each other:
- Provider lock-in. Changing who answers your AI calls means editing, testing, and deploying every application you own.
- No attribution. When several products share a key, the provider’s invoice tells you the total and nothing else. You cannot price a feature you cannot measure.
- Correlated failure. One shared credential means one shared outage. Everything fails together, which is the worst possible failure mode because it looks like an infrastructure problem rather than a billing one.
The fix: one seam between your products and the model
We put a gateway in the middle. Every product now sends its AI requests to one internal address instead of directly to a provider. The gateway holds the provider credentials, decides which model actually answers, records what each product spent, and enforces a monthly budget per product.
The applications did not get more complicated. Most of them changed by a single line: the address they send requests to became configurable instead of hardcoded. That one change buys three things at once.
Switching providers becomes a configuration change
Our products ask for models by name. The gateway decides who answers that name. When our Anthropic balance ran dry, we pointed those names at OpenAI models and the entire fleet kept working. eighteen products, zero code changes, zero deployments, about thirty seconds of downtime while the gateway restarted.
Going back is the same operation in reverse, and it is now a button on an internal dashboard. Before the switch runs, it tests that the destination provider can actually answer. If it cannot, nothing changes. If the gateway comes back unhealthy, it restores the previous configuration by itself.
Every dollar gets a name
Each product now holds its own key with its own monthly ceiling. Instead of one $245 mystery, we see which product spent what, how many requests it made, and how many tokens it used. A product that reaches its ceiling stops spending; nothing else on the fleet notices.
That per-product ceiling is the part most teams skip, and it is the part that converts a scary unbounded cost into a known monthly maximum. It is the difference between “AI might cost us anything” and “AI costs at most this, and here is the split.”
Failures stop being invisible
The original outage was invisible because AI calls lived inside scheduled jobs with nothing watching them. Now every request, successful or failed, is logged against a product. Our dashboard reads those logs and flags the patterns that matter: a product that is failing right now, a product wired up that has never once made a call (which almost always means broken configuration rather than low usage), a premium model quietly doing bulk work, a runaway prompt.
Why not just use one provider and be careful?
Because the last two years have made the answer obvious. Model quality leapfrogs between vendors every few months. Prices move. Accounts run out of credit at inconvenient hours. A capability you depend on gets deprecated, or a new one appears somewhere else.
If switching providers costs you a week of engineering across every product you own, you will not switch, and you will quietly pay whatever the incumbent charges. If switching costs you a config edit, you can take the better model or the better price the week it appears. The gateway is not really about cost. It is about keeping the option to change your mind.
What this looks like in practice
A concrete example from our own stack: our chamber management platform runs voice-critical work on a premium model and cheap classification on the smallest available one. Those are named choices in one configuration file, not scattered constants across a codebase. Moving all of it to a different vendor is one line per model name. Moving a single feature is one line, full stop.
Another: our client-facing writing tools and our internal analysis jobs share nothing except the gateway. They have separate budgets, separate spend reporting, and separate blast radius. When one hits its cap, the other does not care.
The honest cost of building it
The gateway itself took an evening. The interesting part was not the software, it was the discipline it forced: finding every place in the fleet that called an AI provider, discovering several we had forgotten, and admitting how many products had been quietly sharing one credential because it was convenient at the time.
We also found that the failure modes were sneakier than expected. A configuration file can say one thing while the running service uses another. A dashboard can show a perfectly correct number in a unit that makes a human read it wrong by a factor of a hundred. Both happened to us while building this, and both are now checked automatically rather than trusted.
If you are running AI in production, ask these four questions
- If your provider’s price doubled tomorrow, how long would it take to move? If the answer is measured in weeks, you do not have a provider, you have a dependency.
- Which of your features spent the most on AI last month? If you cannot answer, you cannot price your product properly, and you cannot tell a runaway job from normal growth.
- What is the maximum your AI spend could be next month? If there is no ceiling, there is no answer, and one bad loop can produce a bill that is genuinely frightening.
- How would you find out that your AI stopped working? If the honest answer is “a customer would tell us,” add that alarm before you add anything else.
None of this requires a big platform or a new vendor. It requires deciding, once, that your applications should not know or care which model is behind the curtain.
How Champlin Enterprises can help
We build AI-native software for our own products first, which is exactly why we hit this problem before our clients did. If you are running AI features across more than one application and you cannot answer the four questions above, that is the work we do: provider independence, per-product cost control, and the monitoring that makes an AI outage loud instead of silent.
Tell us what you are building, and we will tell you honestly whether this is worth doing for you yet.
