The True Cost of Implementing AI: Startup Lessons Learned Post-Launch

For most of the past 20 years, software startups pitched investors on a simple piece of math: once the product was built, serving one more customer costs almost nothing. This is why mature SaaS companies could run gross margins of 75% to 85%, and why investors were willing to pay high multiples for recurring revenue.

AI features don’t follow those rules. Every time a user asks an assistant a question, gets a summary or has it draft something, the company pays for the compute behind that answer.

The cost doesn’t fall away as the consumer base grows, however. It rises with every new account and every returning user, which means the most popular features are often the most expensive to run.

ICONIQ Capital’s January State of AI snapshot, for one, based on 300 executives at software companies building AI products, projected average AI product gross margins of 52% for 2026. Though that is an improvement on the 41% reported in 2024, it’s still well short of what investors expect from software.

The same survey found model inference grows as a share of total spending as products mature, from 20% before launch to 23% at the scaling stage, while talent costs shrink. For a company whose AI feature is working, the model bill keeps taking a bigger slice.

These figures describe companies whose products are built around AI. A business adding a single AI feature to an established subscription product will feel a smaller hit—at least at first. The pressure grows as the feature reaches more customers, and it grows fastest when the feature is switched on for everyone.

Some projects don’t survive that pressure. Gartner listed escalating costs first among the reasons it expects more than 40% of agentic AI projects to be canceled by the end of 2027, ahead of unclear business value and weak risk controls.

The companies managing this tend to share a few habits; they work out what a single interaction costs before they ship, design prompts so they aren’t paying again and again to process the same text, and they track spending as closely as they track errors and downtime.

A recent deployment at Doppler, an email and marketing automation platform serving businesses across Latin America and Spain, shows what those habits look like once a feature is in front of thousands of customers.

A cost that grows with every signup

Doppler’s assistant, ECO IA, sits inside the product, reading live account data, pointing to problems with how an account is performing, and drafts campaigns, which users then edit and approve before sending.

It also runs on Anthropic’s Claude, and was built by Making Sense, a U.S. consulting firm that recently reached the Select tier of Anthropic’s Claude Partner Network, with Doppler as its published case.

The decision that shapes its economics is that it’s switched on by default. An opt-in feature only costs money when a customer decides to try it, but a default feature starts costing money the moment an account is created—meaning the AI bill grows in step with the sales pipeline.

ECO IA is now available in more than 4,800 accounts, per Making Sense. The firm says 95% of users who engaged with the assistant returned across multiple months, which means the repeat use the product team hopes for, though the recurring cost, too.

According to the company, each interaction costs $0.066. With such a per-interaction number, a finance team can put the feature next to subscription revenue per account, and work out how much usage the business can carry before margins suffer.

Without it, though, the monthly API invoice mixes growth and waste together, and nobody can tell which one pushed it up.

Pricing is another lever. ICONIQ describes an emerging practice among AI builders of pairing a light subscription fee for platform access with usage-based charges for volume, an approach that ties revenue more closely to what each customer costs to serve.

Paying once for text that never changes

While looking at what an AI feature sends to a model on each request and much of it is identical every time. The system instructions, the description of available tools, formatting rules, and reference material often don’t change between calls. Only the user’s question and some live data are new.

Paying full price to process that repeated material on every call is one of the most common sources of waste, and model providers now offer ways around it. Anthropic’s prompt caching lets developers store the unchanging part of a prompt so the model doesn’t reprocess it each time. When the company launched the feature, it said caching could cut costs on long prompts by up to 90%.

Results in practice vary, and widely. A developer who tested caching on a business intelligence chat agent reported a 57% cost reduction from a few lines of code, according to Hackernoon, and also warned readers not to treat the figure as universal without testing their own prompts.

What decides the savings is the cache hit rate, the share of requests that reuse content already stored. Because catching matches from the start of the prompt, the stable sections need to come first and stay put, since editing an instruction near the top of a prompt can invalidate everything stored after it.

At Doppler, for example, the team reports a 97% hit rate, which reflects careful prompt structure as much as anything about the model.

Who keeps the numbers in check after launch?

Few startups would run a production service without monitoring latency and errors, yet many still launch AI features with no view of what they cost until the invoice arrives.

For ECO IA, the team built a real-time dashboard covering usage, spending, latency and errors, so engineering and finance look at the same figures.