What will your AI chatbot cost to run each month?
Model the real running cost of a customer support bot, internal assistant or document Q&A tool. Set traffic, prompt sizes, retrieval and model tier, then see how prompt caching, model routing and human handoffs change the monthly bill.
Model your assistant
Start from a preset, then adjust traffic, prompts and the model. Every cost lever updates the estimate as you go.
- Model input tokens$51
- Model output tokens$11
- Embeddings$0.340
- Vector database$70
- Hosting and observability$450
- Human-only$30,000
- AI running cost$580
- Human handoffs (25%)$7,500
- Frontier model$910
- Mid-tier model (selected)$580
- Small fast model$530
- Self-hosted open-weight$2,300
- Prompt caching at a 60% hit rate is saving about $11 a month under these assumptions.
- Routing 40% of simple turns (greetings, order status, FAQs) to a small fast model would save about $20 a month. Test quality on real transcripts first.
- Platform costs outweigh tokens at this volume. Optimise for answer quality and containment rather than token price.
- Resolving 75% of conversations without a person saves about $22,000 a month ($260,000 a year) compared with handling every support ticket manually, under these assumptions.
- Keep a human review loop on escalated and low-confidence answers, and track containment weekly as your content changes.
Assumptions: cached tokens billed at 25% of the input price, history capped at 10 prior turns, embeddings at $0.05 per million tokens. Tier comparisons use default tier prices. Excludes build cost and staff time to maintain content.
Indicative only. We will send the inputs above with your message so a senior engineer can sanity-check them.
Tokens first, then everything around them.
Every message a user sends triggers a model call, and that call carries far more than the question. It includes your system prompt, the chunks of documentation retrieved for this question, and a window of earlier turns in the conversation. The calculator rebuilds that request message by message, so long conversations and generous retrieval show up in the cost the way they do on a real invoice.
Input and output tokens are priced separately using the tier you choose. The defaults are indicative prices for generic frontier, mid-tier, small and self-hosted models, and you can overwrite them with the rates you are actually paying. Prompt caching discounts the stable prefix of each request, and model routing sends a share of simple turns to a cheaper model.
On top of the model bill sit embeddings for your knowledge base, a vector database, and hosting with tracing and evaluation. Finally, the tool compares the total with handling every conversation manually, after allowing for the share that still needs a person.
Typical monthly running cost of an AI chatbot
Indicative ranges for production assistants using mid-tier or frontier models with retrieval. Your prompts, traffic and model choice can move these a long way.
| Typical volume | Main cost driver | Indicative monthly cost (USD) | |
|---|---|---|---|
| Small business FAQ bot | 1,000 to 5,000 conversations | Platform and hosting fees | $100 to $600 |
| Growing support assistant | 10,000 to 50,000 conversations | Input tokens and observability | $800 to $6,000 |
| Internal knowledge assistant | A few hundred staff | Retrieved context per question | $300 to $2,500 |
| Document Q&A over large archives | Heavy retrieval, long answers | Context size and vector search | $1,000 to $10,000 |
| High-volume consumer assistant | 500,000+ conversations | Output tokens, routing strategy | $15,000 and up |
Ranges exclude build cost and human handoffs. At high volume, the gap between a well-optimised and a naive setup is often larger than the gap between model tiers.
Six ways to lower the cost of running an AI chatbot
Most of these also improve answer quality or speed, which is why we look at them before anyone suggests a cheaper model.
Cache the stable prefix
System prompts and conversation history repeat on every turn. Prompt caching bills the repeated part at a fraction of the normal input price.
Route simple turns
Greetings, order lookups and yes-or-no questions rarely need a frontier model. A router can send them to a small fast model and keep the big one for hard questions.
Retrieve less, retrieve better
Stuffing ten document chunks into every request is expensive and often confuses the model. Better chunking and reranking usually cut context and improve answers.
Summarise long conversations
Resending every earlier turn makes long chats grow in cost with each message. A running summary keeps the context the model needs without the full transcript.
Ask for shorter answers
Output tokens cost several times more than input. Clear length guidance in the prompt lowers the bill and usually reads better in a chat window.
Fix containment before tokens
Every conversation that reaches a person costs far more than the model call. Filling knowledge gaps that trigger handoffs is often the biggest saving available.
The token bill is rarely the number that decides whether an AI assistant pays off.
For most businesses, model usage is a modest line next to the cost of people answering the same questions. What decides the return is how many conversations the assistant resolves well, how often it hands off with useful context, and how quickly your team fixes the gaps it reveals.
Build cost has shifted too. AI-native engineering teams, working with coding agents and AI-assisted testing, can typically stand up a retrieval-based assistant with fewer engineer-hours than a traditional team. The work that still needs senior judgement is evaluation, guardrails and integration with your systems, and that is where we would spend the budget.
Questions we often hear
How much does it cost to run an AI chatbot per month?
A small business assistant handling a few thousand conversations commonly costs a few hundred dollars a month including hosting. A busy support assistant on a frontier model can run to several thousand dollars, and very high-volume consumer assistants go well beyond that. Traffic, prompt size, retrieved context and model tier are the main drivers.
How are LLM API costs calculated for a chatbot?
Providers charge per token, with separate prices for input and output. Each message sends the system prompt, any retrieved documents, recent conversation history and the new question as input, and the reply is billed as output. That is why long prompts and long conversations cost more than the length of the user message suggests.
Does prompt caching really reduce chatbot costs?
It can, when a large share of each request repeats, such as a long system prompt or conversation history. Cached tokens are billed at a discount, though the discount and rules vary by provider. Retrieved context changes with each question, so assistants dominated by retrieval see smaller savings.
Is it cheaper to self-host an open-weight model for a chatbot?
Usually not at low or moderate volume, because you pay for GPU capacity whether or not it is used, plus the engineering time to run it. Self-hosting starts to make sense at sustained high volume or when data must stay in your own environment. Our GPU vs API inference calculator models that break-even in detail.
How much can an AI chatbot save compared with human support?
It depends on the share of conversations the assistant resolves and on what a human-handled ticket costs you. Where many questions are repetitive and well documented, savings can be substantial, but a poorly scoped bot that hands off most conversations can cost more than it saves. The calculator shows both outcomes so you can test your own assumptions.
Does AI-native development make an AI chatbot cheaper to build?
Generally yes. Teams that build with coding agents and AI-assisted testing spend fewer hours on integration code, admin screens and test suites. The parts that deserve senior attention, such as evaluation sets, guardrails, security and data access, still take real effort, and they are what separates a useful assistant from a risky one.
Related reading and tools
Planning an AI assistant? Get the costs right first.
Send us your estimate and a few lines about the use case. A senior engineer will check your token assumptions, point out the cheapest levers for your setup and give you an honest view on whether the assistant will pay for itself.
Working with companies globally · Response within 24 hours