RAG Chatbot Development Company: How to Choose One (and When to Skip RAG)

What a RAG chatbot development company actually builds

The deliverable is not a chatbot. It is a retrieval pipeline with a chat interface bolted on the end, and roughly 80% of the engineering sits upstream of the model. If a proposal only describes the chat widget, you are being quoted for the smallest part of the job.

Ask any shortlisted vendor to walk you through all nine. The ones who have shipped this before will have opinions about chunking and permissions. The ones who have not will steer the conversation back to the model they use.

The first thing a good vendor does is try to talk you out of RAG

RAG is a workaround for one specific constraint: you have more knowledge than fits in the model's context window, and it changes. If that is not your situation, retrieval adds cost, latency and a new failure mode for no benefit. A vendor whose only product is RAG will never tell you this.

A concrete example from our own site: the assistant on netalith.com answers questions about services, blog posts and products, and it has no vector store at all. It calls functions that query the live database, because prices and article lists are structured data that change. Nothing can go stale and nothing can be retrieved out of date. We would have shipped a worse product by embedding all of it — and it would have cost more to run.

So the useful filter when you compare proposals is not who promises the highest accuracy. It is who asks to see a sample of your documents and the twenty questions users will actually ask, before quoting an architecture. That is what our AI development service scopes in the first conversation, and it is the step that decides whether the rest of the project is worth doing.

The retrieval stack that separates a demo from production

When a RAG chatbot gives a wrong answer, the model is almost never the cause. The correct passage was not in the context window. Three fixes address that, and the order matters more than the brand of any component.

1. Hybrid retrieval: dense plus lexical

Embeddings match meaning and fumble exactness. Ask for error code TS-999 and a pure vector search returns a pleasant essay about error codes in general. BM25, the lexical ranking function behind classic search engines, does the opposite: precise on rare tokens, blind to paraphrase. Production systems run both and fuse the results. Your users will test the system on an identifier in the first week, so this is not an optional refinement.

2. Context attached to chunks before indexing

A chunk reading the allowance increased by 15% from the start of the quarter is useless on its own — which allowance, which quarter, which policy version? The fix is to prepend a short generated description that situates the chunk in its parent document before embedding it:

Raw chunk:
"The allowance increased by 15% from the start of the quarter."

Contextualised chunk:
"From the 2026 Vietnam employee handbook, section 4.2 on remote-work
stipends, revised March 2026. The allowance increased by 15% from the
start of the quarter."

It is a one-time preprocessing cost, and it is small: with prompt caching, Anthropic reports about $1.02 per million document tokens to generate that context.

3. Reranking

Retrieve broadly and cheaply, then rerank aggressively. Anthropic's published tests retrieved the top 150 candidates, reranked, and passed the top 20 into the prompt.

The numbers from Anthropic's contextual retrieval research are worth reading as an ordering of priorities. Measured as the share of relevant passages missing from the top 20 results, contextual embeddings alone cut failures by 35%; adding contextual BM25 took it to 49%; adding reranking reached 67%, moving the failure rate from 5.7% to 1.9%.

Two pieces of interpretation that the headline numbers do not carry. First, those are results on their test corpora — your mileage depends on your documents, and the only way to know is an eval set built from your own questions. Second, and more useful commercially: every one of those gains came from the retrieval layer, none from changing the model. When a vendor's answer to accuracy problems is a more expensive model, they are skipping the part that would have worked.

What a RAG chatbot costs to build, and what it costs to run

Build cost is quoted per scope and varies with the number of source systems, document formats and permission rules. The number buyers consistently fail to ask for is the second one: the monthly running cost, at their real question volume.

Two practical consequences. Cost per answer is set by how much context you retrieve, which means aggressive reranking is a cost control as well as a quality fix. And a proposal with no monthly estimate attached is incomplete — ask for a cost per 1,000 questions at your expected chunk count, and ask what happens to it if the corpus triples.

Twelve questions to ask a RAG chatbot development company before you sign

These are ordered by how much signal they carry. The first four will separate the field.

1. Show me an eval set from a past project. How do you score retrieval quality separately from answer quality? No eval set means no evidence, and no way to tell improvement from regression.
2. What does the assistant do when the answer is not in the documents? You want a designed refusal with a handoff path, not a graceful-sounding guess.
3. Does every answer carry a citation to the source document and location? Non-negotiable for internal knowledge tools. It is how staff verify, and how you debug.
4. How does the index stay in sync when a document changes? Listen for webhooks, scheduled re-indexing, deletion handling. Listen for whether deleted documents actually leave the index.
5. Are permissions enforced at retrieval time? Filtering candidates by the asking user's rights, not instructing the model to keep secrets in the prompt. A prompt is not an access-control layer.
6. Which components can run on infrastructure we own? Ask specifically where embeddings are computed — sending a contract corpus to a third-party embedding API is a data transfer, whatever the chat model does.
7. What is the monthly running cost at our volume, itemised? See the table above. A single blended figure hides which lever to pull later.
8. Who owns the code, the prompts and the index? Get it in writing. Prompts and eval sets are the real intellectual property of the build.
9. How do you handle tables, scanned PDFs and spreadsheets? Most real corpora are full of them, and most generic pipelines mangle them. Ask for the parsing strategy by file type.
10. What is logged, where is it stored, and for how long? Conversation logs contain whatever staff paste into them.
11. Can we run a bake-off on our own documents before committing? A paid pilot on your corpus beats any demo on theirs.
12. What does handover look like? Could your own developer re-index, adjust a prompt and deploy without the vendor? If not, you have bought a dependency, not a system.

Compliance: what changed for chatbots on 2 August 2026

If any of your users are in the EU, this is now a procurement question rather than a future one. Under Article 50 of Regulation (EU) 2024/1689, the AI Act, systems that interact directly with people must make clear that the person is dealing with an AI. The transparency obligations applied from 2 August 2026, and the delays agreed for high-risk systems did not move them. Penalties for transparency breaches run to EUR 15 million or 3% of worldwide annual turnover, whichever is higher.

What matters when picking a vendor:

• The disclosure duty is split. The provider that builds the system carries the design obligation under Article 50(1); as the deployer, you carry the rest. A vendor who has never heard of this is not one who will have built it in.
• The duty is retroactive to existing bots. An assistant that has been live since 2025 needs retrofitting, not grandfathering. A narrower transition window to 2 December 2026 covers only the machine-readable marking duty under Article 50(2), for generative systems already on the market before August.
• Disclosure at first interaction. A line in the privacy policy does not satisfy it; the first chat bubble does.
• GDPR is a separate obligation. You need a lawful basis for processing conversation data regardless of the AI Act notice.
• Data residency is an architecture choice, not a checkbox. If the corpus is contracts, patient records or employee files, ask about a self-hosted model for generation and a locally computed embedding model. We expect this to be the default question for regulated buyers rather than the exception.

None of this is legal advice — the point is narrower. These are questions your vendor should be answering without being prompted, and the answers shape the architecture, so they belong in the first scoping call rather than the final review.

Red flags, and what a realistic timeline looks like

Signals that a quote is not backed by delivery experience:

• A fixed price before anyone has looked at a sample of your documents.
• Accuracy claims as a percentage, with no eval set and no definition of what was measured.
• A demo on their own curated corpus, with no offer to run on yours.
• A proposal to embed your product catalogue, price list or order history.
• Fine-tuning offered as the fix for the model not knowing your business.
• No mention of permissions, or permissions described as instructions in the system prompt.
• Pricing in opaque credits with no token breakdown behind them.
• No answer to what happens when retrieval returns nothing relevant.

And a realistic shape for a first internal assistant over a single document corpus:

Multiply that for several source systems, multilingual corpora or per-document access rules. Anyone quoting a week for an assistant across five systems with role-based permissions is quoting for the demo.

If you are still deciding whether retrieval, tool calling or a long prompt is the right answer for your data, that is the conversation worth having before any quote means anything. Tell us what your documents look like and what people will ask, and we will tell you which of the three your case needs — including when the honest answer is that you do not need RAG at all.