LLM Integration: 6 Decisions to Make Before You Ship

LLM Integration: 6 Decisions to Make Before You Ship

LLM integration is six decisions, not one build: model, context, tool access, trust boundary, cost ceiling and evaluation. Decide them in that order and the code is ordinary. The largest single lever is caching. Anthropic's published pricing charges a cache read at 0.1x the base input price, a 90% cut on repeated context.

Published 24 September 2026. For CTOs, VPs of Engineering and engineers shipping a language model to real users.

The short answer

• Model choice is routing, not a purchase. Anthropic's published prices in August 2026 ran from $1 per million input tokens for Claude Haiku 4.5 to $5 for Claude Opus 5. Route cheap work to the cheap model.
• Context strategy comes before framework choice. A longer prompt, retrieval and fine-tuning fix different problems, and the wrong pick is a rewrite, not a config change.
• Tools are the integration. The model is a text function. Everything useful happens in the tools you expose, so design them like a published API.
• The trust boundary is the security control. The OWASP GenAI LLM Top 10 2026, published 4 August 2026, still ranks prompt injection first. Bound what a manipulated model can reach.
• Context size and cache hit rate set the bill, not the model name.
• Pin the version. Anthropic gives at least 60 days notice before retiring a model, OpenAI at least six months for generally available ones and two weeks for previews.
• No eval, no ship. If you cannot score a change, nobody can approve one.

What are the six decisions in an LLM integration?

Six, in this order: which model, how context reaches it, which tools it can call, what it is allowed to touch, what it may cost, and how you will know it works. Framework choice is not on the list. It falls out of the first three, and teams that start there spend a month on plumbing before agreeing what a correct output is.

#	Decision	Sensible default	Cost of reversing it later
1	Model and routing	One mid-tier model, then split off the cheap path	Low, if versions are pinned
2	Context strategy	Retrieval over the store you already have	High. Fine-tuning first is a rewrite
3	Tool access	A few typed, narrowly scoped tools	Medium
4	Trust boundary	Read by default, writes behind confirmation	High, once permissions ship
5	Cost ceiling	Cost per request measured before launch	Medium
6	Evaluation	A golden set and an agreed pass threshold	High. Retrofitting evals is slow

Decisions 2 and 4 are the expensive ones to reverse, so argue them out up front. That week of scoping is the smallest line item in a generative AI development engagement.

Which model should I pick, and does the price difference matter?

Pick one mid-tier model as the default, then move work off it in both directions once you have usage data. Anthropic's published pricing in August 2026 listed Claude Haiku 4.5 at $1 per million input tokens and $5 per million output, Claude Sonnet 5 at $2 and $10, and Claude Opus 5 at $5 and $25. That is a 5x range on input inside one catalog.

Two rules keep this from becoming a religious argument. No vendor SDK call belongs in a request handler: put every call behind one interface that takes a task name and returns a typed result, so a model swap is a config change you can test. And work nobody is waiting on belongs on a batch path, because Anthropic prices its Batch API at a 50% discount.

Do I need RAG, fine-tuning, or just a longer prompt?

Start with retrieval, and fine-tune only when the problem is format rather than facts. Most stalled integrations use one of these four to fix a problem that belongs to another.

Approach	What it fixes	What it does not fix	Cost shape
Longer prompt	Background the model never had	Data that changes often, or exceeds the window	Paid every call, unless cached
Retrieval	Answers grounded in current data	A model that ignores your format	Index, embeddings, retrieved tokens per call
Fine-tuning	Consistent format, tone, classification	Facts, stale the day after training	A training run per behavior change
Tools	Live state, and taking an action	Reasoning, which stays with the model	Extra round trips, so latency

The test: if a subject expert could answer correctly given the right document, you have a retrieval problem. If that expert would be right and your model still returns the wrong shape, it is a prompt or fine-tuning problem. The two look identical in a demo.

How should the model reach my tools and data?

Through a small set of typed tools with narrow scopes, never a general database connection or a shell. A tool definition is a contract: a name, a schema, a scope and a test. Ten well-named tools beat one run_query tool, because each can be permissioned, logged and evaluated on its own.

If more than one assistant will use those tools, publish them over the Model Context Protocol. MCP is now governed under the Linux Foundation, and its 2026-07-28 specification revision moved it to a stateless request and response core that ordinary web infrastructure can host. That is what MCP server development covers.

What stops a prompt injection from reaching my systems?

The trust boundary, not the prompt. The OWASP GenAI LLM Top 10 2026, published 4 August 2026, ranks prompt injection at LLM01, because a model cannot reliably separate your instructions from text it was handed. Filtering helps at the margin. Containment is what holds.

• Treat every retrieved document, page and uploaded file as untrusted input, as you would a query string.
• Give the model its own credentials, minimally scoped, never the caller's session.
• Gate writes, payments and outbound messages behind a human or a deterministic rule.
• Keep secrets and other tenants' data out of the context window. What is in the window can leave it.

Score the integration on what a fully manipulated model could reach with today's permissions. If that answer is uncomfortable, the fix is the permissions, not the system prompt.

What actually controls the monthly bill?

Context size per call and cache hit rate, in that order. A prompt carrying 20,000 tokens of instructions costs the same whether the user typed three words or three hundred, and you pay it every call unless cached.

Token type	Price relative to base input	Break even
5-minute cache write	1.25x	Pays for itself after one cache read
1-hour cache write	2x	Pays for itself after two cache reads
Cache read	0.1x	Applies for the life of the entry
Batch API	0.5x on input and output	Work nobody is waiting on

Those are Anthropic's published prompt caching multipliers. The design consequence is concrete: put stable content first and variable content last, because caching works on a shared prefix. Then measure cost per request at projected volume before launch, with a monthly ceiling and an alarm.

What happens when the model version changes underneath me?

Pin the exact version and treat an upgrade as a release, because floating aliases move without asking. Both vendors publish a notice window, and they differ.

Vendor	Minimum notice before retirement	Published example
Anthropic	At least 60 days for publicly released models	Claude Sonnet 4 and Opus 4 were deprecated 14 April 2026, retired 15 June 2026
OpenAI	At least 6 months for generally available models, 3 months for specialized variants, 2 weeks for previews	GPT-5 and o3 snapshots listed for shutdown on 11 December 2026

Read that first row again. Sixty-two days from announcement to a hard failure on every request. That is survivable with a pinned version and an eval suite: point the config at the replacement, run the suite, compare the score. Without one it is a fire drill.

Is my LLM integration ready to ship?

Score each decision out of 5, for 30 points. Five means it is written down and demonstrable outside the team. Zero means it is one engineer's opinion.

Decision	What a 5 looks like	Points
Model and routing	Version pinned, one interface, routing rule written down	5
Context strategy	Retrieval measured against a golden set, no stray fine-tune	5
Tool access	Every tool typed, scoped, tested and logged	5
Trust boundary	Retrieved content untrusted, writes gated, no secrets in context	5
Cost ceiling	Cost per request measured at volume, cap and alarm live	5
Evaluation	Golden set, automated score in CI, threshold agreed first	5

How to read the score. At 24 or above, ship. Between 18 and 23, you have two to four weeks of work before a launch date is honest. Below 18, do not put it in front of customers. Decisions 4 and 6 are load bearing: if either scores zero the total means nothing.

Who builds this, and what does it cost?

Empiric Infotech is a remote software development company in Surat, India, founded in 2020. For AI work we put one named senior engineer on your codebase full time, and monthly is a flat USD 2,000 in the US and India, EUR 2,000 in Europe and AUD 3,000 in Australia for any service, AI included. That buys 160 to 172 hours from one exclusive engineer, billed monthly upfront. Standard engineering on the same terms is USD 2,000 a month in the US and India, EUR 2,000 in Europe and AUD 3,000 in Australia, and hourly AI work is USD 25 against USD 15 for standard development.

The terms are short on purpose. A 7-day risk-free trial, then month to month, cancel on 7 days notice. You keep the repository, the cloud accounts and the model keys throughout, and a senior team lead reviews and tests every release.

If your score came back under 24, that gap is a fixed number of weeks of work, not an open-ended research project. Start with generative AI development if the feature is retrieval, extraction or content generation, or MCP server development if the job is exposing your systems to models you do not control.

Sources: Anthropic, "Pricing", "Prompt caching" and "Model deprecations", platform.claude.com, August 2026. OpenAI, "Deprecations", developers.openai.com. OWASP GenAI Security Project, "OWASP Top 10 for LLM Applications 2026", 4 August 2026. Model Context Protocol, "The 2026-07-28 Specification".