What changes in a company when it adopts LLMs

A pilot with a language model almost always looks convincing: the twenty-example demo works, leadership is impressed, timelines look short. The trouble starts in month two, and it is not about the model.

The three things that break first

1. Determinism. A classical service returns the same output for the same input. An LLM does not, and every QA process built on exact matches stops working.
2. Economics. Cost moves from "developer hours" to "cost per request". Without per-request accounting you learn about the problem from the invoice.
3. Data. People start pasting into prompts what used to never leave the building: contracts, personal data, source code.

None of the three is solved by picking a different model. All three are solved by process and by the engineering around it.

What appears in engineering practice

The first thing a team needs is a log of every model call. Without it you cannot account for the money, cannot investigate an incident, and cannot show an auditor that personal data stayed inside.

{
  "request_id": "01J9K2Q7B4N8ZP",
  "user_role": "support-agent",
  "prompt_version": "faq-v3",
  "model": "internal-llm",
  "tokens_in": 812,
  "tokens_out": 143,
  "cost_kzt": 4.7,
  "grounded": true,
  "sources": ["kb/1042", "kb/1187"],
  "escalated_to_human": false
}

Next come versioned prompts (prompt_version), a regression evaluation suite in CI, and one rule: an answer without a source is not shown to the user.

What changes in the roles

• An analyst labels data and owns the evaluation set — a standing role, not a one-off task.
• Engineers own cost per request the way they used to own latency.
• Legal and security join before the pilot, not after the incident.
• Leadership accepts that some answers will be wrong and decides in advance what compensates for that.

A company wins not because it connected a model, but because it learned to measure the model’s value and its cost.

Corporate programmes built to a specific brief are on the for business page.