Open-source models vs OpenAI for regulated work: a UK buyer’s guide
For regulated work in the UK, the choice between open-weight models (Qwen, Gemma, Mistral, DeepSeek, Llama) and OpenAI's models is mostly a choice about control. Open models give you the weights, the hosting location and the serving stack. OpenAI gives you the strongest models on the hardest tasks. On many everyday regulated tasks, the quality gap is now small enough that control decides the question. This guide sets out where the gap is real, what each option costs, and how to decide.
It compares models, not vendors. You can run open models on many platforms, kvrun among them, and reach OpenAI's models through OpenAI directly or through Azure.
Quality, honestly, by task
Public leaderboards move every month and rarely match your workload, so treat this as a starting point to test, not a verdict.
| Task | Typical gap today | Notes |
|---|---|---|
| Extraction and classification | Small or none | Structured outputs from documents, routing, tagging. Open models in the 8B to 30B range often match, and fine-tuning closes what is left. |
| Summarisation of your documents | Small | Quality depends more on prompt design and retrieval than on the model. |
| Retrieval-augmented Q&A | Small to moderate | Grounded answers over your own corpus. Larger open models do well; retrieval quality matters most. |
| Routine code | Moderate | Good open coding models exist. OpenAI still leads on larger refactors and unfamiliar codebases. |
| Long multi-step reasoning | Large | Hard analytical problems, long chains of tool calls, and agentic work over many steps. Frontier closed models are still clearly ahead. |
| Broad world knowledge | Moderate to large | Questions that depend on what the model memorised rather than what you gave it. |
The pattern is consistent. The more a task depends on your data, provided in the prompt or taught through fine-tuning, the smaller the gap. The more it depends on the model's own reasoning and knowledge, the larger it is.
Build an evaluation set from 100 to 300 of your own real cases before you choose. Score every candidate model on it, with the same prompts. It is the single most useful thing you can do, and it takes days, not months.
Cost
OpenAI charges per token. Open models are charged however you host them: per token on a shared API, or per GPU-hour on dedicated hardware.
- Per token, at low volume, OpenAI's smaller models and shared open-model APIs are both cheap, and the difference rarely matters.
- At steady, high volume, a dedicated GPU running an open model usually costs less per token, because you pay for the hardware, not the tokens.
- Smaller can be enough. A fine-tuned 8B model that matches a frontier model on your extraction task is far cheaper to run at any volume.
Include the cost of controls in the comparison: retention approvals, data-transfer paperwork, and any engineering you need to meet your auditors.
Control
This is where open models are strongest, and for regulated work it is often decisive.
| Control | OpenAI | Open models |
|---|---|---|
| Model version | OpenAI retires and updates models on its schedule | You pin the exact weights for as long as you like |
| Where it runs | OpenAI's infrastructure, with regional residency options for eligible customers | Anywhere you choose, including London or your own servers |
| Retention | No training on API data by default; zero retention for eligible customers | Whatever your host provides, or nothing at all if you self-host |
| Fine-tuning | Supported for selected models, run by OpenAI | Full control: LoRA or QLoRA on your data, on your terms |
| Inspection | Closed weights | Open weights you can evaluate, red-team and archive |
Version pinning matters more in regulated work than people expect. If a model is part of a validated process, such as triaging complaints or drafting regulated advice, a silent change of model behaviour can invalidate that validation. Open weights do not change unless you change them.
What OpenAI has improved
OpenAI's enterprise position is stronger than it was. API data is not used for training by default, zero data retention is available to eligible customers, and it has added regional data residency options for enterprise and API customers. Check which regions, products and endpoints your agreement covers, and whether a UK-only requirement is met. If it is, and the quality gap on your task is large, OpenAI may simply be the right answer.
How to decide
- Classify your data. Personal, special-category, privileged or confidential data raises the bar for control.
- Build the evaluation set from your own cases, and score two or three open models and one OpenAI model.
- Look at the gap by task, not overall. Use the frontier model only where it clearly wins.
- Price the controls, not just the tokens.
- Consider a split. Many teams send regulated data to open models they control, and non-sensitive reasoning-heavy work to a frontier model.
kvrun runs open models on dedicated GPUs in London behind a governed KV cache, with an OpenAI-compatible API, so moving a workload between the two is usually a change of base URL and model name. See Private LLM hosting in the UK for the hosting options in more detail.
Questions
- Are open-source models good enough for regulated work?
- For extraction, classification, summarisation and grounded Q&A, usually yes, especially after fine-tuning. For long multi-step reasoning, frontier closed models still lead. Test on your own cases.
- Is "open source" the right term?
- Most are open-weight: the weights are published, but the training data and code may not be, and licences vary. Read each model's licence before commercial use.
- Can I fine-tune an open model on personal data?
- Yes, with a lawful basis and data minimisation. Treat the resulting adapter as containing personal data. See LoRA vs QLoRA.