Governed inference
Inference you can prove, not just promise.
kvrun treats the KV cache, the working copy of every prompt, as governed data: encrypted, isolated per tenant, erasable on request and logged on every access. Without slowing your models down.
- Time-to-first-token overhead
- <3%
- Keys, cache and GPU
- Per tenant
- Provable erasure of a session
- <60s
- Audit evidence export
- 1 click
The problem governed inference solves
On long-context work, the KV cache holds 60 to 80% of inference memory, and it is a faithful copy of what users sent. In most serving stacks it is unencrypted, can be shared between tenants through prefix caching, is never logged, and cannot be provably deleted. It is the most sensitive copy of your data, and the least governed.
Five controls, on every request
| Control | What kvrun does |
|---|---|
| Encryption | Every KV block sealed with AES-256-GCM under a key unique to the workspace and session |
| Isolation | Separate key hierarchies per tenant; prefix caching never crosses tenants |
| Erasure | Destroying a session's key makes every copy, on every tier, unreadable at once, with a certificate |
| Residency | Policy-as-code decides where context may live; policies ship through Git, not engine rebuilds |
| Evidence | Allocation, access, movement and erasure written to an append-only log you can export |
More context on the same GPU
Governance and capacity come from the same design. Sealed blocks can move safely from GPU memory to host memory, local disk and object storage, so a deployment holds far longer contexts than its VRAM alone allows. Early internal benchmarks show up to 4× the context per 80 GB GPU.
Evidence for the frameworks you answer to
| Framework | Requirement | Evidence produced |
|---|---|---|
| UK GDPR Art. 17 | Right to erasure | Erasure certificate per session |
| UK GDPR Art. 25 | Data protection by design | Encryption and isolation on by default |
| SOC 2 CC6.1 | Logical access | Per-tenant key isolation |
| SOC 2 CC7.2 | Monitoring | Continuous audit log |
| ISO 27001 | Secure disposal | Cryptographic erasure |
| EU AI Act | Evaluation records | An evaluation on every fine-tune, kept with the weights |
kvrun produces evidence that maps to these controls. Certification of your organisation remains between you and your auditor.
Questions
- What is governed inference?
- Running models so every request is under enforceable, provable controls: isolation, encryption, residency, erasure and evidence. Read the full explainer.
- Which serving engines does it work with?
- The governance layer sits between the serving engine and cache storage, and has been benchmarked with vLLM, SGLang, TensorRT-LLM and llama.cpp.
- How fast is erasure?
- Erasing a session destroys its keys, which takes effect immediately across every tier. The certificate is issued in under a minute.