AI/ML engineering · Infrastructure
Private LLM deployment
We carry out private LLM deployment for enterprises that cannot send data to a public service: hosting language models in your own cloud account, region or data centre, with network isolation, a gateway that controls and logs access, retrieval over your own documents, evaluation and capacity planning. Staff get the benefit of generative AI, and your data stays within the boundary you set. You own the platform and the code.
Checklist · AI readiness
Enterprise AI readiness and governance checklist
A checklist for enterprises planning machine learning, language model or agent projects: use case, data, hosting, security, evaluation, human oversight, governance, monitoring and cost.- 01Nine areas to settle before a model goes near production.
- 02The security items specific to language models and agents.
- 03Evaluation and monitoring questions that regulators and auditors ask.
- 04Seven questions to put to any vendor or development partner.
Free download
Get it on screen in a few seconds.
What the system does, module by module.
- 01
Deployment options
Assessment of hosted models in your own cloud tenancy and region, private endpoints of managed services, and self-hosted open-weight models on your own GPUs. - 02
Model selection and evaluation
Comparison of candidate models on your own tasks and languages, measuring quality, latency and cost, not just public benchmarks. - 03
Network isolation
Private networking, no public endpoints, egress controls and separation of environments, so prompts and documents never leave the boundary. - 04
Model gateway
One entry point for applications that handles authentication, routing between models, rate limits, quotas, caching and cost attribution by team. - 05
Retrieval over private data
Indexing of internal documents and databases with permissions carried through to search results, so a user only retrieves what they may see. - 06
Guardrails and data protection
Redaction of sensitive data, input and output checks, defences against prompt injection and policies for what may be sent to which model. - 07
Serving and capacity
Inference servers, batching, quantisation and autoscaling sized to actual load, with a plan for peak use and failure of a node. - 08
Logging, audit and monitoring
Records of who asked what, with controls on retention and access, and metrics on quality, latency, usage and cost. - 09
Application enablement
SDKs, templates and a catalogue of approved patterns so teams can build assistants and agents on the platform without repeating security work.
Reference architecture.
Standards and interfaces
What we design against.
- OWASP Top 10 for LLM Applications
- Security risks for LLM applications, used as the review checklist for the platform and for each application built on it.
- NIST AI RMF and ISO/IEC 42001
- Frameworks for AI governance, risk management and accountability, mapped to platform controls.
- ISO/IEC 27001 and SOC 2
- Information security management controls that the platform's operation must satisfy.
- Data residency and sovereignty rules
- Requirements on where data and models may be hosted and processed, by country and sector.
- OpenAI-compatible and standard APIs
- A stable interface so applications can switch models without code changes.
- OpenTelemetry
- Tracing and metrics standard for observing requests across gateway, retrieval and serving.
Data model
The core entities.
- Model and deployment
- Model, version, host, region, capacity, status and approved uses.
- Application and client
- Calling application, owner, permissions, quotas and cost centre.
- Request log
- Caller, model, time, size, latency, guardrail results and, where permitted, content.
- Index and document
- Source, access control list, chunks, embeddings and update time.
- Policy
- Rules on data classes, models and uses, with version and owner.
- Evaluation suite
- Tasks, reference answers, scores by model and prompt version.
- Usage and cost
- Tokens or GPU time by team, model and period, with budgets and alerts.
When custom development is the right call.
Controls designed in, not added later.
Boundary and egress control
Private endpoints and denied-by-default outbound access, so data cannot leave except through approved paths.Identity and permissions
Single sign-on, group-based access to models and indexes, and retrieval filtered by the user's own rights.Content protection
Redaction of sensitive fields, classification-based routing and controls on logging content.Prompt injection defence
Untrusted content cannot override policy, and tools available to models are restricted and logged.Supply chain controls
Provenance, scanning and approval of model weights and serving software before use.
From discovery to a system you run.
- 01
Requirements and constraints
We agree the use cases, data classes, residency and security constraints, and the load expected, with security, legal and the business. - 02
Model evaluation and architecture
Candidate models are evaluated on your tasks, and the hosting, network and gateway architecture are designed and reviewed. - 03
Build the platform core
Gateway, identity integration, serving and logging go live in a pilot environment with one application. - 04
Add retrieval and guardrails
Permissioned retrieval, redaction and evaluation are added, and the first assistants and agents are built on the platform. - 05
Scale and govern
More applications and teams join, with capacity, cost and policy reviews on a regular cycle.
Relevant work
Where to see our work.
Common questions.
- Is a private LLM as good as the public services?
- Hosted models inside your own cloud tenancy are the same models, with your boundary around them. Self-hosted open-weight models are strong for many tasks and weaker on others. We measure quality on your tasks before you commit.
- Do we need our own GPUs?
- Not necessarily. Managed model services with private endpoints in your region avoid operating GPUs. Self-hosting needs GPUs and operational skill, and makes sense for strict data boundaries or large, steady volumes.
- How do you stop staff retrieving documents they should not see?
- Document permissions are indexed alongside content, and every search is filtered by the signed-in user's rights, so the assistant can only quote what the user could open themselves.
- How do you control cost?
- Through a gateway that attributes usage to teams, applies quotas, caches repeated requests and routes simple tasks to smaller models, with reports and alerts on spend.
- Who owns the platform?
- You do. The platform, the configuration, the evaluation sets and the code stay yours.