Private LLM deployment on your own servers
NDDev deploys local LLMs for companies whose policies or contracts forbid sending data to external providers. An open language model runs on your GPUs through vLLM, prompts, answers and logs stay inside your infrastructure, and internal services connect to the model through a single API. NDDev has been working since 2019.
Who it is for
Banks, public bodies, law firms, healthcare companies, manufacturers and anyone whose documents and correspondence must stay on their own servers.
- The model on your GPUs
- One API for every service
- Logs kept inside the company
- Model chosen by benchmark
What we install on your servers
We run the model on vLLM, which serves many users’ requests at once and exposes an API in the format common client libraries understand. In front of the model we put a gateway on LiteLLM or FastAPI with access keys, per-department limits and a request log. The staff chat, the knowledge base and internal systems all connect to one address.
- a task set and a comparison of open models
- server and GPU sizing for your load
- model deployment on vLLM in containers
- an access gateway with keys, limits and a request log
- connection of a staff chat and internal services
- monitoring of GPU load, latency and errors
- a procedure for model updates
We choose the model by benchmark. We build a set of tasks from your work, run several open models of different sizes on it and compare quality, speed and GPU memory needs. From the results we specify the server configuration or check what will run on hardware you already own.
For a prototype, Ollama on one machine is enough; production workloads move to vLLM with monitoring. If some tasks need a stronger external model, we set up a route: anonymised requests go out under rules agreed in advance, and everything else is processed on your side.
Deploying a local model
Hardware is bought after the benchmark, once it is clear which model you need.
Tasks and requirements
We record which data and tasks will go through the model, who will use it and how many requests to expect.
Model benchmark
We compare open models on your tasks and choose the size and serving setup.
Servers
We prepare the hardware specification or deploy everything on your servers: vLLM, the gateway and monitoring.
Connecting services
We connect the chat, knowledge base and internal systems and issue keys to departments.
Handover
We hand over documentation and instructions for updates and monitoring and stay on for support.
Questions about local language models
What hardware do we need?
That depends on the model and the load. After the benchmark on your tasks we specify the configuration: GPU memory, number of GPUs and server requirements. If you already have hardware, we check which models run on it at the speed you need.
How much weaker is a local model than a cloud one?
It depends on the task. For document search, classification and drafts an open model is often enough; for complex reasoning a strong cloud model can be more accurate. So we compare models on your task set first and show you the numbers before any hardware is bought.
Does our data really stay with us?
In local mode the model, prompts, answers and logs are on your servers. External calls appear only if you decide to send some tasks to an external model, and then we agree in advance which data it may receive and what we mask.
Can we train the local model on our documents?
For answers from documents, a search over your knowledge base is usually enough. Fine-tuning helps when the model must learn a style, format or terminology, and we do it with LoRA and QLoRA on your hardware.
Who looks after the model after launch?
Your administrators receive documentation and instructions. We also always offer support: updates to the model and components, improvements and a monthly report, usually for 10% of the project price per month.
How much does deployment cost?
The starting price in the price block covers our work. Servers and GPUs are paid for separately. The total depends on the number of connected services, the load and the access requirements; we fix the price and timeline in the specification.
from $6,700
- Free warranty
- 60 days
Prices are starting prices for information only and are not a public offer. The final price is fixed in the specification.
Language models and company knowledge
Related
Tell us which data must stay inside the company
Describe the tasks for the model, the number of users and the hardware you already have. We will suggest a configuration and a plan for benchmarking models.
We reply within 24 hours