Local AI Inference Inside Your Perimeter
From a two-person startup to a 1000+ seat corporation: your models, your hardware, your data.
API bills grow with every seat and every token, and the math breaks long before enterprise scale. We design, build, and operate local inference: on-prem GPU clusters and edge fleets on the Apple Silicon your team already owns, with the economics calculated before a single unit of hardware is bought. The goal is simple: turn runaway inference OPEX into smart one-time CAPEX.
What we take on
Inference economics and TCO modeling
The decision layer first: numbers, not vibes.
- Token-volume profiling and workload analysis across your teams
- TCO model: API spend vs cluster amortization, break-even point, sensitivity to growth
- Model selection for the quality bar you actually need (open-weights vs frontier API)
On-prem inference platforms
Production clusters inside your network perimeter, including air-gapped.
- GPU cluster design and deployment (vLLM/SGLang serving, quantization, batching)
- Open-weights model fleet with evals against your real tasks
- Kubernetes or bare metal, observability, autoscaling, on-call runbooks
Edge inference on devices you already own
The cheapest datacenter is the one already on your desks.
- Local inference on employee Apple Silicon (MLX, unified memory does the heavy lifting)
- Consumer-grade GPU clusters on RTX 3090/5090 where they beat datacenter cards on cost
- Smart routing between edge, on-prem cluster, and frontier APIs
Security, compliance, and operations
Local inference is a practice, not a one-time install.
- Data never leaves the perimeter: isolation, access control, audit trails
- Model updates, eval regression gates, capacity planning
- Handover to your team or ongoing operations under a retainer
Frequently asked questions
When does local inference actually pay off?
When usage is steady and seat counts grow. The typical pattern: a team spends five to seven figures a year on per-token APIs for workloads that open-weights models handle at the required quality. A cluster bought once amortizes over years; our assessment shows your break-even point in weeks of usage data, not opinions.
Which models can run inside our perimeter?
Current open-weights families (Llama, Qwen, DeepSeek, Mistral and others) across sizes from edge-friendly to datacenter-grade. We benchmark candidates against your actual tasks and quality bar before recommending hardware.
Is Apple Silicon a serious inference platform?
For many workloads, yes. Unified memory on M-series machines runs surprisingly large models locally, and your team already owns the hardware. We deploy MLX-based inference on employee devices for latency-tolerant and privacy-critical tasks, with the heavy shared load on the cluster.
Consumer GPUs like RTX 3090/5090 in production, really?
Where it makes sense, absolutely: for many mid-size models a consumer-grade cluster delivers several times better cost per token than datacenter cards. We are honest about the limits too: no NVLink, less VRAM per card, and we design around them or say when datacenter hardware is the right call.
What about data privacy and compliance?
That is usually the reason to go local. Prompts, documents, and outputs never leave your network; air-gapped deployments are supported. You get isolation, access control, and audit trails as part of the design.
Can we mix local inference with frontier APIs?
Yes, and most rational setups do: local models for the bulk of tokens, frontier APIs for the hardest reasoning. We build the routing layer so each request goes to the cheapest tier that meets its quality bar.
How does an engagement start and what does it cost?
It starts with a fixed-scope economics assessment on your real usage data. You get the TCO model, hardware recommendation, and rollout plan. Build-out is then quoted fixed; ongoing operations run on a monthly retainer.
Who operates the cluster afterwards?
Your choice: we document everything and hand it to your team, or we keep operating it under a retainer, including model updates and eval gates.
Count the economics before buying hardware
Send a note about your workloads and current API spend: we reply within one business day with the shape of an assessment and a fixed price for it.
Email ivan@relux.works