Air-Gapped LLM Deployment: What It Means and When You Need It
TL;DR
An air-gapped LLM runs on infrastructure with no connection to the public internet — the model, the data, and the serving stack all live inside your boundary. This guide explains what air-gapped deployment actually requires, which industries need it, and the practical constraints of running a model with no external dependencies.
What air-gapped actually means
An air-gapped deployment runs a model on infrastructure that has no route to the public internet. The model weights, the data, the serving runtime, and the monitoring all live inside your network boundary. Nothing leaves, and nothing needs to come in.
This is the strictest form of data control. It is not the same as a private cloud VPC, which is isolated from other tenants but still connected to the internet. Air-gapped means physically or logically disconnected — the model must be fully self-contained.
Who needs it
Air-gapped deployment is driven by regulation, national-security requirements, and the most sensitive commercial data. The industries that most often require it:
- Defense and government — where data classification and network isolation are non-negotiable.
- Banking and insurance — where customer data and model behavior are subject to strict audit and jurisdiction rules.
- Healthcare — where patient data carries legal obligations that a hosted API cannot satisfy.
- Manufacturing and energy — where operational data and control systems must stay off the public network.
If your compliance posture requires data to stay in a jurisdiction or on a specific network, air-gapped is the architecture that makes it possible.
What air-gapped deployment requires
Running a model with no external dependencies changes the requirements in three concrete ways:
- Self-contained weights.The model must be delivered as an artifact you can load offline — no runtime downloads, no telemetry calls, no dependency on a vendor's infrastructure.
- On-prem serving. The runtime (vLLM, MLX, or llama.cpp) must run entirely on your hardware, with your keys and your data plane.
- Offline operations. Monitoring, evaluation, and upgrades must work without a connection. Versioned releases you can bring in deliberately are essential.
The trade-off: capability vs. isolation
The honest trade-off is that air-gapped deployment rules out the convenience of a hosted frontier model. You cannot call a bigger model for the hard cases, and you cannot rely on the vendor to keep improving it for you. You are committing to a model that runs entirely inside your wall.
That is why the model choice matters so much. For regulated and sovereign workloads, the right approach is a model that is high-capability enough to handle the task on its own, delivered as a versioned, self-contained artifact — not a raw checkpoint you have to operationalize yourself.
When air-gapped is the right call
Choose air-gapped deployment when the cost of a data or network boundary breach exceeds the cost of running your own model. For most enterprises that threshold is crossed by regulation or by the sensitivity of the data itself. If you are not sure whether you need it, you probably do not — but if your compliance team is asking, the answer is usually yes.
More research
Optimized LFM2.5-VL-3B: FFN Width Pruned, Distillation Aligned, INT4 Quantized
We surgically compress Liquid AI’s LFM2.5-VL-3B — an architecture-searched hybrid conv+attention vision-language model — by cutting FFN width where the search never optimized it: joint-SwiGLU Wanda pruning (10752→8192 and →7168), distillation-to-baseline LoRA recovery, and hand-rolled W4A16 GPTQ + W8 mixed-precision quantization. All three tiers are near-lossless against the base on our paired eval shards — PPL ratios at or below baseline (0.79–0.85), top-5 token agreement ≥ 0.95, quantization adding only +0.02–0.06 nats — across 5.30 GB (−15%), 4.92 GB (−21%), and 1.93 GB packed / 2.22 GB (Konic Optimized) — native INT4/INT8 compressed-tensors in vLLM (−69%). Live comparison on the smallest tier preserves scenes, facts, and tool calling — 10/10 tool rounds, including correct abstention where the base over-triggered. Recorded runs, not general benchmarks.
LFM2.5 Encoder 230M + SigLIP2: A Compact Multimodal Encoder
We augment Liquid AI’s LFM2.5 Encoder 230M — a 230M-parameter bidirectional masked-language encoder — with a SigLIP2 vision tower and 32 learned soft tokens to build an encoder-only multimodal model for retrieval and image-text matching. Clean image-only retrieval on 12,500 held-out MONET pairs reaches 0.1194 image→text R@1 with the BF16 reference; the GPTQ INT4 release retains 0.1091 while shrinking the package from 923.65 MB to 370.46 MB (−59.9%). Cyclic-negative matching AUROC is 0.9733; text-nearest hard negatives and image-conditioned masked-token prediction remain at chance, and the model does not generate.
Two Stages, Much Smaller MoE: REAP Expert Pruning Followed by AWQ INT4 Quantization
We chain two complementary compression stages on Liquid LFM2.5-8B-A1B — REAP CUDA expert pruning (32→16 experts across 22 MoE layers) then external AWQ INT4 quantization (W4A16_ASYM, group 128). The published packed artifact is 1.15B packed-weight equivalent / 2.79 GB; recorded MATH500 and BFCLv3 results use a separate 9.18 GB AWQ-scaled BF16 derivative for vLLM evaluation, not direct packed-INT4 runtime. These are recorded run summaries, not general benchmarks.
From-Scratch AWQ INT4 Quantization on Qwen3-8B
We validate a from-scratch, pure-PyTorch AWQ implementation on Qwen3-8B: group-wise INT4 with per-channel AWQ scaling produces a 4.0× smaller model (13.9 GB → 3.5 GB linear weights) at 1.034× FP16 perplexity on WikiText-2 (10.08 vs 9.75), loaded and run in a real INT4 GEMM runtime. The decisive factor is norm-folding the AWQ scale — 20× more accurate per weight than weight-dequantization.
REAP Expert Pruning for On-Device MoE Models
We present REAP-MLX, an Apple Silicon implementation of Router-weighted Expert Activation Pruning (REAP) for Mixture-of-Experts language models. We evaluate quality retention across compression ratios on the LFM2.5-8B architecture: REAP preserves 96.8% of code generation performance at 25% compression and 91.4% at 50% compression, with less than 0.4 percentage point variance across independent calibration draws.