Two Stages, Much Smaller MoE: REAP Expert Pruning Followed by AWQ INT4 Quantization

Mixture-of-Experts (MoE) models trade dense parameter count for routed capacity, which is powerful but expensive to ship: the full Liquid LFM2.5-8B-A1B checkpoint weighs in at 8.47B parameters and 16.94 GB on disk. Two compression techniques attack that footprint from different directions — expert pruning removes underused routed parameters, and weight quantization shrinks the parameters that remain. They compose: pruning first cuts the expert count, then quantization packs the surviving weights into INT4.

This post records a two-stage pipeline that chains REAP CUDA (Router-weighted Expert Activation Pruning) with AWQ INT4 quantization via the external llm-compressortoolchain. We prune LFM2.5-8B-A1B from 32 to 16 experts across its 22 MoE layers, then quantize the pruned model to W4A16_ASYM (group 128). The packed artifact is 7.4× smaller by packed-weight equivalent and 6.1× smaller on disk. The quality and throughput results below were measured with its 9.18 GB AWQ-scaled BF16 evaluation derivative for vLLM — not by directly running the 2.79 GB packed INT4 artifact.

These are recorded run summaries, not general benchmarks.

The pipeline is deliberately sequential and inspectable. Stage 1 (REAP) mutates the live MoE model in place, shrinking the expert stack. Stage 2 (AWQ) is an external step run through llm-compressor — it is not a REAP feature, and REAP has no knowledge of quantization. The two stages share only the pruned checkpoint on disk.

LFM2.5-8B-A1B (FP16, 32 experts/layer, 22 MoE layers, 8.47B / 16.94 GB)
        │
        ▼  Stage 1 — REAP CUDA (reap-cuda)
        │   calibrate  →  observe router activations  →  rank experts by saliency
        │   prune 32 → 16 experts per MoE layer  →  save pruned checkpoint
        ▼
REAP-50 pruned model (16 experts/layer, 4.59B / 8.57 GB)
        │
        ▼  Stage 2 — AWQ INT4 (external llm-compressor, NOT a REAP feature)
        │   256 calibration samples  →  W4A16_ASYM, group_size = 128
        │   pack surviving weights into INT4  →  export
        ▼
REAP-50 + AWQ INT4 packed artifact (1.15B packed-weight equivalent / 2.79 GB)
        │
        └── vLLM evaluation only: AWQ-scaled BF16 derivative (9.18 GB)

Each stage shrinks a different axis. REAP halves the expert count (and therefore the routed parameter budget); AWQ then packs the surviving FP16 weights into 4-bit groups. The packed-weight equivalent for the final stage is 4.59B pruned weights divided by four: it excludes scale and zero-point metadata, which is instead reflected in the on-disk artifact size.

We measure two capability axes against the upstream base: MATH500 (competition mathematics) and BFCLv3 single-turn function calling. The AWQ-scaled BF16 evaluation derivative is the headline comparison; the pruned-only (REAP-50) row isolates the effect of applying the AWQ stage on the same pruned backbone.

To separate pruning from the AWQ stage, we compare REAP-50 (pruned only) against the AWQ-scaled BF16 evaluation derivative on MATH500 difficulty levels. Both rows share the same pruned backbone (16 experts, identical routing), which isolates the AWQ scaling path on that backbone — not direct packed-INT4 runtime behavior.

The recorded pattern is concentrated at the hard end: L4 takes a roughly 10-point hit, and L5 the largest relative drop. L1–L2 retain the same displayed score; L3 needs the denominator caveat below.

BFCLv3 splits function-calling behavior into several capability slices. The pruned-only and AWQ-scaled BF16 rows are recorded against the same upstream base. The irrelevance slice is the one case where the AWQ-stage evaluation improves on the pruned-only number — likely noise rather than a real signal, given this is a single recorded run.

The peak throughput figures below were recorded under different batch / concurrency settings and serving paths (offline generation vs. HTTP). The AWQ values use the 9.18 GB AWQ-scaled BF16 evaluation derivative, not the packed INT4 artifact. They are nota controlled A/B comparison or direct packed-INT4 speed measurement.

Stage 1 (REAP) and Stage 2 (AWQ) use independent calibration passes with different sample counts and token budgets. REAP calibration drives the expert saliency ranking; AWQ calibration drives the per-group scale search. They do not share data.

• Pruning does most of the damage. Going from Base to REAP-50 costs 11.76 points on MATH500 and 5.72 points on BFCLv3; the AWQ-scaled BF16 evaluation derivative loses a further 5.0 and 1.71 points respectively. This recorded AWQ-stage loss is concentrated at the hard end of each benchmark (MATH L4–L5, BFCL live parallel); it is not a direct packed-INT4 runtime result.
• Hard problems are most fragile. MATH500 L5 falls from 58.2% to 49.3% in the AWQ-stage evaluation, and BFCL live parallel from 31.3% to 12.5%. The largest recorded losses are in high-difficulty / live-serving slices; L1–L2 retain the same displayed score, while L3 needs the denominator caveat above.
• One run, one seed. Every number is a single recorded run. The variance across calibration seeds is not reported here (raw eval logs were not committed), so the point estimates carry unknown noise. The irrelevance slice improving in the AWQ-stage evaluation is the clearest signal that this is single-run noise rather than a robust effect.
• Base is upstream, not re-run. The Base row is the model-card / upstream published evaluation, not a controlled re-run in this pipeline. Comparisons against Base inherit whatever harness differences exist between the upstream evaluation and this run.
• Throughput is configuration-bound. The two AWQ-scaled BF16 peak figures and the pruned peak use different batch sizes and serving paths; they are reported as separate observations, not as a controlled speedup or packed-INT4 runtime claim.

Chaining REAP expert pruning with external AWQ INT4 quantization takes LFM2.5-8B-A1B from 8.47B / 16.94 GB to a 1.15B packed-weight equivalent / 2.79 GB artifact — 7.4× lower by packed-weight equivalent and 6.1× smaller on disk. MATH500 72.0% and BFCLv3 single-turn 57.36% were recorded on a 9.18 GB AWQ-scaled BF16 derivative used for vLLM evaluation, not on the packed INT4 artifact. The loss is staged: pruning accounts for most of the recorded drop, with the AWQ-stage evaluation concentrated at the hardest end.

The takeaway for deployment is that the two techniques compose cleanly because they are independent — REAP shrinks the expert set,llm-compressor shrinks the surviving weights — but the quality cost is not uniform across capability slices. Teams considering this stack should measure on their own distribution, not extrapolate from this single recorded run.

REAP CUDA is open-source at github.com/egesabanci/reap-cuda, and the from-scratch AWQ implementation at github.com/egesabanci/awq.

• REAP CUDA implementation: github.com/egesabanci/reap-cuda — Stage 1 (expert pruning, saliency calibration).
• From-scratch AWQ implementation: github.com/egesabanci/awq — companion AWQ work; the Stage 2 run in this article used the external llm-compressor package instead.
• External AWQ toolchain: llm-compressor, W4A16_ASYM recipe, group_size = 128, 256 calibration samples.
• Model: Liquid LFM2.5-8B-A1B — 8.47B parameters, 32 experts/layer, 22 MoE layers. Base accuracy figures (MATH500 88.76%, BFCLv3 single-turn 64.79%) are upstream published.
• Evaluation: MATH500 and BFCLv3 (single-turn) on the 9.18 GB AWQ-scaled BF16 derivative; the 2.79 GB packed INT4 artifact was not directly benchmarked. Raw eval logs were not committed.