---
title: "Two Stages, Much Smaller MoE: REAP Expert Pruning Followed by AWQ INT4 Quantization"
description: "We chain REAP pruning 32→16 experts per MoE layer on LFM2.5-8B-A1B with AWQ INT4 group-128 quantization into a 2.79 GB packed artifact for fast inference."
url: https://konic.io/research/reap-awq-moe-compression
published: 2026-07-20
updated: 2026-07-20
section: "Insights"
publisher: "Konic Labs, Inc."
authors: ["Gokalp Katkat", "Ege Sabanci"]
keywords: ["REAP AWQ", "MoE compression", "INT4 quantization", "two-stage compression", "LFM2.5-8B"]
about: ["Two-stage MoE compression", "REAP pruning and AWQ", "MoE quantization", "LFM2.5 compression"]
---

# Two Stages, Much Smaller MoE: REAP Expert Pruning Followed by AWQ INT4 Quantization

> We chain REAP pruning 32→16 experts per MoE layer on LFM2.5-8B-A1B with AWQ INT4 group-128 quantization into a 2.79 GB packed artifact for fast inference.

*Konic Research · Insights · 11 min read*

## Artifacts
- [github.com/egesabanci/reap-cuda](https://github.com/egesabanci/reap-cuda)
- [github.com/egesabanci/awq](https://github.com/egesabanci/awq)
- [huggingface.co/konic-labs/LFM2.5-8B-A1B-REAP-50](https://huggingface.co/konic-labs/LFM2.5-8B-A1B-REAP-50)
- [huggingface.co/konic-labs/LFM2.5-8B-A1B-REAP-50-AWQ-INT4](https://huggingface.co/konic-labs/LFM2.5-8B-A1B-REAP-50-AWQ-INT4)

## Introduction

Mixture-of-Experts (MoE) models trade dense parameter count for routed capacity, which is powerful but expensive to ship: the full Liquid LFM2.5-8B-A1B checkpoint weighs in at 8.47B parameters and 16.94 GB on disk. Two compression techniques attack that footprint from different directions — **expert pruning** removes underused routed parameters, and **weight quantization** shrinks the parameters that remain. They compose: pruning first cuts the expert count, then quantization packs the surviving weights into INT4.

This post records a two-stage pipeline that chains **REAP CUDA** (Router-weighted Expert Activation Pruning) with **AWQ INT4 quantization** via the external `llm-compressor` toolchain. We prune LFM2.5-8B-A1B from 32 to 16 experts across its 22 MoE layers, then quantize the pruned model to W4A16\_ASYM (group 128). The packed artifact is 7.4× smaller by packed-weight equivalent and 6.1× smaller on disk. The quality and throughput results below were measured with its 9.18 GB AWQ-scaled BF16 evaluation derivative for vLLM — not by directly running the 2.79 GB packed INT4 artifact.

These are **recorded run summaries**, not general benchmarks.

## The two-stage pipeline

The pipeline is deliberately sequential and inspectable. Stage 1 (REAP) mutates the live MoE model in place, shrinking the expert stack. Stage 2 (AWQ) is an **external** step run through `llm-compressor` — it is not a REAP feature, and REAP has no knowledge of quantization. The two stages share only the pruned checkpoint on disk.

```text
LFM2.5-8B-A1B (FP16, 32 experts/layer, 22 MoE layers, 8.47B / 16.94 GB)
        │
        ▼  Stage 1 — REAP CUDA (reap-cuda)
        │   calibrate  →  observe router activations  →  rank experts by saliency
        │   prune 32 → 16 experts per MoE layer  →  save pruned checkpoint
        ▼
REAP-50 pruned model (16 experts/layer, 4.59B / 8.57 GB)
        │
        ▼  Stage 2 — AWQ INT4 (external llm-compressor, NOT a REAP feature)
        │   256 calibration samples  →  W4A16_ASYM, group_size = 128
        │   pack surviving weights into INT4  →  export
        ▼
REAP-50 + AWQ INT4 packed artifact (1.15B packed-weight equivalent / 2.79 GB)
        │
        └── vLLM evaluation only: AWQ-scaled BF16 derivative (9.18 GB)
```

## Pipeline compression

Each stage shrinks a different axis. REAP halves the expert count (and therefore the routed parameter budget); AWQ then packs the surviving FP16 weights into 4-bit groups. The *packed-weight equivalent* for the final stage is 4.59B pruned weights divided by four: it excludes scale and zero-point metadata, which is instead reflected in the on-disk artifact size.

*Recorded run summary. The packed artifact is the published INT4 export; quality and vLLM rows use the separate AWQ-scaled BF16 evaluation derivative.*

| Stage | Params | On-disk size | Experts / layer | What changed |
| --- | --- | --- | --- | --- |
| Base | 8.47B | 16.94 GB | 32 | Upstream FP16 checkpoint |
| REAP-50 (pruned) | 4.59B | 8.57 GB | 16 | 32 → 16 experts pruned |
| REAP-50 + AWQ INT4 (packed) | 1.15B packed-weight eq. | 2.79 GB | 16 | W4A16_ASYM, group 128 |
| AWQ-scaled BF16 (evaluated) | 4.59B BF16 | 9.18 GB | 16 | Decompressed for vLLM; AWQ scales preserved |

## Quality — grouped comparison

We measure two capability axes against the upstream base: MATH500 (competition mathematics) and BFCLv3 single-turn function calling. The AWQ-scaled BF16 evaluation derivative is the headline comparison; the pruned-only (REAP-50) row isolates the effect of applying the AWQ stage on the same pruned backbone.

*Recorded run summary. Base scores are upstream published; the final column is the 9.18 GB AWQ-scaled BF16 derivative, not the 2.79 GB packed INT4 artifact.*

| Benchmark | Base | REAP-50 (pruned) | AWQ-scaled BF16 (evaluated) |
| --- | --- | --- | --- |
| MATH500 | 88.76% | 77.0% | 72.0% |
| BFCLv3 single-turn | 64.79% | 59.07% | 57.36% |

| Metric | AWQ-scaled BF16 (evaluated) | Base (LFM2.5-8B-A1B) |
| --- | --- | --- |
| MATH500 | 72 | 88.76 |
| BFCLv3 single-turn | 57.36 | 64.79 |

## Isolating the AWQ stage — controlled comparison

To separate pruning from the AWQ stage, we compare **REAP-50 (pruned only)** against the **AWQ-scaled BF16 evaluation derivative** on MATH500 difficulty levels. Both rows share the same pruned backbone (16 experts, identical routing), which isolates the AWQ scaling path on that backbone — not direct packed-INT4 runtime behavior.

The recorded pattern is concentrated at the hard end: L4 takes a roughly 10-point hit, and L5 the largest relative drop. L1–L2 retain the same displayed score; L3 needs the denominator caveat below.

| Metric | AWQ-scaled BF16 (evaluated) | REAP-50 (pruned) |
| --- | --- | --- |
| MATH500 L1 | 90.7 | 90.7 |
| MATH500 L2 | 84.4 | 84.4 |
| MATH500 L3 | 81.9 | 81.9 |
| MATH500 L4 | 72.7 | 82.8 |
| MATH500 L5 | 49.3 | 58.2 |

*Recorded run summary, controlled for pruning (both rows share the 16-expert backbone). The final column is the AWQ-scaled BF16 evaluation derivative.*

| MATH500 difficulty | REAP-50 (pruned) | AWQ-scaled BF16 (evaluated) | Delta |
| --- | --- | --- | --- |
| L1 | 90.7% | 90.7% | 0.0 pts |
| L2 | 84.4% | 84.4% | 0.0 pts |
| L3 | 81.9% | 81.9% | 0.0 pts |
| L4 | 82.8% | 72.7% | −10.1 pts |
| L5 | 58.2% | 49.3% | −8.9 pts |

> **Data-quality note on L3**
>
>
>
> L3 raw problem counts are intentionally omitted from this summary. The same displayed 81.9% is calculated over different denominators (pruned 95/116; quantized 86/105), and the raw evaluation logs are not committed. The L3 row is therefore not an equal-sample-size comparison; it is a rounded, recorded observation rather than evidence that the AWQ stage is free below L4.

## BFCLv3 capability breakdown

BFCLv3 splits function-calling behavior into several capability slices. The pruned-only and AWQ-scaled BF16 rows are recorded against the same upstream base. The *irrelevance* slice is the one case where the AWQ-stage evaluation **improves** on the pruned-only number — likely noise rather than a real signal, given this is a single recorded run.

*Recorded run summary. Deltas are calculated from the report’s unrounded values; the irrelevance improvement is within single-run noise.*

| BFCL capability | REAP-50 (pruned) | AWQ-scaled BF16 (evaluated) | Delta |
| --- | --- | --- | --- |
| Simple (Python) | 78.5% | 69.3% | −9.3 pts |
| Parallel | 70.0% | 69.5% | −0.5 pts |
| Irrelevance | 80.0% | 83.8% | +3.8 pts |
| Live simple | 60.5% | 51.9% | −8.5 pts |
| Live parallel | 31.3% | 12.5% | −18.8 pts |

> **Comparable BFCL simple slice**
>
>
>
> The Simple row is **Python only**. The AWQ-scaled BF16 evaluation run also recorded 44% on Java and 38% on JavaScript, while the pruned run reported N/A for both; those language-specific results cannot form a pruned-versus-AWQ-stage comparison.

## Throughput (recorded, not a controlled speedup)

The peak throughput figures below were recorded under different batch / concurrency settings and serving paths (offline generation vs. HTTP). The AWQ values use the 9.18 GB AWQ-scaled BF16 evaluation derivative, not the packed INT4 artifact. They are **not**a controlled A/B comparison or direct packed-INT4 speed measurement.

*Recorded run summary. †benchmark-results.md calls the 2,282 tok/s run offline BS 256; ‡compression-metrics.md calls it HTTP concurrency 256. Evaluation throughput is separate from peak serving throughput.*

| Run | Path | Setting | Throughput |
| --- | --- | --- | --- |
| REAP-50 (pruned) | reported-path conflict | BS 256 offline† / concurrency 256 HTTP‡ | 2,282 tok/s |
| AWQ-scaled BF16 | offline generation | BS 128 | 4,271 tok/s |
| AWQ-scaled BF16 | HTTP serving | concurrency 256 | 4,809 tok/s |
| REAP-50 MATH500 eval | evaluation | 815 s total | 1,280 tok/s |
| AWQ-scaled BF16 MATH500 eval | evaluation | 858 s total | 1,281 tok/s |

> **Why these are not comparable as speedups**
>
>
>
> The pruned peak has conflicting path attribution across the two source reports, while the AWQ-scaled BF16 peaks use batch size 128 (offline) or concurrency 256 (HTTP). Evaluation throughput is lower and measured under its own harness. A ratio across these rows would conflate model, batch size, sequence length, runtime, and measurement mode, so we report them only as separate observations.

## Calibration & procedure

Stage 1 (REAP) and Stage 2 (AWQ) use independent calibration passes with different sample counts and token budgets. REAP calibration drives the expert saliency ranking; AWQ calibration drives the per-group scale search. They do not share data.

*Recorded run summary. AWQ is run through the external llm-compressor package; it is not a REAP feature.*

| Parameter | REAP (Stage 1) | AWQ (Stage 2, external) |
| --- | --- | --- |
| Model | LFM2.5-8B-A1B | REAP-50 pruned checkpoint |
| Calibration examples | 100 | 256 |
| Batches / tokens | 40 batches / 40,808 tokens | — |
| Hardware | NVIDIA L4 (23 GiB) | NVIDIA L4 (23 GiB) |
| Expert change | 32 → 16 across 22 MoE layers | none (16 retained) |
| Recipe | REAP saliency, in-place slice | W4A16_ASYM, group 128 |
| Tool | reap-cuda | llm-compressor (external) |

> **REAP calibration count**
>
>
>
> The 100 examples, 40 batches, and 40,808 tokens come from the run logs for the evaluated artifact. Some summary reports say 200 because the source directory is named `evol-codealpaca-calib-200`; this run used its first 100 examples.

> **AWQ is external, not part of REAP**
>
>
>
> The AWQ INT4 step is performed by the external `llm-compressor` package with a **W4A16\_ASYM** recipe (weights 4-bit, activations FP16, asymmetric quantization) and `group_size = 128`, calibrated on 256 samples. REAP produces the pruned checkpoint and stops; it has no quantization code, no scale search, and no INT4 packing. Chaining the two is a deployment choice, not a built-in REAP capability.

## Discussion & limitations

- **Pruning does most of the damage.** Going from Base to REAP-50 costs 11.76 points on MATH500 and 5.72 points on BFCLv3; the AWQ-scaled BF16 evaluation derivative loses a further 5.0 and 1.71 points respectively. This recorded AWQ-stage loss is concentrated at the hard end of each benchmark (MATH L4–L5, BFCL live parallel); it is not a direct packed-INT4 runtime result.
- **Hard problems are most fragile.** MATH500 L5 falls from 58.2% to 49.3% in the AWQ-stage evaluation, and BFCL live parallel from 31.3% to 12.5%. The largest recorded losses are in high-difficulty / live-serving slices; L1–L2 retain the same displayed score, while L3 needs the denominator caveat above.
- **One run, one seed.** Every number is a single recorded run. The variance across calibration seeds is not reported here (raw eval logs were not committed), so the point estimates carry unknown noise. The irrelevance slice improving in the AWQ-stage evaluation is the clearest signal that this is single-run noise rather than a robust effect.
- **Base is upstream, not re-run.** The Base row is the model-card / upstream published evaluation, not a controlled re-run in this pipeline. Comparisons against Base inherit whatever harness differences exist between the upstream evaluation and this run.
- **Throughput is configuration-bound.** The two AWQ-scaled BF16 peak figures and the pruned peak use different batch sizes and serving paths; they are reported as separate observations, not as a controlled speedup or packed-INT4 runtime claim.

## Conclusion

Chaining REAP expert pruning with external AWQ INT4 quantization takes LFM2.5-8B-A1B from 8.47B / 16.94 GB to a 1.15B packed-weight equivalent / 2.79 GB artifact — 7.4× lower by packed-weight equivalent and 6.1× smaller on disk. MATH500 72.0% and BFCLv3 single-turn 57.36% were recorded on a 9.18 GB AWQ-scaled BF16 derivative used for vLLM evaluation, not on the packed INT4 artifact. The loss is staged: pruning accounts for most of the recorded drop, with the AWQ-stage evaluation concentrated at the hardest end.

The takeaway for deployment is that the two techniques compose cleanly because they are independent — REAP shrinks the expert set,`llm-compressor` shrinks the surviving weights — but the quality cost is not uniform across capability slices. Teams considering this stack should measure on their own distribution, not extrapolate from this single recorded run.

REAP CUDA is open-source at [github.com/egesabanci/reap-cuda](https://github.com/egesabanci/reap-cuda), and the from-scratch AWQ implementation at [github.com/egesabanci/awq](https://github.com/egesabanci/awq).

## Sources

- REAP CUDA implementation: [github.com/egesabanci/reap-cuda](https://github.com/egesabanci/reap-cuda) — Stage 1 (expert pruning, saliency calibration).
- From-scratch AWQ implementation: [github.com/egesabanci/awq](https://github.com/egesabanci/awq) — companion AWQ work; the Stage 2 run in this article used the external `llm-compressor` package instead.
- External AWQ toolchain: `llm-compressor`, W4A16\_ASYM recipe, `group_size = 128`, 256 calibration samples.
- Model: Liquid LFM2.5-8B-A1B — 8.47B parameters, 32 experts/layer, 22 MoE layers. Base accuracy figures (MATH500 88.76%, BFCLv3 single-turn 64.79%) are upstream published.
- Evaluation: MATH500 and BFCLv3 (single-turn) on the 9.18 GB AWQ-scaled BF16 derivative; the 2.79 GB packed INT4 artifact was not directly benchmarked. Raw eval logs were not committed.

Key takeaways

- **2.79 GB** — Final on-disk size (from 16.94 GB)
- **72.0%** — MATH500 retained (Base 88.76%)
- **−18.8 pts** — Largest drops: BFCL live relevance / parallel
- **3 peak runs** — Throughput observations (not a controlled speedup)

---

Source: https://konic.io/research/reap-awq-moe-compression