Prune, heal, quantize

1. Research question

Expert pruning and weight quantization solve different parts of the MoE deployment problem. Pruning removes whole functions and reduces the number of expert slots. Quantization keeps the functions and reduces the cost of each slot. Applied independently, either one can use up the quality budget before the model reaches a useful memory footprint.

REAP changes the pruning signal from frequency alone to router-weighted expert activation: an expert matters when the router sends it real weight and its output has real magnitude. The first question is whether the surviving network can be healed enough that low-bit quantization becomes a small, additive loss rather than a compounding one.

Does the order prune, heal, quantize keep more generative function than treating expert count and precision as independent compression knobs?

The next question is finer: after pruning, should every surviving expert get the same precision? A uniform format ignores that experts differ in routing frequency, output impact, domain specialization and spill cost.

2. Completed protocol

Subject

Qwen3-30B-A3B, a 128-expert sparse MoE, is a tractable middle scale: large enough for real expert redundancy, small enough for controlled pruning and healing experiments.

Pruning signal

For each routed token, the experiment combines the router gate value with the norm of the expert output. Scores are aggregated per expert and the lowest-saliency half are removed from routing. The shared expert is not pruned.

Healing

The decisive arm updates the surviving model function, not only the router. A 600-step paged 8-bit AdamW run heals experts, attention, norms, router, embeddings and output head. Masked experts receive no useful gradient. This is still a functional experiment; physical tensor slicing is a separate artifact step.

Quantization

The healed survivors go through an NVFP4 simulation with block-scaled 4-bit weights. The study then reruns ARC, ARC-Easy and perplexity with the same evaluation harness and split.

3. Preliminary results

At this scale, one-shot 50% REAP pruning is already close to baseline. Full-function healing restores the small loss and slightly beats the baseline on all three reported metrics. Adding NVFP4 costs 0.014 ARC against the healed arm and finishes 0.012 below baseline, while perplexity stays better than baseline.

The inference is narrow: in this setup, quantizing after healing adds a small loss instead of reopening the full pruning loss. It does not show that the pipeline is near-lossless on another MoE, at a deeper prune ratio, or on a generative benchmark suite.

4. Failed arms that shaped the protocol

1. FailedHeal only the router.

   Router-only healing dropped the 50%-pruned model from 0.518 to 0.510 ARC, worse than no healing at all. Routing cannot rebuild expert function that is no longer there; the surviving weights are what need the gradient.

2. RejectedGo straight to 75% pruning.

   The healed 75% arm reached 0.464 ARC against 0.522 baseline. The loss was gradual, not catastrophic, but too large for the intended artifact.

3. WarningUse a small MoE as a pruning proxy.

   OLMoE-7B lost 0.12 ARC at a 50% prune where Qwen3-30B-A3B lost 0.004, a thirtyfold difference. Expert redundancy grows with scale, so small-MoE pruning results do not transfer upward.

5. Extension: several precision tiers

The completed arm quantizes every surviving expert the same way. The next study asks whether a fixed memory budget is better spent unevenly. Candidate signals:

• router frequency and accumulated gate mass;
• router-weighted output magnitude, aligned with the pruning signal;
• domain-conditional specialization and calibration-set drift;
• sensitivity of target logits to expert-local quantization noise;
• residency and spill cost in the actual inference runtime.

The first deployment target is a physically pruned Hy3 REAP50 artifact. The checkpoint and a CPU-only loader path are ready: the artifact retains 96 routed experts across MoE layers and uses top-8 sigmoid routing. Runtime quality and performance stay unverified until the target-rig checks run. That distinction matters: a working loader is not evidence that the compression works.

6. What this does not establish

• The Qwen experiment masks experts; it does not measure the memory or latency of a physically compacted checkpoint.
• NVFP4 is simulated in the completed study, so kernel and layout effects are outside the result.
• ARC, ARC-Easy and perplexity are not enough for code, tool-use or long-form generation claims.
• The multi-precision policy is a hypothesis and a figure annotation, not a completed result.
• Hy3 loader preparation is CPU-only; no GPU quality or throughput conclusion is claimed.
• Weight similarity does not predict compression safety: across the merge experiments before this protocol, expert similarity correlated with damage at 0.07. Saliency has to be measured in activation space, which is why the pruning signal is router-weighted.
• Per-layer tolerance varies by an order of magnitude with depth. Compressing layers independently compounds badly without a full-function heal.

7. Artifacts and release plan

The runtime work lives in memra, including the model loading and expert-spill code the deployment measurements run on. The first physical artifact is public: Hy3-REAP-Layer103p5, a receipt-bound expert overlay for Hy3 built from this study's recipe and selected on routing displacement.

Hy3 screening results, 16 July 2026

• The community REAP50 mask failed the bounded screening gate: 0.239 domain-macro accuracy against 0.727 for the unpruned ceiling on the same screen. A published mask is not a usable starting point here.
• Layer-balanced pruning with private calibration held 83.5% retention at the 100 GB deployment cap.
• The published Layer103.5 overlay finished 76 of 115 screening tasks against the matched Layer100 allocation's 73 of 115: ahead, not yet a final verdict.
• Per-projection damage was then measured across seven precision formats (15,168 expert rows). It feeds one global integer program that assigns precision and pruning together under the byte ceiling, with formats competing on measured error and no bit-width prior.

1. Available now Method, completed Qwen table, failed arms, limits and extension protocol.
2. Next Physical expert compaction, checkpoint manifest, generative evaluation and runtime measurements.
3. Paper v1.0 Hy3 replication, multi-tier precision ablation, multi-corpus calibration, PDF and BibTeX.