Applied ML research, written as it runs

Models should be measured where they run.

I study how to make large-model inference smaller, faster, and still trustworthy: train the compact head, prune the experts, assign the precision—then measure the claim in a runtime built to expose it. Everything here is a working record; the results are the signal. That runtime is memra, and it serves a public API at inference.tiyuvta.ai — the same measurements, on a card you do not have to own.

Systems software engineer, AWS ElastiCache · Maintainer, Valkey GLIDE · Founder, agent-sh

Research / 01

Working papers, before the polish hides the uncertainty.

Each page separates the question, protocol, evidence, negative results, and what remains unresolved. The numbers are measured, the failed arms stay visible.

WP–01 · speculative decoding Study concluded

Small-vocabulary MTP heads for memory-bound speculative decoding

Can a compact, separately trained MTP head replace the inherited full-width block and 248k-row output projection without surrendering the acceptance that makes speculation useful? The audit answered: no—function cuts pay an off-distribution tax that fidelity cuts do not, and the zero-training trimmed-vocabulary recipe is the one that survives.

Current signal 1.82× measured wall time at 9B; 2.88× best composed cell at 27B Measured · memra engine · trimmed NVFP4 head · quantized-base compose

Open study ↗

WP–02 · MoE compression Artifact published

Prune, heal, quantize: composing expert pruning with low-bit MoE deployment

A study of compression order: remove experts with router-weighted activation evidence, heal the surviving function, then assign low precision—rather than asking quantization or pruning to carry the full loss alone.

Current signal 128 → 64 experts; final ARC within 0.012 of baseline Preliminary · Qwen3-30B-A3B · NVFP4 simulation

Open study ↗

Research index and methodology

Evidence ledger / 02

The result can be yes, no, or not yet.

I keep the failed arms because they define the research more honestly than a clean success narrative does.

1.82× / 2.88× measured wall time, trimmed NVFP4 drafter vs plain decode — 9B and composed 27B

2.61 vs 2.99 perplexity after 50% expert prune + heal, against the unpruned baseline

−6.0% PPL imatrix-aware NVFP4 scale search at 9B, in review at llama.cpp

Refuted

A public REAP50 mask is a usable starting point for Hy3.

At the bounded screening gate the community mask collapsed—0.239 domain-macro against 0.727 for the unpruned ceiling on the same screen. Layer-balanced pruning with private calibration holds 83.5% retention at the 100GB cap, and the published Layer103.5 overlay finishes 76/115 against matched Layer100's 73/115.

16 Jul 2026

Refuted

A distilled compact head can match the trimmed-vocabulary drafter off-distribution.

Function cuts—pruning, low-rank, distillation—keep 97% of their acceptance on calibration-register text and pay 10–19 retention points under real shift; fidelity cuts pay at most 1.9. The mechanism is statistical, not a size problem: parameters serving rare content accumulate no mass on any finite calibration sample. The zero-training recipe stands.

19 Jul 2026

Supported

The trimmed-vocabulary head pays for itself in measured wall time.

End-to-end in the memra engine, the trimmed NVFP4 head reaches 1.82× on MT-Bench at 9B, and composes with a quantized base to 137.5 tok/s at 27B—2.88×, the best measured cell so far.

19 Jul 2026

Supported

A smaller MTP head can learn useful draft behavior at a fraction of the compute.

Chain-distilled StudentSV reached 0.507 recursive chain acceptance against the teacher's 0.593—85.5%—at roughly an eighth of the draft compute, with slots 1–3 individually exceeding the teacher.

12 Jul 2026

Refuted

NVFP4 compression alone degrades the co-trained MTP head.

It did not. Across forced replay, generation-distribution replay, and seeded sampled runs, two clean quantization arms showed no acceptance loss versus BF16. The original healing premise was closed.

10 Jul 2026

Supported

Draft heads should train on the model's own generations.

Own-output training reached 0.548 held-out acceptance versus 0.376 for a generic corpus at one third of the steps. The training distribution has to match the serving distribution.

11 Jul 2026

Supported

Looped-and-scaled layers beat a FLOPs-matched vanilla transformer once data is the wall.

Layer-loop weight sharing with ε=λ/(N√L) residual scaling—two published results that had never been combined. In data-constrained pretraining at d=768 the looped model won on 3 of 3 seeds (−0.039±0.003 best-achievable val loss) with 34% fewer parameters, and the gain over unscaled looping follows 0.0122·(N−1) at R²=0.993. The fresh-data version of the claim was refuted at every N tested.

4 Aug 2026

Refuted

A post-hoc memory-layer retrofit keeps its published forgetting advantage at 4B.

The retrofit itself is essentially free once the routing warmup runs at lr/10—but the forgetting advantage over LoRA failed to reach significance in 12 of 12 probe-arm intervals across three configurations, including a reference-faithful reproduction. Power analysis shows the published-style n=3 comparison could never have resolved it.

3 Aug 2026

Open

Should surviving REAP experts receive different precision budgets?

The measured groundwork exists: per-projection damage across seven formats—15,168 expert rows—feeding one global integer program that chooses precision and pruning together under a byte ceiling. The generative evaluation decides whether the allocation survives.

21 Jul 2026

Experimental systems / 03

The apparatus is part of the argument.

Research claims about inference depend on loaders, kernels, caches, evaluation code, and hardware behavior. I build enough of that stack to know what the number includes.

memra decode throughput, one RTX 5090 Laptop, one session—N=5 interleaved medians, same prompts, speculative decode gated byte-identical to plain. Short-code prompt class shown. The tuned flagship runs elsewhere: Qwen3.8-27B on an RTX PRO 6000, 140 tok/s with a masked-ranks draft head and 0.156 s to first token, served at inference.tiyuvta.ai.

01 / engine memra

A from-scratch Rust + CUDA inference engine used as an experimental instrument for MTP speculative decoding—greedy and sampled—NVFP4 and FP8 checkpoint serving, MoE spill, and exactness gates on Blackwell. Safetensors is the tuned path, GGUF stays supported, and speculative decode is gated byte-identical to plain decode per request. It also serves a public endpoint, so the behaviour can be tried without the card.

public · MIT · hosted at inference.tiyuvta.ai ↗ 02 / study hqmtp

The research harness for quantized-trunk MTP agreement and compact StudentSV heads: extraction, training, replay evaluation, corpus controls, and negative-result ledgers.

public · released ↗ 03 / compress REAP × quant

Expert saliency, pruning, full-function healing, low-bit simulation, and the loader work needed to carry REAP50 artifacts into a constrained inference runtime.

working paper 04 / train training-recipe studies

recipe-lab: looped-and-scaled layers beat FLOPs-matched vanilla on 3/3 seeds in the data-constrained regime. mem-retrofit: the memory-layer graft is free, but its published forgetting advantage never cleared a confidence interval. block-routed-swiglu: a near-free routing kernel whose capability test failed. All pre-registered, receipts in each public repo.

public · result notes ↗ 05 / migrate CRIU live migration

Copy-on-write live migration of a 200 GB loaded in-memory datastore with under 50 ms of freeze: userfaultfd page prefetch, priority-ordered VMA transfer, and lazy-page lifecycle fixes.

research fork ↗ 06 / systems Valkey GLIDE

Production systems work at a different scale: an official multi-language Valkey client. 158 authored pull requests merged and 480 reviewed—reliability, releases, and the support that starts after an implementation works once.

maintainer ↗ 07 / desktop ChatGPT desktop for Linux

Bringing OpenAI's desktop app to Linux: Wayland input, accessibility-tree automation, packaging across five formats, and the updater plumbing on a 3k-star community project.

collaborator ↗ 08 / tools agent-sh

Open-source agent tooling organization: agnix, a 444-rule linter and language server for agent configuration; computer-use-linux and agent-workspace, isolated desktop control for agents on Linux; plus code review and repository intelligence.

founder ↗

Notes / 04

Long-form writing, with room to think.

The paper pages are formal and evidence-heavy. The notes are where systems lessons, design arguments, and open-source experience can breathe.

28 Mar 2026

How AI workloads changed the queue I was already building

What streaming, spending, waiting, and long-lived work revealed about the queue's model of execution.

5 min read 12 Feb 2026

Your AI agent configs are probably broken

Why silent failures in skills, hooks, memory, MCP, and instruction files needed source-backed validation.

5 min read

All notes, talks, and answers

About / 05

A nontraditional route into research, built in public.

Avi Fenesh smiling at a table — Avi Fenesh · Tel Aviv

I am an independent ML researcher and a systems software engineer at AWS ElastiCache. My route into research runs through implementation: reproduce the baseline, expose the hidden assumption, train the missing comparison, and let the result change the plan.

I study computer science at The Open University of Israel while maintaining open-source systems and running research outside a traditional lab. That path has made method unusually important to me. Claims need matched baselines, saved artifacts, explicit noise floors, and a visible record of negative results.

The research is trained, not only measured: compact MTP draft heads with their own vocabularies, healed pruned MoEs, and a fine-tuned ColBERTv2 retriever scored by a trained LoRA relevance judge that beat its prompted 27B teacher. Before focusing on efficient inference I worked deeply in datastores, language clients, queues, retrieval, and developer tools. I still maintain Valkey GLIDE, stay active across the valkey-io ecosystem, and help build desktop developer tools; that record—review, releases, support—runs alongside the studies. Model research becomes real software surprisingly quickly.

I also volunteer as a mentor with Yotzim LaShinui, helping a junior developer from a formerly ultra-Orthodox background find their way into software.

Collaboration · replication · careful disagreement

Working on efficient models under real constraints?

aviarchi1994@gmail.com ↗ GitHub / avifenesh ↗ LinkedIn ↗