Howdy! I'm Dwijen Patel, a software engineer working in distributed systems, data infrastructure, and production LLM products. I co-founded Aeqium, and have worked at Day AI, Walmart's Intelligent Retail Lab, Jet, and Palantir. Right now I'm going deep on coding agents and local LLM inference on Apple silicon. The projects and experiments here are that work.
Measurement-first toolkit for pushing local LLM inference on Apple M-series GPUs toward the hardware's actual limits.
The hardware under parts 1 to 3, measured level by level: a four-step latency ladder from lane registers to DRAM, a 128 KiB per-core on-chip pool that registers, threadgroup memory and L1 share, a chip-wide cache of 4 to 8 MiB, and the per-core Neural Accelerators at 15.4 TFLOPS, driven bit-exact from an ordinary Metal kernel. Then the same numbers applied to inference: the 128 FLOP-per-byte ridge that puts prefill on the compute side and decode 32 to 128x below it, how a matrix multiply is split across cores and simdgroups, which MLX kernel runs at each head dimension, and a ranked list of the work the measurements say will pay.
The measurement rig from parts 1 and 2, pointed at MLX itself. Five changes merged between August and the start of September, each shown mechanism first, with the instrument that pinned it down. A block-size heuristic for mixture-of-experts prefill, built on a one-line law the tiles obey, is worth 1.30 to 1.45x at the shape a real model runs; the maintainer reproduced it (2.74 ms to 2.23 ms) before merging. A small-batch matmul tuning was replicated at 1.32 to 1.60x on an M5 Max. Two silent template bugs sat in the build path that compiles kernels from source at run time, which no automated test exercises. And a third-party one-line fix for a kernel that silently returned wrong results above 32,767 rows was verified against the exact state a merge would produce. And a two-line dispatch change that lets head-dimension-256 prefill with an array mask run the fused NAX kernel: 1.44 to 1.56x at kernel level, 5 to 6% of time to first token on Gemma-3-12B.
Run a 35B MoE on a MacBook in ~2 GB of RAM: experts streamed from SSD, near-roofline Metal kernels, and a KV cache that reloads from disk instead of re-prefilling.
Spec-compiled ETL: profile the input, record every decision in an auditable spec, compile to deterministic Python. Bad rows quarantined and error-coded, never silently coerced.
Experiments in evidence-grounded machinery for long-horizon coding agents.
Graded, source-mirrored research corpora for agents. Evidence strong enough to rest decisions on.
How butterflies, bureaucracies, and coding agents fail in the same way