Failures and successes, shared with equal preference
The hardware under parts 1 to 3, measured level by level: a four-step latency ladder from lane registers to DRAM, a 128 KiB per-core on-chip pool that registers, threadgroup memory and L1 share, a chip-wide cache of 4 to 8 MiB, and the per-core Neural Accelerators at 15.4 TFLOPS, driven bit-exact from an ordinary Metal kernel. Then the same numbers applied to inference: the 128 FLOP-per-byte ridge that puts prefill on the compute side and decode 32 to 128x below it, how a matrix multiply is split across cores and simdgroups, which MLX kernel runs at each head dimension, and a ranked list of the work the measurements say will pay.
The measurement rig from parts 1 and 2, pointed at MLX itself. Five changes merged between August and the start of September, each shown mechanism first, with the instrument that pinned it down. A block-size heuristic for mixture-of-experts prefill, built on a one-line law the tiles obey, is worth 1.30 to 1.45x at the shape a real model runs; the maintainer reproduced it (2.74 ms to 2.23 ms) before merging. A small-batch matmul tuning was replicated at 1.32 to 1.60x on an M5 Max. Two silent template bugs sat in the build path that compiles kernels from source at run time, which no automated test exercises. And a third-party one-line fix for a kernel that silently returned wrong results above 32,767 rows was verified against the exact state a merge would produce. And a two-line dispatch change that lets head-dimension-256 prefill with an array mask run the fused NAX kernel: 1.44 to 1.56x at kernel level, 5 to 6% of time to first token on Gemma-3-12B.
An LLM-driven evolutionary search, run three times against the map from part 1. The first run returned 0.96x, because its fitness score sat on a noise floor. The second produced a real 1.23x kernel that was worth only 1.04x end-to-end, because it targeted the wrong operation. The third, with the target chosen by measurement and the score gated against noise, produced a fused attention kernel at 80% of the hardware ceiling where Apple's stack ran at 34%. Every campaign is mined candidate by candidate: what each jump changed, what analysis provided and what the search invented, and the verification steps between a winning candidate and a shipped number.
Apple documents almost nothing about how its GPUs work: no instruction set, no latencies, no occupancy rules. Rather than reverse-engineer the chip, this project measures it, then scores every operation in Apple's own inference stack against the ceiling the chip sustains. The result: MLX's matrix-multiply kernels already run at 85 to 102% of their ceiling, its attention kernels range from 33 to 87%, and the largest gap sits in prompt processing rather than in generation. The page ends with the eight ways a plausible measurement turned out to describe something the software never does.
An LLM-driven evolutionary loop rewrites Postgres's row-count combining logic, judged only on data it never optimized against. Run 1 fell into the reward-hacking trap the harness was built to catch; run 2 hardened the objective and re-ran the same search. Built at the AGI House Auto Research Summit with Athreya Chandramouli.