Apple Silicon · MLX · Metal
Know the model.
Know the machine.
Local inference is a systems problem: weights, kernels, memory, and storage all have a say. My work connects correctness fixes upstream with experiments across that stack.
Correctness at the kernel and conversion layers.
My merged MLX and MLX-LM contributions address row-count overflow, scale and bias indexing, and a repeated normalization shift in converted Qwen checkpoints. These are specific correctness fixes; their review threads document the supported paths and upstream decisions.
Native multi-token prediction.
MTPLX work explores the runtime integration required for native MTP. The contribution thread for PR #142 was closed rather than merged; the maintainer identified the work as included in the curated 2.1.0 release. The AI hub links to the original thread and release context.
When the weights don’t fit in memory.
iliria explores SSD-backed expert streaming through a C/Metal inference engine. Its operating assumptions differ from a standard MLX load-and-run workflow. The engine guide and model-specific documentation explain those boundaries.
Resources with enough context to use.
The public collection brings model releases together with data, demos, source repositories, and documentation. Read the model card and hardware notes before selecting a runtime or treating a benchmark as a fit for your machine.
Browse models, data, demos, and docs ↗