A blog series about Fornax, a project building a Mojo/MAX-native distributed inference engine for one large sparse Mixture-of-Experts model across a fleet of commodity machines: consumer NVIDIA GPUs, Apple Silicon Macs, qualified AMD devices, and CPU workers. The product target is a checkpoint whose complete runtime footprint exceeds the usable memory of every selected node.

The series works from the problem down to the implementation. The early parts state the objective and the industry context. The middle parts cover MAX, the compiler and runtime Fornax is built on, from its Python surface down to the point where its work lowers into PTX, AMDGPU code, and Metal shaders. The later parts cover the Fornax architecture, the model bring-up evidence collected through Transformers and MAX, and the local kernel work required to make one bounded DeepSeek path execute on Apple Silicon.

New to MoE, MAX, or the accelerator software stack? Part 1 provides a compressed view of accelerator support and links to the complete eleven-part series. Part 2 opens with a short MAX primer. Readers familiar with those layers can start with Part 3.

Evidence policy. I keep three kinds of result apart throughout the series. Contract tests and simulated runs establish process, socket, and failure behavior and make no hardware claim. Cost-model output is a prediction. A performance or platform claim is a measurement only when it names the model, hardware, MAX build, dtype, shape, and command that produced it.

The series

  • Part 0 - Objective and Constraints - what Fornax is trying to do, why sparse MoE makes a heterogeneous fleet the right shape, and the constraint that governs everything (when a model spans nodes, every token crosses the network).
  • Part 1 - What Accelerator Support Requires - what a new accelerator vendor has to build under the PyTorch API and above the silicon to reach parity with the CUDA stack: operators, lowering, graph compilation, distribution. How that job changes for a training chip versus an inference chip, and what it means for MAX to take on all of it without owning any silicon.
  • Part 2 - MAX Platform Assessment - what MAX is, how mature its operators are across NVIDIA, AMD, Apple, and CPU, and how far down the stack it reaches before its work merges into LLVM and each vendor’s toolchain. Then the narrower contribution Fornax targets beyond MAX’s current distributed features: a cross-vendor complete-layer pipeline and its contracts.
  • Part 3 - Fornax Architecture - the architecture: the MoE seam it cuts, the pipeline-parallel spine, the Stage ABI and versioned activation frames that let unlike machines exchange tensors safely, the planner and evidence model, and the executable contracts the physical Mojo/MAX backends must satisfy.
  • Part 4 - Model Bring-Up - a Transformers/PyTorch Qwen3-Omni smoke across four H100s, MAX model bring-up on an M3 Max, the Qwen and Kimi failures there, and the local kernel work that got DeepSeek-V2-Lite to emit tokens on the Mac.
  • Part 5 - Porting an M5-Targeted Apple MLA Kernel to M3 - a detailed account of why a 16x16, M5-targeted specialization could not run on M3, why M3 required a separate 8x8 implementation, and which conclusions come from M3 execution, source inspection, Apple documentation, or third-party microbenchmarks. This part contains no M5 hardware measurement.

Fornax is in active development. The public repository is a pre-alpha executable specification and Engine v0 prototype. It contains the planner, the historical FNX1 mechanism, a candidate FNX2 ragged reference engine, independent loopback workers, validators, and hardware bring-up tools. It does not yet contain a physical cross-vendor MaxStageBackend or a frontier-capacity run. Those remain the next physical validation gates. The current implementation and commands are recorded in the Fornax repository.

Back to the blog home.