Abdul Muneer

I build things and write up what they teach me.

Part 2 of the series. Fornax uses MAX as its per-node execution substrate, so this part examines what MAX bundles, how mature its operators are across NVIDIA, AMD, Apple, and CPU, and how far down the stack it reaches before vendor toolchains take over. It then defines the remaining Fornax work: the cross-vendor stage boundary, planner, and failure model needed to turn mismatched machines into one engine.

Modular's MAX platform stack. MAX and this graphic are trademarks and assets of Modular Inc.; shown here for identification.

A five-minute primer

If you have already served a model with MAX, skip to What MAX bundles. This gets a newcomer oriented.

MAX is Modular’s AI inference platform. You hand it a supported model, typically a Hugging Face checkpoint such as Qwen/Qwen3-8B, and it compiles the model’s computation graph to target-specific kernels. The packaged serving path supports CPU, NVIDIA, and AMD configurations. Apple support is build-, model-, and operator-specific, so the Apple experiments in this series use a pinned nightly and local source patches. The common entry point is a server:

pip install modular
max serve --model Qwen/Qwen3-8B
# an OpenAI-compatible server is now on localhost:8000
curl localhost:8000/v1/chat/completions -d '{
  "model": "Qwen/Qwen3-8B",
  "messages": [{"role": "user", "content": "hello"}]
}'

Because the endpoint implements a documented subset of the OpenAI API, many existing clients can use it after changing their base URL and, where required, their authentication configuration. Under that server is a Python library, max.pipelines, that you can drive directly in-process, and under that is the graph compiler and the Mojo kernels that do the actual arithmetic. The architectural split is a Python orchestration surface over Mojo kernels, and Fornax lives on both sides of that line.

Modular was started by Chris Lattner, who created LLVM and initiated MLIR. That compiler background shows up in how deeply MAX integrates graph lowering, kernel authoring, and target-specific code generation.

In June 2026, Qualcomm agreed to acquire Modular in a transaction expected to close in the second half of 2026. If the transaction closes, Fornax’s pinned execution substrate will be owned by an accelerator vendor. Capability claims remain tied to local probes on pinned builds.

What MAX bundles

MAX bundles several layers that other stacks ship separately. When you say “MAX,” you are naming at least four things at once.

  1. A graph compiler, MLIR-based, that lowers a model to optimized kernels across hardware.
  2. An LLM serving and pipeline engine: paged KV cache, prefix caching, continuous batching, speculative decoding, structured output, and the max.pipelines API.
  3. A kernel-authoring language, Mojo, for writing custom ops with @compiler.register and dropping them into the graph.
  4. A portability promise: the same stack on NVIDIA and AMD and, increasingly, Apple, explicitly marketed as “no CUDA required.”

Together, these layers let MAX target several GPU vendors from a common compiler and serving stack. Part 1 laid out the work required below that source-level portability. Whether a model works remains a per-operator, per-platform measurement. The next section records the support observed for the builds used here.

Operator maturity, platform by platform

While working on Fornax I built a support chart for MAX operators across platforms, because Modular does not publish a single official per-operator hardware matrix. The chart combines the public max.graph.ops list, the AI-kernel package docs, the vendor-specific kernel subtrees, and local probe results. Support is uneven, which is why Fornax has to be careful about where it places work.

The platform-level baseline for this July 2026 assessment is:

Platform Baseline What that means for a spanned model
NVIDIA GPU Supported Documented and containerized. Attention has SM90 (Hopper) and SM100 (Blackwell) subtrees; SM100 explicitly covers both MHA and MLA prefill and decode.
AMD GPU Supported Documented and containerized. Attention has RDNA and gfx950 subtrees; gfx950 (MI355X) explicitly covers MHA and MLA prefill and decode.
Apple Silicon Build-specific / local evidence The MAX 26.4 changelog describes support for many common models on M3 and later, followed by nightly support for M1 and M2. The current packages page still says large GenAI inference is unavailable on Apple silicon. The Fornax result uses a pinned nightly and local MLA patches.
Intel/AMD CPU Supported CPU serving works; the Linux baseline is x86-64-v3. Some kernels ship explicit CPU reference paths.

Attention contains some of the widest platform gaps and can dominate transformer execution at long context or attention-heavy shapes. NVIDIA has separate Hopper and Blackwell attention trees. AMD has RDNA and gfx950 trees. Apple has a Metal flash-attention prefill kernel and a naive flash-attention decode, and the pinned build I tested had no Multi-head Latent Attention (MLA) path for the DeepSeek-V2-Lite shape. Multi-GPU collectives (allgather, allreduce, reducescatter) exist for NVIDIA and AMD and do not exist for Apple, and they are intra-vendor in any case, so they are not a substitute for moving tensors between an NVIDIA box and a Mac.

A documented operator enters the planner as an unverified candidate. Its presence in max.graph.ops establishes an API surface. A local probe on the pinned build closes the gate for a specific dtype, shape, and device. Release history shows why: MAX 26.4 described Apple Silicon GPU serving for many common models on M3 and later, followed by nightly support for M1 and M2. It also made split-K decode attention the default Apple token-generation path for supported paged-KV MHA and GQA shapes. Those statements still do not establish MLA for the model and build used here.

Fornax uses the following evidence order. A local run on the pinned build and target hardware comes first. Package requirements and the supported-model catalog for that build come second. Release notes and changelogs come third. Blog posts and launch announcements supply leads for testing only. Any disagreement leaves the target unproven until a local run passes. This prevents a changing documentation page from silently becoming a Fornax capability claim, and Part 4 shows a case where the changelog and the local run disagreed outright.

MAX’s boundary with vendor toolchains

The emitted artifacts and compiler diagnostics show where MAX’s work meets vendor toolchains. Follow one operator from the Python graph toward the code the GPU runs. The middle of this path contains internal dialects and transformations that Modular does not document as a stable public architecture. The diagram records the observed model; Modular does not publish it as an API contract.

You express the model in max.graph.ops, the high-level Python operators (matmul, softmax, rms_norm, gather, and the rest). That graph is lowered into MAX’s own MLIR dialects, the internal graph IR and the kernel-generation layer the logs call KGEN. The kernels themselves are written in Mojo, a compiled language built on MLIR. From there the observed path is:

max.graph.ops  (Python)
   |  lower to MAX graph IR (MLIR dialects, KGEN)
Mojo kernels
   |  compile through MLIR
target-specific lowerings
   +--> NVIDIA:  PTX --> SASS                 (NVIDIA compiler/driver)
   +--> AMD:     AMDGCN/AMDGPU code           (ROCm/LLVM toolchain)
   +--> Apple:   Metal-targeted LLVM IR/AIR   (Metal toolchain, metallib)

The target fan-out is:

Mojo’s documented compiler outputs include PTX for NVIDIA, AMDGCN output for AMD, and LLVM IR sidecars for Metal targets. MAX adds the graph IR, fusion, scheduling, runtime integration, and Mojo kernel library above and across those target paths. Portability comes from the common language and compiler infrastructure together with target-specific kernels, runtime work, numerical validation, and vendor toolchains. MLIR and LLVM make the layered design possible while hardware-specific kernels, runtime integration, and validation remain necessary.

This also explains the failure signatures Fornax runs into later. They point deep into lowering: “Metal Compiler failed to compile metallib,” or “graph compiler could not elaborate the generated KGEN,” or a Metal-invalid instruction like a double-precision floor that NVIDIA tolerates and Apple’s GPU does not. Those are messages from the bottom of the stack. They identify target-specific lowering and instruction support as the failing layer. Part 4 and Part 5 live down there.

What Fornax adds beyond current MAX

MAX supports selected homogeneous multi-GPU model configurations. The MAX 26.3 changelog also records NIXL-backed distributed KV transfer and heterogeneous prefill/decode layouts. The release notes describe mechanisms and selected layouts. Exact availability still depends on the release, model configuration, hardware, topology, and deployment surface. These capabilities narrow the space a separate distributed engine should claim.

The remaining Fornax target is more specific: one model split by complete layer ranges across machines whose accelerators and runtimes may come from different vendors, including an NVIDIA or AMD Linux node and an Apple Silicon Mac. The public MAX documentation does not establish that NVIDIA-to-Apple stage pipeline as a supported deployment. Fornax’s case rests on the following narrower differences.

Cross-vendor stage boundaries require their own protocol. NCCL and RCCL can operate across nodes, and MAX can use distributed homogeneous GPU layouts. The activation contract between an NVIDIA process and a Metal process remains outside those collectives. Fornax defines that boundary as a versioned Stage ABI with explicit tensor layout, ownership, acknowledgement, cancellation, and failure behavior. The baseline transports activations only. KV remains stage-local, and remote expert batches remain deferred.

Cross-vendor tensor parallelism exceeds the target network budget. Common tensor-parallel transformer layouts require one or more collectives per layer. That pattern is designed for a fast, homogeneous fabric and is a poor fit for a commodity LAN between different vendors. Between unlike nodes Fornax defaults to pipeline parallelism by contiguous layer group, where one activation frame crosses each stage boundary. Matching GPUs can still form a homogeneous island inside a stage and use the parallelism MAX supports there.

The planner is built around mixed-node evidence. Fornax inventories each node, records measured links and build-specific operator probes, rejects infeasible placements, and estimates complete stage routes. Its planner remains advisory until physical calibration closes the gap between predicted and measured service time.

Operator gaps become placement constraints. A Mac becomes a legal stage host after every required operation, dtype, shape, and runtime contract passes on the pinned build. Local Apple MLA patches establish one DeepSeek smoke. Packaged support and a planner-authorized role remain open.

Fornax is a narrow extension of MAX. MAX provides the graph compiler, kernels, model runtime, KV primitives, serving machinery, and supported distributed features. The Fornax target adds the NVIDIA/AMD-to-Apple stage contract, cross-node activation transport, mixed-fleet planning, global request ownership, and distributed failure semantics. If MAX grows a supported cross-vendor stage pipeline with equivalent contracts, Fornax should reuse it and reduce its own surface. The next part describes the current Fornax contracts.

Previous: What Accelerator Support Requires. Next: Fornax Architecture. Series index.

Sources: Modular MAX documentation; max serve CLI; MAX REST API reference; MAX packages and hardware support; MAX distributed-inference FAQ; MAX 26.3 changelog; MAX 26.4 changelog; Qualcomm to acquire Modular.