A usable accelerator product needs a software stack that executes real workloads correctly, efficiently, and predictably. The required software spans model APIs, framework semantics, compilers, kernels, runtimes, drivers, firmware, memory, and interconnects, and must operate within the chip’s power and thermal limits. A failure anywhere in that path can erase hardware gains.
The series starts with the behavior expected from model.to(device) and
torch.compile(model), defines the hardware execution model, and traces the
software required to expose that hardware as a supported framework target. The
scope is cross-functional:
framework engineers, compiler and kernel teams, driver and firmware developers,
collective-library and networking engineers, media-engine teams, performance and
reliability engineers, and the architects responsible for compute, memory,
packaging, and process technology all shape the result.
The final part compares where PyTorch, JAX, and MAX place their public hardware boundaries, then aligns the CUDA, ROCm, and Intel XPU stacks at the driver, runtime, compiler, library, and framework seams.
Scope of accelerator support
“Supports PyTorch” claims compatibility across tensor semantics, operator coverage, compilation, numerics, distributed execution, packaging, profiling, and failure handling. Tensor allocation is an early bring-up gate. Production training and serving also require existing models to continue working across framework upgrades, with diagnosable failures.
Interface semantics include tensor strides, storage, offset, dtype, device, aliasing, and mutation; stream ordering and visibility; and collective ordering, progress, topology, and failure behavior.
High-level graphs retain model meaning until target information is available. Later representations introduce layout, tiling, memory spaces, and parallelism.
Registry presence establishes availability. Correctness across shapes, dtypes, layouts, gradients, and dynamic cases requires tests. Workload and benchmark results also need numerical comparison, fallback accounting, compile and transfer costs, and a pinned software stack.
Articles
- Part 1 - The Support Contract defines the levels hidden inside “supports PyTorch” and the evidence required at each one.
- Part 2 - The Hardware-Software Contract covers compute, memory, fabrics, fixed-function engines, packaging, lithography, power, and the information software needs from the chip.
- Part 3 - Driver, Runtime, and Memory follows a process from device discovery through command submission, allocation, streams, events, faults, resets, virtualization, and profiling hooks.
- Part 4 - Tensors, Dispatch, and Operators explains storage and view semantics, dispatch keys, schemas, native kernels, decompositions, metadata implementations, and fallbacks.
- Part 5 - Kernel Libraries covers tiling, data movement, matrix instructions, attention, sparsity, low precision, autotuning, and the irregular and small-shape cases that determine workload coverage.
- Part 6 - The Graph Compiler traces Python through capture, guards, decompositions, AOTAutograd, scheduling, progressive lowering, target code generation, caching, and diagnostics.
- Part 7 - Autograd, Precision, and Numerical Validation covers backward support, AMP, accumulation, RNG, determinism, reference comparison, gradient checks, and training convergence.
- Part 8 - Scale-Up and Scale-Out connects physical topology to collective libraries, ProcessGroup, DeviceMesh, DTensor, parallel strategies, checkpointing, overlap, and failure recovery.
- Part 9 - Training, Inference, and Media Engines shows how the common platform branches into different products, including serving runtimes and fixed-function video pipelines.
- Part 10 - Qualification and Release assembles the qualification gates, ecosystem matrix, compatibility policy, packaging, CI, observability, regression systems, release ownership, and maintenance processes.
- Part 11 - Framework Boundaries and Vendor Stacks compares framework integration boundaries and traces their lower-stack implementations through CUDA, ROCm, and Intel XPU.
Cross-team dependencies
The boundaries are organizational as well as technical. A matrix instruction affects compiler legalization, kernel tile shapes, numerical qualification, and performance counters. A new HBM configuration changes the allocator, the serving capacity model, thermal behavior, and collective overlap. A page-fault mechanism needs hardware support, firmware policy, a kernel driver, runtime APIs, framework behavior, and tests that distinguish migration from a hang.
Cross-team work begins before tape-out. Hardware and software architects need an explicit contract while the machine is still changing. Kernel and compiler teams need simulators, emulators, and performance models before production silicon. Framework engineers need to identify semantics whose cost determines platform viability. Reliability engineers need error-injection coverage before release.
Evidence policy
The series separates four kinds of statement:
- A specification or API describes an intended contract.
- Source inspection shows how one implementation represents that contract.
- A focused test establishes behavior for a named build, device, dtype, and shape.
- A workload result establishes end-to-end behavior for a named configuration.
Specifications and source inspection guide implementation. Release claims require focused tests and workload results. Vendor figures are identified as vendor figures, and fast-moving product details stay out of the central argument when a stable interface provides sufficient evidence.
Shorter treatment in the Fornax series
Part 1 of the Fornax series is the compressed version of this subject. It explains enough of the accelerator stack to motivate MAX and heterogeneous inference, then returns to Fornax. This series expands that material into the contracts and qualification work required for a production-supported framework target.
Start with Part 1 - The Support Contract. Back to the blog home.