Build ROCm Kernels: 5 Easy Steps and Mistakes to Avoid

To build ROCm kernels that other people can load with one line of Python, Hugging Face offers a kernel builder based on Nix and a kernels library that loads compiled kernels from the Hub. This article is based on the official Hugging Face guide Easily Build and Share ROCm Kernels with Hugging Face (published 17 November 2025, read on 2 November 2026), not on a hands-on test: building and running these kernels needs an AMD Instinct GPU, which I do not have. The old version of this post said the builder came with pip install huggingface_hub; the guide uses Nix and the kernels tooling instead, so that part was wrong and is gone.

Contents

• What you need to build ROCm kernels this way
• Step 1: lay out the project to build ROCm kernels
• Step 2: describe the build in build.toml
• Step 3: build ROCm kernels with Nix
• Step 4: build every variant and upload
• Step 5: load the kernel from the Hub
• Mistakes to avoid when you build ROCm kernels
• Is this the right tool for you?
• FAQ

What you need to build ROCm kernels this way

The guide assumes you already have a working HIP kernel. To build ROCm kernels with this tool you need:

• A HIP kernel and a target architecture. The guide's example is the RadeonFlow FP8 GEMM kernel for the AMD Instinct MI300X, compiled for gfx942, which the guide says targets the MI300 series.
• Nix. On Linux the guide recommends the official Nix installer; on macOS the Determinate Nix installer and Xcode 16.x.
• A Hugging Face account if you want to share the result on the Hub.

The example kernel is narrow by design. It takes FP8 inputs in the e4m3fnuz format with per-block scaling, writes a bf16 output, and is precompiled for specific matrix shapes. The guide warns that other shapes or memory layouts need changes to the kernel launcher.

The five steps of the official guide on one page.

Step 1: lay out the project to build ROCm kernels

The kernel builder expects a fixed structure: build.toml as the manifest, a folder with the kernel sources, flake.nix for a reproducible environment, and torch-ext/ with the PyTorch binding and a Python package. The guide renames the original HIP .cpp files: .h for headers with declarations, inline functions or templates, and .hip for files that must be compiled, such as kernel launchers. The builder uses these extensions to decide how to compile each file.

Step 2: describe the build in build.toml

[general]
name = "gemm"
universal = false

[torch]
src = [
  "torch-ext/torch_binding.cpp",
  "torch-ext/torch_binding.h",
]

[kernel.gemm]
backend = "rocm"
rocm-archs = [
  "gfx942",
]
depends = ["torch"]
include = ["include"]

The guide's file lists every source file in src under [kernel.gemm]; I shortened it here. backend, rocm-archs and depends are required. universal = true is for pure Python kernels such as Triton kernels, which skip the compiled sections. The flake.nix is short and points at the builder:

{
  description = "Flake for GEMM kernel";
  inputs = {
    kernel-builder.url = "github:huggingface/kernels";
  };
  outputs = { self, kernel-builder, }:
    kernel-builder.lib.genFlakeOutputs {
      inherit self;
      path = ./.;
    };
}

The C++ binding registers a real PyTorch operator with TORCH_LIBRARY_EXPAND, so the kernel appears under torch.ops. Note one detail from the guide's binding: the checks use a.device().is_cuda() and the op is registered for torch::kCUDA. The guide's usage example also creates its tensors with torch.device("cuda") on the AMD GPU, so do not be surprised by the CUDA names in ROCm code.

Step 3: build ROCm kernels with Nix

nix flake update
cachix use huggingface
nix build . -L

nix flake update writes flake.lock, and the guide says to commit both files so builds stay reproducible. The Hugging Face binary cache matters, because the builder depends on many packages, including every supported PyTorch version. Without cachix installed, the guide gives nix run nixpkgs#cachix -- use huggingface. For development, nix develop opens a shell with build2cmake, and a named shell such as .#devShells.torch27-cxx11-rocm63-x86_64-linux gives you Torch 2.7 with ROCm 6.3.

Step 4: build every variant and upload

When you build ROCm kernels for other people, one version is not enough. Before sharing, run build2cmake clean build.toml to remove development files, then nix build . -L outside the dev shell. The guide warns this takes a while because it builds for all supported PyTorch and ROCm versions. The results go into build/:

mkdir -p build
rsync -av --delete --chmod=Du+w,Fu+w result/ build/
kernels upload <path_to_kernel> --repo_id hub-username/img2gray

The repo name in the last line is the guide's own example. A plain git push to a Hub repo also works; the guide uses git xet track "*.so" for the binaries.

Step 5: load the kernel from the Hub

This is where the work to build ROCm kernels pays off for your users: they never run a compiler.

import torch
from kernels import get_kernel

gemm = get_kernel("kernels-community/gemm")

You do not install the kernel. get_kernel loads it from the Hub and registers the operator. The guide's call then passes FP8 tensors made with torch.float8_e4m3fnuz, fp32 scale tensors and a bf16 output, with the supported shape M, N, K = 1024, 1536, 7168.

Mistakes to avoid when you build ROCm kernels

• Wrong file extensions. Keep headers as .h and compiled HIP code as .hip, or the builder will not treat them correctly.
• Wrong architecture. A kernel built only for gfx942 is for the MI300 series. Add each target you need to rocm-archs.
• Unsupported shapes. The example launcher prints Error: Unsupported shape M=%d, K=%d, N=%d and calls abort() for any other shape.
• Skipping the lock file. Without flake.lock in git, another build may use different dependencies.
• Uploading the dev mess. Run build2cmake clean first, and add only the needed files.

Is this the right tool for you?

Use it when you want to build ROCm kernels once and let users load them for many PyTorch and ROCm versions without compiling. If you only need a kernel inside one project, a normal PyTorch extension may be enough. For the CUDA side, the guide links its CUDA-focused original. For running models on a Linux box without any GPU, the Ollama on Linux post is tested on CPU only. A Metal (Apple Silicon) path also exists, according to the guide author's reply in the comments.

FAQ

Do I need an AMD GPU to build ROCm kernels? To run and test them, yes; the example targets the MI300X. This article did not run any of the commands.

Where does the output go? nix build writes to result, which the guide copies into build/ for the kernels library.

Can I share a Triton kernel? Yes, as a universal kernel with universal = true, according to the build.toml section of the guide.