How to Fix 'fatal error: cuda.h: No such file or directory' in Triton and vLLM on Linux
PyTorch & CUDA(Updated )

How to Fix 'fatal error: cuda.h: No such file or directory' in Triton and vLLM on Linux

fatal error: cuda.h: No such file or directory means a particular C/C++ compilation step could not find the CUDA driver development header. It does not prove that the NVIDIA driver is absent or that every PyTorch/Triton installation needs a complete system CUDA Toolkit. A missing nvcc executable, a missing cuda_runtime.h, a PyTorch extension compiled with the wrong ABI, and a JIT cache issue are related but different failures.

The older edition of this article installed several CUDA packages unconditionally, constructed CUDA_HOME by assuming a particular Python site-packages layout, appended exports to ~/.bashrc, and deleted the entire Triton cache before verifying the error. Those changes may be unnecessary or damage a working environment. Use read-only diagnosis first, then make one scoped correction only if a development header is actually missing.

This guide is for a Linux machine with a compatible CUDA-enabled NVIDIA device when running a real GPU test. Our published helper is intentionally CPU-only and cannot prove GPU kernel compilation success.

1. Find out what is actually missing

Collect the active Python, driver and toolkit facts without installing or changing anything:

python3 --version
python3 -m pip show torch triton
nvidia-smi
command -v nvcc || true
printf 'CUDA_HOME=%s\n' "${CUDA_HOME:-<unset>}"
printf 'CPATH=%s\n' "${CPATH:-<unset>}"

The presence of nvidia-smi shows the driver management interface is available, not that CUDA development headers exist. The presence of torch or triton as Python modules proves imports only, not a successful JIT build.

Test common CUDA header paths using a read-only tool

Download and inspect the published CUDA header path checker Python source, also available as a local download. This site-authored Python code probes common known CUDA header paths and nvcc visibility; it does not compile a GPU kernel, install packages, alter shell profiles or delete cache files. The linked public code is reviewed byte-for-byte against the repository; it does not replace vendor evidence.

curl -fsS https://agentic-pulse.dev/tools/cuda_headers_check.py -o cuda_headers_check.py
python3 cuda_headers_check.py --self-test
python3 cuda_headers_check.py

The public checker source contains four synthetic classification cases exposed through --self-test; readers can reproduce them without CUDA. This is an implementation-level test, not a GPU compilation result. A negative header search does not prove cuda.h is absent from every mount, sysroot or architecture-specific path.

For the actual compiler failure, also note the failing build command and the include paths it passed. The compiler’s -I list matters more than a generic claim that one package “must be missing.”

2. Match the error to the right prerequisite

Observed error Evidence to collect What not to assume
fatal error: cuda.h Exact failing C/C++ compile command, header paths and CUDA Toolkit layout Installing a PyTorch wheel automatically provides developer headers
cuda_runtime.h missing Runtime development include path and compiler options cuda.h and cuda_runtime.h are the same file
CUDA_HOME environment variable is not set CUDA_HOME, nvcc path and the extension build’s expected toolkit root Every Triton workload needs nvcc installed globally
undefined symbol when importing an extension PyTorch, CUDA, C++ ABI and extension version A header installation can fix binary ABI compatibility
JIT kernel fails after an otherwise successful compile Complete traceback, cache path, driver and GPU compute capability It was definitely a corrupt cache
torch.cuda.is_available() == False Driver, runtime, actual GPU access Changing CPATH can repair a driver/GPU problem

The versioned PyTorch C++/CUDA extension reference documents build-time CUDA discovery and describes CUDA_HOME as an option when automatic toolkit-location heuristics fail. That behavior is about PyTorch extensions, not proof that every Triton JIT workflow requires a globally installed nvcc.

3. Correct the verified missing header or path

A. Existing system CUDA Toolkit but incorrect include path

Before changing anything, locate the actual header and compiler. These examples examine common locations without attempting to repair them:

for root in "${CUDA_HOME:-}" /usr/local/cuda /usr/local/cuda-12 /usr/local/cuda-13; do
  if [[ -z "$root" ]]; then continue; fi
  for relative in include/cuda.h targets/x86_64-linux/include/cuda.h targets/aarch64-linux/include/cuda.h; do
    if [[ -f "$root/$relative" ]]; then
      printf 'Found %s\n' "$root/$relative"
    fi
  done
done

If your existing toolkit actually resides at /usr/local/cuda and the header is found there, a temporary command-local environment may help an extension build discover it:

# Only after confirming the directory and its headers on your own host.
CUDA_HOME=/usr/local/cuda \
CPATH="/usr/local/cuda/include${CPATH:+:$CPATH}" \
python3 your_existing_authorized_build_command.py

Replace the final filename with the command that originally failed. Do not blindly add it to ~/.bashrc, /etc/profile.d or a global systemd service. For recent Toolkit layouts, the header can be under a targets/<architecture>/include subtree, so use the directory that actually exists.

B. Development headers are genuinely missing

Use the official NVIDIA CUDA installation guide to determine which development headers and compiler packages are supported on the specific OS and CUDA Toolkit version. A Python package name alone cannot prove this compiler has the correct cuda.h; inspect the installed header location and failing include search. Do not assume installing a generic runtime wheel repairs all missing development headers.

Do not upgrade the host display driver, change the system-wide CUDA symlink, or mix unrelated CUDA major versions just because a tutorial has a hard-coded 12.6 package list. Those changes require a maintenance plan and compatibility checks.

If containerized, verify whether the actual failing compiler runs inside the image and whether the necessary header and include path are present there, not merely on the host. Select an operator-approved development image with headers when the workload needs to compile; do not assume that selecting a container named runtime or devel resolves unrelated PyTorch or GPU driver incompatibilities. Keep image versions pinned after controlled validation.

4. Treat cache changes as a separately validated hypothesis

The earlier guide deleted all of ~/.triton/cache. Do not do that for a missing cuda.h error. A compiler which cannot locate a header can fail in a fresh cache too; first establish the actual source and include search order. If later evidence points to a stale generated artifact, consult the installed Triton version’s supported cache configuration, record the exact path and test in an isolated development environment rather than deleting an unknown global cache. This article does not claim that a particular TRITON_CACHE_DIR override isolates every cache used by every PyTorch/vLLM/Triton release.

The official Triton vector-add tutorial shows a concrete Triton JIT kernel and reference comparison. It can be used as a version-appropriate verification task after repairing a real compile environment, but the tutorial itself does not prove the cause of a missing NVIDIA header on this host. Do not treat an import or CPU-only header scan as a successful GPU kernel launch.

5. A real verification sequence on a supported GPU

After making a single verified correction, validate each level separately:

  1. python -c "import torch, triton" succeeds. This only tests imports.
  2. The needed cuda.h and other headers can be opened along the actual include path of the failed compiler.
  3. Use the official Triton vector-addition tutorial or your minimal affected kernel to trigger a JIT compile on an approved, compatible GPU.
  4. Compare the kernel output against a reference implementation using torch.allclose and record the result, device, driver, CUDA version and package versions.
  5. Only after that, retry the original vLLM/PyTorch workload in a suitable maintenance window.

The official Triton tutorial includes both a minimal JIT kernel and a reference comparison. A passing vector-add test is stronger evidence than an import, but it still does not guarantee every FlashAttention, custom extension or vLLM model will compile.

Our actual testing for this updated article was limited to Python syntax and four synthetic path-status classifications for the linked helper. We did not use an NVIDIA GPU, compile a real Triton kernel, install CUDA packages or mutate an existing cache. The tool is provided to let readers reproduce the relevant prerequisite check locally.

6. When the header exists but the failure persists

Do not repeat installations. Inspect the exact compiler invocation and whether it is running in a Python virtual environment, container, isolated wheel-build environment or different user namespace. Compare sys.prefix, CUDA_HOME, the failing build’s compiler, and the actual include paths. If the failure is an extension ABI mismatch, use PyTorch’s CUDA extension notes rather than changing only environment variables.

For vLLM and PyTorch deployments, also distinguish insufficient GPU memory from missing headers. The vLLM memory troubleshooting guide covers a different failure family.

Bottom line: an exact error traceback plus the location of cuda.h tells you which environment to fix. Repeated broad installations and cache deletion before diagnosis make that task harder.

Primary sources

Corrected and reviewed 2026-11-05; original article URL and first publication date preserved. No actual GPU runtime benchmark claimed.

BC

Written by B. Cho

Systems & AI Infrastructure Engineer

Specializing in GPU inference engines (vLLM, SGLang, Ollama), CUDA memory allocation, and Model Context Protocol (MCP) infrastructure on Linux workstations.