
How to Fix FlashAttention-2 Installation and CUDA Kernel Mismatch Errors in PyTorch on Linux
You ran pip install flash-attn --no-build-isolation. The installer began compiling CUDA objects, and cc1plus was killed. Or installation succeeded, but importing flash_attn_2_cuda now fails with undefined symbol.
Those are different failures. One happens during compilation; the other often means an extension no longer matches the PyTorch environment. Do not solve both by reinstalling the whole CUDA toolkit.
The checks below follow the FlashAttention upstream installation contract, wheel build and selection logic, and PyTorch extension guidance, rechecked against the cited current source pages on November 12, 2026. These are documented diagnostic steps, not a claim of having reproduced each error on physical hardware.
1. Read the version tuple before changing packages
Run this in the same virtual environment where FlashAttention fails:
python3 - <<'PY'
import sys
import torch
print("Python:", sys.version.split()[0])
print("PyTorch:", torch.__version__)
print("PyTorch CUDA:", torch.version.cuda)
print("CXX11 ABI:", getattr(torch._C, "_GLIBCXX_USE_CXX11_ABI", "unavailable"))
print("CUDA available:", torch.cuda.is_available())
if torch.cuda.is_available():
print("GPU:", torch.cuda.get_device_name(0))
print("Compute capability:", torch.cuda.get_device_capability(0))
PY
command -v nvcc && nvcc --version
python3 -m pip show flash-attn
Record Python, PyTorch, the CUDA runtime associated with PyTorch, the host nvcc version if a source build is planned, ABI, and GPU compute capability.
These numbers answer different questions. torch.version.cuda identifies the runtime compatibility of the installed PyTorch build; nvcc --version identifies a compiler on your path. The CUDA version reported by PyTorch and the compiler version on the host serve different purposes. When compiling a CUDA extension, the upstream PyTorch extension implementation contains an explicit CUDA-version compatibility check; whether it errors or warns depends on the version combination and the installed PyTorch release.
Changing a global /usr/local/cuda symlink without first identifying the failing compiler is a system-wide risk. Prefer examining the exact build log.
2. A normal pip install does not guarantee a wheel
FlashAttention-2’s official installer command is:
# Run on a supported GPU host with a matching PyTorch build.
python3 -m pip install flash-attn --no-build-isolation
This may retrieve a compatible prebuilt wheel or attempt local compilation. The upstream setup.py CachedWheelsCommand attempts to obtain a matching cached release wheel and can fall back to source compilation. This describes the reviewed upstream source, not a guarantee that the same path exists unchanged in every release. If pip invokes Ninja, it is not proof you selected an official binary wheel.
For a manual wheel install, select a file that actually exists in the maintainer’s GitHub release assets. Check these separately:
| Field | Why it matters |
|---|---|
Python tag, for example cp312 |
Must match the interpreter ABI |
| PyTorch release tag | Native extension compatibility |
| CUDA release wheel variant | Published CUDA runtime compatibility, not a guessed filename |
cxx11abiTRUE/FALSE |
Must match the PyTorch C++ ABI where relevant |
| Platform and GPU support | A Linux wheel does not make unsupported GPUs compatible |
Do not paste a fabricated release URL assembled from an old release number and guessed CUDA tags. Also do not add FLASH_ATTENTION_FORCE_BUILD=TRUE as an innocent install flag. In upstream setup.py it forces a local build, which is precisely the work a wheel-first user may want to avoid.
The official README describes PyTorch 2.2 or later, a supported CUDA or ROCm toolchain, and FlashAttention-2 CUDA support for NVIDIA Ampere, Ada and Hopper GPUs. Turing GPUs require a separate supported implementation.
3. Stop a Ninja host-memory OOM without inventing a RAM formula
A compiler failure log may include:
c++: fatal error: Killed signal terminated program cc1plus
ninja: build stopped: subcommand failed.
Check host memory and OOM events, not just GPU VRAM:
free -h
# Some systems restrict kernel logs to privileged readers.
journalctl -k --no-pager -n 80 | grep -Ei 'oom|out of memory|killed process'
The maintainer recommends MAX_JOBS when Ninja compilation exhausts host RAM. Current upstream setup also accounts for NVCC_THREADS. For a compatible, authorized development host, a hypothetical low-concurrency build example is:
MAX_JOBS=2 NVCC_THREADS=2 python3 -m pip install flash-attn --no-build-isolation
This command was not executed on a FlashAttention CUDA host for this article. Those values are examples, not universal safe limits. Host-memory availability, compiler versions, extensions and architectures change the peak usage. If the machine is shared, coordinate before any source build.
Avoid the claim that each compiler job always uses exactly 4 GB or that an architecture override always reduces a 40-minute build to 5 minutes. No such universal benchmark was established in this article.
4. Diagnose the actual failure signature
| Symptom | Likely layer | First check |
|---|---|---|
The detected CUDA version ... mismatches |
Extension compilation toolchain | Host nvcc, CUDA_HOME, torch.version.cuda, PyTorch extension traceback |
cc1plus ... Killed or Ninja OOM |
Host-memory pressure | Available RAM, concurrent compiler jobs, kernel OOM record |
undefined symbol in flash_attn_2_cuda |
Extension/PyTorch binary mismatch or stale artifact | Torch version, ABI flag, wheel path, rebuild history |
no kernel image is available |
GPU architecture absent from binary | Compute capability and published compatible kernels |
ModuleNotFoundError |
Wrong Python environment or incomplete install | python3 -m pip show flash-attn and interpreter path |
If a previously working native extension starts failing after a PyTorch upgrade, treat binary compatibility as a hypothesis and compare the original and current PyTorch, CUDA, ABI and flash-attn wheel or build metadata. A changed PyTorch version alone does not prove that this caused the exception.
For fatal error: cuda.h: No such file or directory, refer to the related CUDA development headers and Triton build troubleshooting. For an architecture error, see no kernel image available.
5. Verify the extension without a fabricated hardware log
An import check verifies packaging, not kernel execution. The following optional, small kernel test needs a supported GPU and enough free memory, and should run only on hardware you are authorized to use.
import torch
from flash_attn import flash_attn_qkvpacked_func
assert torch.cuda.is_available(), "CUDA GPU unavailable"
qkv = torch.randn(
1, 128, 3, 4, 64,
device="cuda",
dtype=torch.float16,
)
output = flash_attn_qkvpacked_func(qkv, dropout_p=0.0, causal=True)
torch.cuda.synchronize() # expose deferred CUDA errors before printing success
print("output shape:", tuple(output.shape))
The expected output shape for this example is (1, 128, 4, 64) if it runs successfully. This article does not assert that it ran, that the host was an RTX 4090 or that the operation completed without errors. Synchronizing the device helps surface deferred kernel errors before the shape is printed, but the example remains unexecuted here. An import and one small forward pass do not certify long-context training or production inference.
If your real workload still fails, preserve the exact traceback, version tuple, extension wheel identity, relevant GPU model and measured host/GPU memory state. Each piece helps identify which layer needs changing.
Sources and correction history
- Dao-AILab README: supported platforms, install command, Ninja memory
- Dao-AILab setup.py: cached wheels, force-build behavior, compiler jobs
- FlashAttention official releases and actual wheel filenames
- PyTorch 2.15 C++/CUDA extension documentation
- PyTorch CUDA compiler compatibility implementation
Independent source review updated November 12, 2026. Installation and kernel commands are examples, not a claim of successful GPU execution. Original publication date: September 13, 2026.