
How to Fix 'CUDA Setup failed despite GPU being available' in bitsandbytes on Linux
If an existing environment reports CUDA Setup failed despite GPU being available, capture the full stack trace and installed package version first. The wording is not, by itself, a version-independent root-cause diagnosis and does not establish that creating a driver-library symlink or globally exporting LD_LIBRARY_PATH is safe. Current bitsandbytes versions support multiple backends, so the installed package version, supported platform and accelerator matter before making changes.
Updated November 12, 2026: Removed blanket pip uninstall, global CUDA path edits, system libcuda.so symlink creation, host IPC recommendations and an unmeasured claim that a diagnostic command proves production readiness. This revision follows version-specific installation and backend compatibility documents.
Step 1: Identify the exact environment
Record your accelerator backend, Python/PyTorch/bitsandbytes versions and whether you are on a desktop, container or Jetson-style embedded Linux environment.
python3 -m pip show bitsandbytes torch
python3 --version
nvidia-smi
The last command applies to environments with NVIDIA management utilities; its absence alone does not diagnose bitsandbytes on a non-NVIDIA backend. Importing a GPU package can initialize runtime resources. The bitsandbytes installation guide gives platform-specific support tables rather than promising that every build can use any CUDA version or CPU/GPU combination.
Step 2: Compare the installed wheel to current support
The official installation requirements list Python 3.10+ and PyTorch 2.4+ for the current general package. NVIDIA support depends on exact CUDA toolkit, architecture and chosen operation. In that guide, 8-bit optimization and NF4/FP4 quantization have different compute-capability requirements from LLM.int8(). These are current documented requirements, not claims that older bitsandbytes versions had the same matrix.
If the existing application pins an older bitsandbytes version, consult its matching release documentation before using current installation commands. Record the original lockfile, pip freeze or environment specification and preserve a rollback path. Do not silently upgrade a working deployment just to resolve an unexplained error.
The current guide also distinguishes NVIDIA Jetson L4T/JetPack from generic Linux aarch64 wheels; a server-class aarch64 package may not be compatible with Jetson’s L4T runtime. Use the platform-specific source-build instructions where required, rather than assuming architecture alone guarantees binary compatibility.
Step 3: Diagnose packaging separately from driver access
| Symptom | Investigation | Avoid |
|---|---|---|
| Import reports CPU-only backend | Installed bitsandbytes release, PyTorch backend, wheel/platform match | Assuming a global CUDA path must be added |
| Container cannot access NVIDIA runtime | GPU device grants and NVIDIA Container Toolkit configuration | Host IPC sharing or arbitrary driver symlinks |
| Unsupported compute capability | The exact operation and version’s supported GPU matrix | Treating all quantization modes as equivalent |
| PyTorch CUDA is unavailable | Installed PyTorch build and accelerator backend | Overwriting system driver libraries |
| Only a larger model fails | Per-process GPU memory and model budget | Concluding that a bitsandbytes wheel is broken |
The NVIDIA Container Toolkit guide covers container GPU exposure and driver capability settings. Docker documents that --ipc=host shares the host IPC namespace, which is unrelated to granting an NVIDIA device to a container. Do not use host IPC as a generic bitsandbytes fix.
Step 4: Repair only a confirmed environment mismatch
If evidence confirms the Python package is incompatible with the installed runtime, create a separate virtual environment or rebuilt container and install a supported, version-pinned stack following the current bitsandbytes installation guide and PyTorch installation selector. Validate the test environment before changing the original deployment.
Do not add /usr/lib/x86_64-linux-gnu to LD_LIBRARY_PATH system-wide, add libcuda.so links under /usr/lib, run ldconfig or install a different NVIDIA driver solely because a bitsandbytes import failed. Such changes can affect unrelated applications and do not prove the root cause.
If source compilation is actually needed, use the official version-matched instructions and a controlled build environment. Do not blindly copy an unpinned git clone plus pip install . sequence into production.
Step 5: Validate only what was tested
After selecting the right backend, carry out an authorized, version-appropriate smoke test and measure the exact operation your application uses (8-bit optimization, NF4 inference, etc.). A successful import or python -m bitsandbytes diagnostic, when supported by your release, does not establish end-to-end model accuracy, speed, or production readiness.
This article includes no executed GPU benchmark, no package uninstall/install and no host/container changes. The safest fix remains a hypothesis supported by versions, logs and workload-specific tests.
Primary references
- bitsandbytes installation, support matrix and Jetson note
- NVIDIA Container Toolkit container GPU visibility
- Docker container run / IPC
- PyTorch supported installations
Reviewed November 12, 2026. Supported backends and packaging details should be checked against the specific installed release.