
How to Fix 'RuntimeError: mat1 and mat2 shapes cannot be multiplied' in PyTorch on Linux
Start at the failing multiplication
For a two-dimensional multiplication (M, K) @ (K, N), the inner dimensions must agree. The PyTorch 2.15 torch.matmul reference documents matrix multiplication shape rules; incompatible contracted dimensions explain the error in the title, but do not identify which upstream decision produced a bad shape.
Use the traceback to find the actual operation before changing a layer:
- Is it an explicit multiplication? Record both complete operand shapes and identify the axes being multiplied. Check whether a transpose or an earlier reshape was intended.
- Is it a linear layer or a projection inside another module? Record the complete input and weight shapes, the configured feature widths, and the axes those widths represent. Compare them with the contract documented for your installed release.
- Where did the unexpected width originate? Trace preprocessing, backbone output, pooling, flattening, concatenation, and projection configuration back to the point where the actual shape diverges from the intended one.
- Which side reflects the intended architecture? Restore an accidental upstream change if the existing layer and checkpoint are correct. Consider changing the layer only if the new feature width is deliberate.
For analysis, a conventional linear mapping with an input shaped (batch, in_features) and a stored weight shaped (out_features, in_features) has this relationship:
input (batch, in_features) @ weight.T (in_features, out_features)
-> output (batch, out_features)
The PyTorch 2.15 nn.Linear API explicitly documents y = xA^T + b, an input (*, in_features), an output (*, out_features) and a weight shaped (out_features, in_features). Its leading dimensions are preserved; they are not necessarily a single batch axis. Confirm your installed release’s semantics before changing an architecture.
Treat a changed layer as an architecture change
Suppose a layer expects 256 features but receives 512. A replacement accepting 512 may make the dimensions compatible, but it has different parameters and does not automatically preserve the learned mapping. An existing checkpoint may not load as intended, and an optimizer created before replacement may still reference the old parameters.
Choose a path deliberately:
- If preprocessing or an upstream transformation changed unintentionally, restore the contract expected by the existing head.
- If the representation is intentionally wider, design and train the changed head, or define and validate an explicit parameter migration.
- In either case, verify checkpoint loading, optimizer parameter references and state, and behavior on representative data.
A successful forward pass establishes neither equivalent predictions nor correct sample-to-label alignment.
Trace spatial features into the head
For a feature map interpreted as (batch, channels, height, width), flattening all non-batch axes gives channels × height × width values per sample. Thus (batch, 16, 16, 16) contains 4,096 values per sample. A head designed for 1,024 features cannot consume that flattened representation directly.
First confirm that those axes have the stated meaning in your model. Then decide which representation the task needs:
- Fixed geometry: Derive the feature width from the actual backbone output for each supported input size. Account for convolution, stride, padding, and pooling throughout the path.
- Variable geometry, fixed-width head: Consider a pooling design with a fixed spatial output if discarding spatial detail is appropriate. With an output interpreted as
(batch, channels, 1, 1), flattening the non-batch axes would leave one value per channel. - Spatial detail matters: Retain the spatial representation or design a head that uses it. Averaging changes the information available to the classifier.
Before selecting an adaptive pooling API, check its pinned-release documentation for accepted input shapes and output-size semantics. This article does not claim that a particular pooling setup was executed or that pooling preserves spatial information.
Preserve sample boundaries during reshaping
A reshape can preserve the number of elements while changing their meaning. A final batch shaped (40, 512) contains 20,480 elements; grouping those values as (64, 320) has the same element count but no longer represents 40 samples. A later layer or loss may fail, or the error may remain silent.
Check the axis order at the reshape boundary. If the first axis represents samples, preserve that sample count while combining only the intended feature axes. Verify the resulting width and compare sample ordering with the labels. Include a smaller final batch if the data pipeline can produce one.
If the requested dimensions cannot account for the original element count, the reshape must be reconsidered. The versioned Tensor.reshape reference documents reshaping with the same number of elements, and the Tensor Views guide explains that the result may be a view or a copy. Neither proves the transformed axes retain their intended meanings.
Check attention interfaces separately
In a custom design, projecting an embedding width of 768 into 16 heads of width 64 produces a total head width of 1,024. After those heads are concatenated, an output projection expecting 768 input features cannot directly consume that 1,024-wide representation.
One design could use 16 heads of width 48, giving a concatenated width of 768. Another could retain 64-wide heads and use an output projection accepting 1,024, provided the rest of the architecture supports it. The equality between embedding width and concatenated head width is a design choice in this example, not a rule for every custom attention implementation.
Check query, key, and value interfaces, head splitting, attention axes, concatenation, output projection, and residual connections individually. Shape agreement at the output projection alone does not establish correct attention behavior.
Check lazy initialization before adopting it
A lazy layer can be attractive when the intended input width is initially unknown, but do not assume it will adapt to every later input width. The PyTorch 2.15 LazyLinear reference documents inferring in_features during the first forward, but this does not mean a lazy layer dynamically changes widths for later inputs. Verify initialization, parameter shapes, device and dtype, optimizer membership and checkpoint loading on your installed release. Plan a representative initialization input and verify the resulting parameter shapes before training or loading a checkpoint.
If variable image resolutions produce different flattened widths, decide whether a fixed-width representation is suitable for the task. A lazy layer does not by itself establish that the representations from those resolutions are compatible with one trained head.
Validate on the actual model
Record the PyTorch version, model and preprocessing configurations, and checkpoint identity. At the failing boundary, inspect complete operand shapes and their semantic axes. Check a normal batch, any supported remainder batch, and each supported input geometry. After an architecture change, validate checkpoint and optimizer handling and evaluate the model with representative data.
Keep GPU diagnostics separate from shape diagnosis. The PyTorch 2.15 CUDA semantics notes state that CUDA graph replay assumes the same tensor sizes and layouts and prohibits dynamic shapes during replay. A variable-batch design therefore needs a compatible capture strategy. The 2.15 torch.cuda.device_count reference defines that function as returning the number of available GPUs; the count does not validate tensor dimensions.
Primary sources
-
PyTorch 2.15 CUDA semantics: CUDA graph replay constraints.
-
PyTorch 2.15
torch.cuda.device_count: the function’s return value.
Validation boundary: The examples are shape analyses, not reports of executed code, training, or measured results. The linked versioned PyTorch Linear, reshape, and LazyLinear references establish their respective shape/initialization contracts. A separate adaptive pooling API reference and a model-specific runtime test are still needed before adopting a pooling-based repair. The supplied source excerpts were reviewed for this correction on November 13, 2026; they do not establish what the documentation contained on the original publication date.