How to Fix 'RuntimeError: Trying to backward through the graph a second time' in PyTorch on Linux
PyTorch & CUDA(Updated )

How to Fix 'RuntimeError: Trying to backward through the graph a second time' in PyTorch on Linux

The PyTorch error “Trying to backward through the graph a second time” means an operation is asking autograd to reuse saved information from an earlier graph after it was freed or changed. It is tempting to add retain_graph=True everywhere. That may mask the immediate error while increasing memory consumption and leaving the real computation or optimizer sequencing incorrect.

Updated November 12, 2026. The central repair here is to distinguish a graph reused unintentionally from a graph deliberately differentiated more than once. In particular, do not update parameters belonging to a shared backbone between two backward passes through that backbone simply because the first pass retained the graph.

A fast decision tree

What the training loop does Likely issue to inspect Safer first action
Adds each batch’s loss Tensor to a persistent running total Autograd history remains linked across batches Use a detached scalar for logging
Computes two losses from one shared backbone and calls backward() twice The second pass needs saved information from the same forward Combine losses and call backward once, when mathematically appropriate
Calls optimizer step() between two backward passes on shared parameters Saved tensors may no longer match parameter versions Complete required backward computations before any overlapping parameter update
Carries a recurrent hidden state across sequence chunks Graph history can span earlier chunks Decide the intended BPTT window, then detach state at the chosen boundary
Intentionally computes separate derivatives from a shared forward Multiple reverse traversals may genuinely be required Apply retain_graph only where justified, and test memory and gradients

The PyTorch backward reference states that retaining the graph is not needed in nearly all cases and often has a more efficient alternative. The autograd tutorial explains that a graph may need to be retained when several backward operations on that same graph are genuinely intended.

1. Logging is not a reason to retain a graph

A metric counter should usually contain ordinary numbers, not an evolving computation graph. For example, this pattern keeps links to earlier training graphs:

# Fragile if running_loss persists across minibatches:
running_loss = running_loss + loss

If the quantity is for display only, use a detached scalar:

running_loss += loss.detach().item()  # metric only; not the training loss

The detach API disconnects the metric tensor from the graph, and Tensor.item() returns a standard Python number for one-element tensors. This intentionally drops gradient history from the metric. Do not detach a quantity that is still needed in the actual training objective.

2. Shared backbone, two losses: one reverse traversal is the baseline

Consider a feature extractor shared by an actor head and a critic head. If both losses train the shared extractor, the simplest arrangement is usually a single weighted objective:

# Illustration, assumes torch and the model modules exist and are initialized.
features = backbone(inputs)
loss_actor = actor_loss(actor_head(features), actor_targets)
loss_critic = critic_loss(critic_head(features), critic_targets)

optimizer.zero_grad(set_to_none=True)
total_loss = loss_actor + 0.5 * loss_critic
total_loss.backward()
optimizer.step()

This is application pseudocode: backbone, actor_head, loss functions and targets are supplied by the reader. The optimizer must own the intended parameters without unintended duplicates. The example was not executed for this editorial review.

If you want different learning rates for different modules, that does not by itself require two backward passes. The official optimizer guide documents per-parameter options, including groups with different learning rates. An illustrative, unexecuted setup is:

# Construct ONCE during setup with the correct, non-overlapping parameters.
optimizer = torch.optim.Adam([
    {"params": backbone.parameters(), "lr": 1e-4},
    {"params": actor_head.parameters(), "lr": 3e-4},
    {"params": critic_head.parameters(), "lr": 2e-4},
])

Do not put the same parameter in multiple groups. Update after the combined backward pass. This example is a scheduling pattern, not a claim that the listed learning rates are optimal.

3. Why stepping between retained backwards can still be wrong

A fragile pattern is:

# Do NOT copy this as a working repair for shared parameters.
loss_actor.backward(retain_graph=True)
optimizer_actor.step()   # May modify parameters needed by second backward.
loss_critic.backward()
optimizer_critic.step()

If the backward passes share upstream parameters or saved tensors, the first optimizer update can change values that the next backward expects, causing an in-place/version counter error or incorrect training semantics. Merely retaining the graph does not freeze those values.

When separate backward passes are truly necessary, finish all backwards before stepping optimizers that touch any shared graph parameters. Check parameter ownership and gradient accumulation carefully. For most straightforward multi-task objectives, prefer the single combined backward instead.

The PyTorch autograd mechanics notes describe recorded operations and in-place correctness checks. Optimizer updates belong after all backward traversals that require the current shared parameter values. The TorchRL training tips discuss common reinforcement-learning graph-reuse mistakes.

4. Recurrent state: choose the intended history length

A hidden state passed through consecutive sequence chunks can preserve links to earlier forward computations. If your training procedure intentionally uses truncated backpropagation through time, the PyTorch detach API returns a new tensor disconnected from the current computation graph. Detaching a recurrent state at an intentionally chosen truncation boundary is a conditional training-design choice, not a verified automatic fix:

# Fragment after completing the gradient update for an intended BPTT window.
hidden = hidden.detach()

For an LSTM returning a tuple of hidden and cell states, review both tensors and their intended history under the chosen algorithm; a bare tensor-only fragment does not handle a tuple. Do not detach after every token when full-sequence gradients are required. A recurrence bug and an intended long backward-through-time operation require different remedies.

5. Debug the graph actually being reused

  1. Preserve the first autograd error, not only the later symptom.
  2. Find where the suspect loss or hidden state was created, whether it lives across steps, and whether any optimizer updated shared parameters before the final derivative.
  3. In a small authorized test, compare gradients for a one-backward combined objective against the intended formulation.
  4. Only after a proven requirement for multiple derivatives, test retain_graph on the specific earlier backward and measure memory behavior.
  5. Verify that no tensor was unintentionally detached, silently updated in-place or accidentally included in two optimizer groups.

PyTorch’s autograd reference also warns that using create_graph=True with backward can create reference cycles; it is a separate higher-order differentiation concern, not a general fix for the error discussed here.

Limits: Code in this article is reviewed as a source-grounded illustration, not a verified training workload or GPU benchmark. A successful small backward does not certify a complex actor-critic or distributed training run.

Primary references

Reviewed November 12, 2026. Model architecture and optimizer ownership determine the correct repair.

BC

Written by B. Cho

Systems & AI Infrastructure Engineer

Specializing in GPU inference engines (vLLM, SGLang, Ollama), CUDA memory allocation, and Model Context Protocol (MCP) infrastructure on Linux workstations.