How to Fix Ray Actor Out Of Memory (OOM) Errors in Tensor-Parallel vLLM on Linux
LLM Serving(Updated )

How to Fix Ray Actor Out Of Memory (OOM) Errors in Tensor-Parallel vLLM on Linux

Correction and scope. An earlier version presented an unsupported crash statistic, broad memory-monitor changes and a guaranteed fix. Those claims have been removed. This guide helps operators distinguish failure layers and plan a bounded recovery; it does not establish the cause of any particular actor failure.

Start with the first causal event

A RayActorError, a worker exit, or a failed vLLM engine task is a symptom. Find the earliest relevant error across the engine, worker, Ray and host logs before interpreting the final traceback. Align timestamps and node identities: a controller may report a failure after the worker that caused it has already exited.

First establish whether the affected deployment actually uses Ray. The current vLLM parallelism documentation describes native Python multiprocessing as the default for single-node inference and Ray as the default for multi-node inference, with an executor override available. Those are current documentation defaults, not proof of the backend selected by an older release or a particular deployment. Check its startup configuration and logs.

Record the image digest, vLLM and Ray versions, model revision, GPU and node topology, executor, container limits, workload concurrency, and the first error on each affected node. Preserve relevant logs without exposing credentials or request data.

Read-only evidence checklist

On authorized systems, use the inspection tools documented for the installed versions to collect:

  • The worker exit status, container termination state, and kernel or cgroup events around the incident.
  • Ray raylet and actor logs, if Ray is the executor, including any monitor message identifying a selected worker.
  • Host and container memory usage and limits, plus /dev/shm capacity and usage in the affected container.
  • Per-GPU memory usage and the earliest CUDA or communication error, if available.
  • Object-store usage and retained references when an object-store error is present.

A snapshot taken after a restart may no longer show the pressure that caused the failure. Missing kernel logs or a negative search also do not prove that an OOM event did not occur. Before using a command or option from another runbook, check the installed tool’s help and the documentation for its pinned release; the evidence supplied for this article does not validate a complete set of inspection commands.

Choose the failure branch

Kernel or container memory pressure

If kernel or cgroup evidence identifies an OOM kill, compare the affected process and time with the container limit, node memory pressure, worker count and workload. A container termination record can corroborate the event. Exit code 137 or a SIGKILL alone is not enough to identify who sent the signal or why.

If the evidence points to a container limit, inspect whether workload concurrency, model loading, other processes or the limit itself explains the pressure. Reduce a measured demand or revise the budget only after considering other workloads sharing the node. Preserve headroom for system and Ray processes.

Ray memory-monitor termination

The Ray OOM prevention documentation says the raylet monitor considers combined worker heap, object-store and raylet memory and can kill a task or actor when the applicable threshold is exceeded. In the documented default configuration, the threshold is 0.95; it is configurable and should not be assumed for an existing cluster.

The same documentation distinguishes resource isolation: beginning with Ray 2.56, its described protections depend on correctly configured system-reserved memory and, for some protections, accurate logical memory requests. Check the installed release and whether isolation is enabled before interpreting its controls. Task and actor retry behavior also differs: repeated task retries can mask continuing pressure, while actor restarts are subject to the actor’s restart policy.

Keep the monitor enabled while investigating. Disabling a protective mechanism does not create memory capacity and can move the failure to the operating system. Review the largest worker heaps, actor and task concurrency, object retention and node budget first.

Object store or shared-memory pressure

An object-store error calls for a different investigation from a CUDA allocation failure. Ray’s memory-management documentation describes Linux object-store allocation in /dev/shm, distributed reference counting, and spilling when the store fills. Local references, pending-task arguments and references captured in other objects can keep objects pinned. Inspect those references and spilling behavior before choosing a new capacity.

The current Ray documentation describes a default object-store size of 30% of available memory, subject to a 200 GB maximum and a Linux /dev/shm constraint. Verify these rules against the pinned Ray release and actual configuration; they do not imply that 30% of physical RAM is always allocated. When comparing worker memory, avoid adding shared memory once per worker: Ray notes that doing so can double-count object-store pages.

vLLM’s Docker documentation explains that PyTorch uses shared memory between processes, particularly for tensor parallel inference, and documents container shared-memory and host IPC options. Shared memory is a plausible branch when allocation errors or measured capacity support it, not a diagnosis by itself. Determine the affected container’s IPC mode and measured need. If a change is justified, assess an appropriately sized dedicated shared-memory allocation against the container runtime documentation. Host IPC changes the isolation boundary and requires a separate security decision; no fixed size is a universal remedy.

GPU memory or communication failure

If the first causal message is a CUDA OOM, investigate GPU allocations: model weights, KV cache, other resident processes, sequence lengths and concurrent requests. The current vLLM scaling guide describes startup logs reporting KV-cache capacity and an estimated maximum concurrency for a specified request length. Treat that estimate as planning evidence for the configuration that produced it, not a measured throughput result.

If the earliest error instead concerns NCCL initialization or transport, preserve the error and per-node context and investigate communication and peer-worker failures. A later actor death can be a consequence rather than the initiating memory event. Verify any transport-specific procedure against the installed stack before changing it.

Recover with a controlled comparison

  1. Select one supported branch. Tie the proposed change to a specific log, limit or measurement. If evidence conflicts or is missing, collect more before tuning.
  2. Define the change and rollback. Record the current setting, expected effect, affected nodes, maintenance window and rollback condition. Check release-specific semantics before applying any option.
  3. Keep the deployment isolated. The vLLM security guidance states that multi-node inter-node communications are insecure by default and must be protected on an isolated network. Review cluster and inference endpoint exposure under the deployment’s security policy.
  4. Repeat a comparable workload. Pin the software, model and topology; document request lengths and concurrency. Compare the first error, completion, memory headroom and effects on neighboring workloads.
  5. Keep or revert based on evidence. A successful restart alone does not show that sustained pressure is resolved. Revert if the change causes regressions or weakens required isolation, and investigate the next supported branch.

Primary references

Validation boundary and source-review notice — November 13, 2026: This correction is based on the supplied first-party documentation excerpts; the vLLM pages identify themselves as developer-preview documentation. No commands, GPU workloads, container deployments or benchmarks were executed for this article. The excerpts do not establish when current behavior was introduced unless a version is explicitly stated above. Verify the pinned releases and local evidence before applying a change.

BC

Written by B. Cho

Systems & AI Infrastructure Engineer

Specializing in GPU inference engines (vLLM, SGLang, Ollama), CUDA memory allocation, and Model Context Protocol (MCP) infrastructure on Linux workstations.