Paste raw PyTorch training output and get a conservative debugging brief that keeps memory evidence while folding progress bars, wandb chatter, NCCL spam, and dependency frames.
CUDA out-of-memory logs are expensive to paste into Codex or Claude Code because the useful memory signal is often surrounded by repeated progress output and distributed-training noise. This page shows the kind of evidence tokensift keeps.
Before and after
Before: raw paste
$ python train.py --model llama-7b --batch-size 64 --precision fp16
[12:04:09] INFO worker=2 prefetch batch=812
[12:04:09] INFO worker=2 prefetch batch=812
Epoch 3: 82%|████████▏| 821/1000 [loss=2.39]
wandb: step=821 loss=2.39 lr=2e-4
wandb: step=821 loss=2.39 lr=2e-4
NCCL INFO Bootstrap : Using eth0
NCCL INFO Bootstrap : Using eth0
Traceback (most recent call last):
File "/workspace/train.py", line 214, in <module>
loss = trainer.step(batch)
File ".../site-packages/torch/nn/modules/module.py", line 1775, in _call_impl
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.25 GiB.
GPU0: 23.65 GiB total; 1.08 GiB free; 21.40 GiB allocated; 22.49 GiB reserved.
After: conservative debugging brief
Debug CUDA OOM.
Type:
- CUDA / PyTorch OOM
Keep:
- cmd: python train.py --model llama-7b --batch-size 64 --precision fp16
- frame: /workspace/train.py:214 -> loss = trainer.step(batch)
- error: torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.25 GiB.
- detail: model=llama-7b; batch-size=64; precision=fp16
- memory: GPU0 23.65 GiB total; 1.08 GiB free; 21.40 GiB allocated; 22.49 GiB reserved
Folded:
- duplicate worker INFO, progress bar, wandb x2, NCCL x2, torch internals
Ask: root cause, smallest fix, verify command.
Preserved evidence
- training command and model arguments
- batch size and precision
- top user-code frame
- PyTorch OOM message
- GPU total/free/allocated/reserved memory stats
Folded noise
- duplicate worker INFO lines
- tqdm-style progress updates
- wandb metric chatter
- repeated NCCL bootstrap lines
- torch dependency internals that are not the likely root cause
How to read a CUDA OOM before changing code
Start with the allocation that failed and the memory snapshot reported at the same moment. “Tried to allocate” describes the immediate request, while total capacity, free memory, memory allocated by PyTorch, and memory reserved but unallocated describe different parts of the process state. They should not be collapsed into one number. A small failed allocation can still be the final request after earlier tensors consumed most of the device.
Next, connect the memory line to the command and the closest user-code frame. Batch size, sequence or image dimensions, precision, model choice, gradient accumulation, and the operation named in the frame give a coding agent concrete variables to investigate. A bare OOM message without that context often produces a generic list of optimizations instead of a testable first change.
In distributed training, identify the affected GPU or rank when the log contains it. Repeated NCCL and worker messages may be secondary fallout after one rank fails. TokenSift folds repeated transport chatter, but it keeps distinct memory lines and non-dependency frames so the brief can show whether the failure appears local to one device.
Choose a small verification experiment
- Reduce the batch size while keeping the same model, input shape, and precision, then rerun the failing step.
- If effective batch size matters, compare gradient accumulation with a smaller per-device batch rather than changing several memory controls at once.
- Record whether mixed precision or activation checkpointing was already enabled before recommending it.
- Investigate allocator fragmentation only when the retained allocated/reserved/free evidence supports that hypothesis.
- For an inference failure, preserve request dimensions and concurrency; training-only advice may not apply.
What TokenSift does with the log
The CUDA parser uses deterministic patterns to retain the command, version and driver hints, model arguments, memory statistics, OOM lines, and user-code frames. It counts progress output, wandb lines, repeated NCCL information, worker chatter, and dependency frames as folded categories. The generated brief reports those categories instead of pretending they never existed.
TokenSift does not profile GPU allocations, inspect tensors, or prove that a particular optimization is safe. It can only organize the evidence present in the pasted text. Use the brief to ask for a root-cause hypothesis, the smallest change, and a command or measurement that could falsify that hypothesis.
Limits and review checklist
- Keep separate OOM events if they come from different ranks, retries, or processes.
- Restore any unique framework line that the folded summary grouped too aggressively.
- Do not treat rough token reduction as a measure of diagnostic quality.
- Remove secrets, customer identifiers, private paths, and internal URLs before submission.
- Review the final brief against the raw failure boundary before sharing it with a coding agent.
For the complete parser boundary, see How TokenSift works, the CUDA support matrix, and the guide to evidence-preserving debugging briefs.
Privacy note: No LLM API call. No raw logs sent to external model providers, ad providers, or third-party analytics. Hosted submissions may be stored in a private first-party database for diagnostics and retained for up to 30 days by default, so review and redact before use.
Open tokensift and paste your own log.