Skip to content
Quantoros

Why can nvidia-smi show GPU memory in use after PyTorch tensors are deleted?

GPU / CUDALevel: IntermediateAllocated versus reserved GPU memory

Why can nvidia-smi show GPU memory in use after PyTorch tensors are deleted?

Hints
  • Distinguish live allocations from reusable cached blocks.
  • Which memory can the allocator release without losing tensors?

Answer

PyTorch caches freed device allocations for reuse, so process-level memory reported by nvidia-smi includes reserved memory and context overhead, not only live tensors. Compare allocated and reserved counters, inspect remaining references and processes. empty_cache() can release unused cached blocks to other applications, but it does not free live tensors or fix a retained computational graph.

Source and licence: PyTorch contributors — FAQ · BSD-3-Clause · answer: Quantoros editorial

Related questions