Skip to content
Quantoros

Why can an intermediate GPU tensor remain allocated after a Python loop finishes?

GPU / CUDALevel: IntermediateTensor object lifetimes

Why can an intermediate GPU tensor remain allocated after a Python loop finishes?

Hints
  • Does a Python loop create a new variable scope?
  • Look for references in containers and object attributes.

Answer

Python loop bodies do not introduce a separate local-variable scope. The last tensor bound to a variable, or stored in a container or object, may remain referenced after the loop. Release unnecessary references or limit their lifetime with a function boundary. Distinguish live tensor storage from allocator-reserved memory; deleting one variable is insufficient if other references still exist.

Source and licence: PyTorch contributors — FAQ · BSD-3-Clause · answer: Quantoros editorial

Related questions