Why can an intermediate GPU tensor remain allocated after a Python loop finishes?
GPU / CUDALevel: IntermediateTensor object lifetimes
Why can an intermediate GPU tensor remain allocated after a Python loop finishes?
Hints
- Does a Python loop create a new variable scope?
- Look for references in containers and object attributes.
Answer
Python loop bodies do not introduce a separate local-variable scope. The last tensor bound to a variable, or stored in a container or object, may remain referenced after the loop. Release unnecessary references or limit their lifetime with a function boundary. Distinguish live tensor storage from allocator-reserved memory; deleting one variable is insufficient if other references still exist.
Source and licence: PyTorch contributors — FAQ · BSD-3-Clause · answer: Quantoros editorial