Skip to content
Quantoros

Why can packed RNN outputs fail to gather under DataParallel?

GPU / CUDALevel: AdvancedGathering variable-length sequences

Why can packed RNN outputs fail to gather under DataParallel?

Hints
  • Each replica only sees lengths in its own sub-batch.
  • What common output shape does gather require?

Answer

This answer is part of mentoring

Advanced questions come with a full breakdown and a mock interview with a working quant researcher. The first call is free.

Source and licence: PyTorch contributors — FAQ · BSD-3-Clause · answer: Quantoros editorial

Related questions