Skip to content
Quantoros

Why can vLLM generate different text for the same prompt even after setting a seed?

LLM & AILevel: AdvancedBatched generation reproducibility

Why can vLLM generate different text for the same prompt even after setting a seed?

Hints
  • Separate random sampling from floating-point execution.
  • How can one early token difference propagate?

Answer

This answer is part of mentoring

Advanced questions come with a full breakdown and a mock interview with a working quant researcher. The first call is free.

Source and licence: vLLM contributors — FAQ · Apache-2.0 · answer: Quantoros editorial

Related questions