Skip to content
Quantoros

How would you investigate slow loading of a large ONNX Runtime model?

ML System DesignLevel: AdvancedInference cold-start costs

How would you investigate slow loading of a large ONNX Runtime model?

Hints
  • Break startup into reading, initialization and first execution.
  • A faster hot run does not imply a faster cold start.

Answer

This answer is part of mentoring

Advanced questions come with a full breakdown and a mock interview with a working quant researcher. The first call is free.

Source and licence: Microsoft — ONNX Runtime FAQ · MIT · answer: Quantoros editorial

Related questions