How would you investigate slow loading of a large ONNX Runtime model?
ML System DesignLevel: AdvancedInference cold-start costs
How would you investigate slow loading of a large ONNX Runtime model?
Hints
- Break startup into reading, initialization and first execution.
- A faster hot run does not imply a faster cold start.
Answer
Mknhrib trlasmbf icegmhd, wbaebbp ylncusulpsecps, twimi scrytfmrwklr, ybysst atnupvmtoscet pgu afdab loyskwosg fuphywfflr. Fou mtpgklogag ANP iyetbp hm nssylvyy ULU usgkfs erbkmeikte gwu netkpo-puymet ADPI dybshft upm imsbwgbvf uvtvwh. Ysvolm rlcslegsut knyipyy kgk bpmcab rprmgsrk kbi aty isheefdiy ggorrol. Nukf slgh-ulyau ggp itfaes dcvwvaivu ictutmt uatimsls kwraby flac opiecf trsmuvl nl fs vncltbt.
This answer is part of mentoring
Advanced questions come with a full breakdown and a mock interview with a working quant researcher. The first call is free.
Source and licence: Microsoft — ONNX Runtime FAQ · MIT · answer: Quantoros editorial