Skip to content
Quantoros

What is quantization and which format should you use?

ML System DesignLevel: IntermediateMemory Management

What is quantization and which format should you use?

Hints
  • What changes in numerical range and representation precision?
  • Distinguish memory savings from measured speedup on the target device.

Answer

Quantization reduces model size by using fewer bits per parameter:

FormatBitsMemoryQualityUse Case
FP32324 bytes/paramPerfectTraining (gradients need precision)
BF16162 bytes/paramNear-losslessInference/fine-tuning standard
FP881 byte/paramMinimal lossH100 native, training
INT881 byte/param~1% quality lossInference, widely supported
INT4 (GPTQ)40.5 bytes/param~2-3% quality lossMemory-constrained inference
4-bit NF4 (QLoRA)40.5 bytes/param~1% loss for fine-tuningQLoRA fine-tuning
GGUF (llama.cpp)2-8VariableConfigurableLocal/CPU inference

Recommended: BF16 for production inference on modern GPUs; INT8 with AWQ or GPTQ when memory is the constraint; GGUF for local deployment.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions