pxlngn blogtagquantization

#quantization

07/26/2026AI

Evaluating quantised models: perplexity, KL divergence & real task evals

How do I know IQ3_M is still good enough? Measuring the quality loss of a quantisation – with llama-perplexity, KL divergence against the base model, and a small task eval of your own.

#tutorial
#quantization
#evaluation
#ai
#llm

07/26/2026AI

Quantising Qwen3-32B by hand – a 32B model on 16 GB of VRAM

How I quantised Qwen3-32B with llama.cpp and an importance matrix so it runs on a 16 GB GPU: GGUF conversion, imatrix, quant levels compared, VRAM budgeting and layer offload.

#tutorial
#quantization
#qwen
#ai
#llm