Glossary

QAT : Quantization Aware Training — means the model was trained to survive being compressed, so it loses less quality than ordinary compression would cost.

MoE : Mixture of Experts — a model architecture where only a subset of parameters activate for any given input, allowing much larger total model size while keeping inference fast.