Floating point, integers & quantization error

§1 IEEE-754 refreshed
A float is sign + exponent + mantissa, with precision proportional to magnitude. Float32, float16, and bfloat16 spend the same total bits in three very different ways — and the choice has real consequences for training.
§2 Integers and fixed-point
Integers give up dynamic range to get uniform precision and roughly 4× the SIMD throughput. Fixed-point — real = scale × integer — is the simplest way to use integer hardware to represent reals.
§3 Quantization & the int8 dot product
Affine quantization, calibration, and the SIMD int8 dot kernel that every quantized inference stack runs on — VPDPBUSD on x86 (one fused instruction), SDOT on ARM (likewise), one accumulation in int32 either way.