Floating-Point Precision and Its Effects to Model Inference
Overview This post is divided into five parts; they are: • How Floating-Point Numbers Work • Common Precision Formats • Loading a Model at a Different Precision • Mixed Precision and Sensitive Operations • Measuring the Result How Floating-Point Numbers Work A floating-point number contains a sign, an exponent, and a significand (often called the mantissa).
We haven't written up this one. Machine Learning Mastery has the full story — the link below goes straight to it.
