Troubleshooting Floating-Point Precision Errors
Understand why 0.1 + 0.2 ≠ 0.3 in computers and how to handle precision issues in calculations.
Key Takeaways
- In virtually every programming language, 0.1 + 0.2 evaluates to 0.30000000000000004 instead of 0.3.
- A 64-bit double-precision float uses 1 bit for sign, 11 bits for exponent, and 52 bits for mantissa (significand).
- Floating-point errors are usually negligible for scientific calculations and graphics.
- For financial calculations: use integer arithmetic in cents/pennies, or dedicated decimal types (Python Decimal, Java BigDecimal, PostgreSQL NUMERIC).
- Python: `from decimal import Decimal` for exact decimal arithmetic.
Percentage Calculator
Calculate percentages, increases, decreases, and ratios
The Infamous 0.1 + 0.2 Problem
In virtually every programming language, 0.1 + 0.2 evaluates to 0.30000000000000004 instead of 0.3. This isn't a bug — it's a fundamental consequence of representing decimal fractions in binary floating-point (IEEE 754). Just as 1/3 can't be exactly represented in decimal (0.333...), 1/10 can't be exactly represented in binary.
How IEEE 754 Works
A 64-bit double-precision float uses 1 bit for sign, 11 bits for exponent, and 52 bits for mantissa (significand). This gives approximately 15-17 significant decimal digits of precision. Numbers like 0.5 (1/2) and 0.25 (1/4) are exact because they're powers of 2. Numbers like 0.1 (1/10) require infinite binary digits, so they're rounded.
When Precision Matters
Floating-point errors are usually negligible for scientific calculations and graphics. They become critical in financial calculations (rounding errors accumulate across millions of transactions), comparison operations (checking if two floats are equal), and summing long series of numbers (errors accumulate).
Solutions by Context
For financial calculations: use integer arithmetic in cents/pennies, or dedicated decimal types (Python Decimal, Java BigDecimal, PostgreSQL NUMERIC). For comparisons: use epsilon-based comparison (abs(a-b) < 1e-10) instead of exact equality. For accumulation: use compensated summation (Kahan algorithm). For display: round to the appropriate number of decimal places before showing to users.
Language-Specific Tools
Python: from decimal import Decimal for exact decimal arithmetic. JavaScript: libraries like decimal.js or big.js (native BigDecimal proposal pending). Java: BigDecimal class. PostgreSQL: NUMERIC type with arbitrary precision. Always specify the precision you need and choose the right tool for that precision level.
関連ツール
関連ガイド
How to Use Scientific Notation and Number Formatting
Scientific notation makes very large and very small numbers manageable. This guide covers notation systems, significant figures, and formatting conventions used in science, engineering, and finance.
Unit Conversion Best Practices for Developers
Incorrect unit conversions have caused spacecraft crashes and medical errors. This guide covers best practices for implementing unit conversions in software, including precision handling and common pitfalls.
Percentage Calculations: Common Formulas and Pitfalls
Percentage calculations appear simple but hide common errors, especially around percentage change, percentage points, and compound percentages. This guide clarifies the math behind everyday percentage problems.
Matrix Operations: A Visual Guide for Developers
Matrices are fundamental to graphics, machine learning, and data processing. This guide explains matrix addition, multiplication, transposition, and inversion with visual examples and practical applications.
Troubleshooting Floating-Point Precision Errors
Floating-point arithmetic produces surprising results that can cause bugs in financial calculations, comparisons, and cumulative operations. This guide explains why these errors occur and how to handle them.