this post was submitted on 29 Apr 2024
1 points (100.0% liked)
Mathematics
553 readers
1 users here now
A community for discussing mathematics and developments in mathematics.
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
@L4r4
This is why, when you convert units, to be aware of the significant digits you rounded off. If you round off digits in intermediate steps, then the final result can end up slightly off. If you want to be safe, use fractions or exact values in everything, or use 10 significant digits for intermediate steps at all times.