|
|
Regarding Calculator and low precision values
In trying out a few calculations, Calculator.app does not correctly implement "precision" to mean significant figures either. For example, with precision set at 2:
10,000.00 + 2,500.00 = 12,500 5.25 * 3.00 = 15.75 Further, if you set precision to 2 and type 15.75, the calculator converts the displayed value to 16. It does, however, return the correct value in the following calculation: 15.75 / 3.00 = 5.25 Sadly, this is just an artefact of the way it understands "precision". For the following operation returns the incorrect answer: 15.75 / 3 = 5.25 Ultimately, however, rules for significant figures don't allow you to specify the number you would like. They are based on the precision of the values used in the calculation. Thus, it would be strange if the Calculator.app were using "Precision" to mean significant figures. Best, darin |
SearchFrom our Sponsor...Latest Mountain Lion HintsWhat's New:HintsNo new hintsComments last 2 daysLinks last 2 weeksNo recent new linksWhat's New in the Forums?
Hints by TopicNews from Macworld
From Our Sponsors |
|
Copyright © 2014 IDG Consumer & SMB (Privacy Policy) Contact Us All trademarks and copyrights on this page are owned by their respective owners. |
Visit other IDG sites: |
|
|
|
Created this page in 0.14 seconds |
|