Miscalculation in Numbers for iOS

I found an miscalculation in Numbers for iPad.


to reproduce is very easy and I've tested it on three diffrent tabels. I'm located in germany and using the German version maybe this is the problem.


last creation was on line 4

B4C4D4E4F4
931=(B4-C4)/108,5=D4-E4
9319,28,50,6999999


not a very complicate calculation... result on f4 should be 0,7


any sugestions?

Numbers 1.6.2-OTHER, iOS 6

Posted on Sep 25, 2012 2:46 AM

Reply
3 replies

Sep 25, 2012 9:03 AM in response to SierraX

Ok I think I found the solution. The auto function for decimal is as useless as the auto for exponential (Scientific Decimals). With more than 15 decimals it changes to 0,699999999999999. OpenOffice and Numbers for Mac do the same with more than 15 decimals.
Change decimals to somethings under 15 should work.
The funny thing is... the Problem seems existing in OO Spreadsheet and Numbers only when the result is between -1 and +1. Will look tomorrow to an old excel.

Hope Apple change the Auto in Numbers for iOS.

Oct 3, 2012 8:25 AM in response to SierraX

The reason you see this when using more than 15 decimal places is that computers can't actually represent the value 0.7 exactly. This is because all numbers are in binary format (i.e. each digit is either a 1 or a 0). It's the same as the reason why it's impossible to represent the fraction 1/3 exactly in base 10. This behaviour occurs on all CPU architectures, operating systems, and applications.


When you *do* see the value 0.7, it's because the routine that converts the number from floating point to text for display on screen does so only to a certain number of decimal places. Once you go less than 15 decimal places, the truncation causes it to come 0.70000000000000, which is then converted to 0.7 for display.


See https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/02Numerics/Double/paper.pdf for more details


The alternative is to use fixed point arithmetic, which is where you essentially represent numbers in smaller units, for example using cents in terms of dollars, if you're dealing with financial data. Banks and other financial institutions use this because money has to be represented as a whole number of cents. Just changing the display format as I've described above, because you can still get calculation errors. For example if you start with 0 and 0.1 to it 1,000,000 times, the result will be 100,000.000001 - not what you'd expect.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Miscalculation in Numbers for iOS

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.