Numbers division error when dividing with a calculated value

Hi,


I am having a major problem with a worksheet that I am developing with Numbers and the problem is that I am dividing a number with a calculated value and the result is wrong.


The raw numbers are 100,000 / .86 = 116,279.07, but the spreadsheet calculates it as 116666.67


In the spreadsheet, the 100000 is divided by a cell that contains a calculation which derived from 2 other cells which ends up being 60000/70000 which is 0.86 (2 decimal spaces). Even if I type in the numbers directly in that cell, i.e. 60000/70000 the the computed result is 0.86, the cell needing to use this in it's operation still works out to be incorrect. The only way for the calculation to work out correctly is to type in the 0.86 manually and overwriting the calculation.


Any idea as to what is going on?


Cheers,


Rick

iMac 27″ 5K, macOS 13.6

Posted on Dec 20, 2023 7:06 PM

Reply
Question marked as Top-ranking reply

Posted on Dec 20, 2023 7:39 PM

What is going on is that 100000 * 70000 / 60000 = 116666.666666667.


You may display 6/7 as .86, but in reality, the value is 0.857142857142857. The complete value is used for the subsequent calculations.


If you really want to use 0.86, then you have to use FIXED

FIXED(6/7,2) = 0.86

Similar questions

8 replies

Dec 20, 2023 9:12 PM in response to Roger Rex

I'd use ROUND(6/7,2) instead of FIXED.


FIXED returns a string (text, not a numerical value).


ROUND returns a number (which is more often what is intended).




When there is ambiguity Numbers is reasonably good at guessing what you mean. Though in general multiplying a number by a string/text doesn't work, Numbers should let you multiply the result of the FIXED formula that looks like a number by another number, for example. But it's generally not a good idea to make Numbers guess unless you have to.


FIXED is more often used to put a correctly formatted "number" within a longer piece of text or string all in one cell. Even then you can easily achieve the same effect with ROUND.




Note that you cannot format a cell with FIXED as Currency (cell B3 in the screenshot) but you can with a cell with ROUND (cell B4). As a result, if using FIXED you have to insert an extra $ (or other currency symbol) when putting together a string (A7, vs A8 in the screenshot have the formulas, with the results in column C).


Net, net, you might want to consider using the more common ROUND rather than FIXED, unless you encounter a special situation where FIXED works better. (I can't think of any right now, but there might be some.)


SG

Dec 21, 2023 6:33 AM in response to Roger Rex

Yes of course ROUND, I had a memory blank when I suggested FIXED.


But think carefully before using either. In 99% of the cases, the result you need is 116666.67, not 116279.07. It's best to carry as many decimals as possible as far as possible in the chain of formulas. Imagine if that result is land area and you then use it to calculate property tax. You certainly don't want to undercharge or overcharge, depending on how the rounding goes.

Dec 20, 2023 7:22 PM in response to Roger Rex

Hi,


I am having a major problem with a worksheet that I am developing with Numbers and the problem is that I am dividing a number with a calculated value and the result is wrong.


The raw numbers are 100,000 / .86 = 116,279.07, but the spreadsheet calculates it as 116666.67


In the spreadsheet, the 100000 is divided by a cell that contains a calculation which derived from 2 other cells which ends up being 60000/70000 which is 0.86 (2 decimal spaces). Even if I type in the numbers directly in that cell, i.e. 60000/70000 the the computed result is 0.86, the cell needing to use this in it's operation still works out to be incorrect. The only way for the calculation to work out correctly is to type in the 0.86 manually and overwriting the calculation.


Here is a link to the error, https://www.icloud.com/numbers/0f8rFrce7EXcjG0Rn7PqtzXyA#Blank_2


Any idea as to what is going on?


Cheers,


Rick

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Numbers division error when dividing with a calculated value

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.