It's not an "error," i.e., it's a fundamental limitation of using binary floating point to represent decimal numbers.  Your examples are not equivalent calculations.

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

try Round ( not round ) and see if it gets you what you wanted.

According to my Mathcad help file: "floor(z) Returns the greatest integer < z."

Looks to me like 9 is the expected result.

.. which fails to explain why units make a difference. ?

Again, the with-units and without-units calculations are not equivalent.  Mathcad casts all quantities with units into an internal SI representation.  Nonetheless, if you divide 12.75 by 1275*10^-3, you'll get something less than 10, and will do so for any variation where the exponent is nonzero.

BTW, my help for floor(z) states "greatest integer <=z"

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

AS IRstuff says, when you create expressions with units, Mathcad converts all units into their SI equivalents (as floating point numbers), and it also converts all numeric terms into floating point numbers (whether written as integers or floating point numbers). Thus the expression which is entered as:

     (12.75 m) / (1275 mm)

is actually calculated internally as something like:

     (12.75 * 1.0) / (1275.0 * (1.0 / 1000.0))

We human beings are "clever" enough to see that this is exactly equal to 10, but Mathcad evaluates this expression as being "equal" to 9.999999999999998 - which is the answer it gives for the expression with units. (Try it!)

http://julianh72.blogspot.com  

jhardy,

That's the result I get in Mathcad. The same computer gives a different result for a similar calculation in Excel. If I format the cell for the maximum 30 decimal places, I get 10 followed by 30 zeros. Any idea how Microsoft gets around the binary floating point limitation?  

@stevenal,

Sorry! I can't help you with explaining why Excel and Mathcad get slightly different results in the 15th decimal place ...

... Other than to say they are programmed by different teams, using different compilers and different algorithms. (In fact, it would be quite remarkable if two different programs gave identical results to 15 decimal places for all mathematical floating-point expressions, given the accuracy constraints of representing real numbers in binary format!)

http://julianh72.blogspot.com  

Actually it's more surprising that they don't give the same result since both just call on the division function of the processor rather than each coding division within the programs.

While the floating point processor instructions might be the same at the assembly language level, the internal representation of numbers are not the same.  Mathcad's and Excel's format handlings are different.  Mathcad's highest input exponent is 307 and Excel's is 308, while the lowest is identical (-307).  While Mathcad can round up to 10^308, you cannot enter that value directly, nor can you get there formulaically.  However, Excel allows numbers up to 1.7976931E308 to be entered.

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

OK, that was only valid for M11, it looks like M14 and up have been modified to allow the same maximum value of 1.79769313485999*10^308.  

Nonetheless, the residual from the original question is 2^-49, which is 3 bits off from the 52-bit fractional part of a double precision float.

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

@davidbeach:
What makes you think that Mathcad and Excel would carry out EXACTLY the same sequence of calculations & function calls with EXACTLY the same binary representations for ALL terms in PRECISELY the same order?

Excel has no explicit means of representing units; in my example, I suggested that Mathcad MIGHT represent "1 mm" as "(1.0 / 1000.0) times 1 metre"; but it MIGHT store it as "0.001 metres"; or it MIGHT (probably!) use some other internal algorithm to handle the units.

I have no idea whether Mathcad does some sort of "symbolic shuffling" to determine the units of the solution, or if it multiplies and divides its internal representations of units (nor the sequence of such "cancelling" operations), or what. Bearing in mind the limits of precision of binary floating point maths, "0.001" and "(1.0 / 1000.0)" will not necessarily yield EXACTLY the same result carried across to the last available binary digit. Thus, multiplying by 0.001 or dividing by 1000.0 will not necessarily yield exactly the same result, because you are actually calling different processor functions (possibly in a different sequence of operations), using different stored binary numbers.

http://julianh72.blogspot.com  

When I made my comparison, I used the unitless (12.75 * 1.0) / (1275.0 * (1.0 / 1000.0)) expression in both Mcad and Excel.

I was also under the impression that math was more about the processessor than the higher level coding, explaining the pentium bug that affected all programs in the same way as I recall.

Good discussion here. Never really questioned floating point math since abandoning my slide rule. Thanks for the education.

BTW: "If I format the cell for the maximum 30 decimal places, I get 10 followed by 30 zeros" is an illusion.  

Do a "=pi()^2" and format the cell for 30 decimal places, and you'll see 16 zeros at the end, even though there are ostensibly a potential for 28 decimal places of precision.

However, Mathcad does have the capability of doing substantially high precision using its symbolic math processor, but that ptocessor is not unit-friendly.  

If you're willing to mess up the units system a bit by redefining mm=0.001m, the symbolic processor will give you the correct numerical answer to 12.75m/1275mm, to whatever precision you want, and assign the answer to a variable. you could then use the numerical processor for the floor function on that variable.

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

Sorry for my late response, but thanks for the answers/discussion.

I understand that software has to be programmed in certain ways (I have written some 'software' myself), but don't 'limitations' like the one discussed, just defeat the purpose of Math-software in some way?

If I wouldn't have used the floor function I would haven't have had a single problem with 10 actually being 9,99...98, but in this specific case it could actually be make-or-break. So is the lesson learned from this that you should be really carefully using 'non-math' functions, like floor? Or should you perhaps 'never' use units, which is one of the reasons I like Mathcad so much.

In my case using round will solve my problem. Are there any more tips/tricks people should know of before using software like this? It scared the h*ll out of me when I found this out though .

I was going to mention this earlier, but senior moment(s)...  Since you are fretting about a quantity that is not particularly meaningful in the real world, you can set the zero threshold to something like 10, which means that Mathcad will ignore any remainders smaller than 10^-10 in its calculations, so that FLOOR(12.75m/1275mm) will result in 10.  10^-10 m is an angstrom, so that should be safe, although you could set it down to

This should done with care; otherwise, a global tolerance setting that high will give you silly results.  Typically, advanced users will set the zero tolerance to 307, i.e., 10^-307 at the get-go.

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

I wasn't realy fooled. I never thought my 32 bit machine would work to 30 decimal places past the 10. Just as easy to go to 30 as it is to 17.

Interesting story about 24 bit floating point leading to the death of 28 and injuring 100 more: http://www.ima.umn.edu/~arnold/disasters/patriot.html  

It's not true that Excel "gets around the binary floating point limitation".  On the spreadheet it does some rounding of the final decimal place, that sometimes works to avoid the problem discussed here, and sometimes doesn't.  On the whole I think it's more trouble than it's worth, especially since it can result in VBA and the spreadsheet giving different results.

More details in the links below, for anyone interested.

http://newtonexcelbach.wordpress.com/2011/12/20/when-does-35-not-equal-35/

http://newtonexcelbach.wordpress.com/2012/01/07/comparing-floating-point-numbers/

Doug Jenkins
Interactive Design Services
http://newtonexcelbach.wordpress.com/
 

Yech...  That sounds like a bunch of baloney, or the programmers were even more inept than described in the report.  If the errors were accumulating as described, the initial range would have been wrong as well, so it should have been impossible to get the target anywhere near the range gate for the second detection.  But, if they got the range correct on the first pulse, they were using delta time, and the second pulse range gate would be based on the delta time relative to the T0 of the second pulse.

A typical scenario is to start range counter at T0 for the first pulse.  Detect SCUD at T1.  Set range gates for second pulse at T2+(T1-T0)-deltaT*Vscud/c-gateT to T2+(T1-T0)-deltaT*Vscud/c+gateT, where deltaT=T2-T0.  Regardless of whether T0 and T1 and T2 are only relative time or absolute time, since the calculations only involve their time difference, the offset error should be irrelevant.  

I can't argue with the notion that the system time might have been off, but it should be moot anyway.
 

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

@Jwouters:
The issue with using ANY software is to understand its constraints and limitations, and be practical in how you use it; in particular, be aware that digital computers have a finite precision, and that there is a difference between "exactly equal to" and "close enough for all practical purposes".
For example, IRStuff suggested using a tolerance of 10^-10 m (one Angstrom); if you are building a bridge, or planning a trip across the country, that should be "near enough". But what if your calculation relates to the "target" of the Large Hadron Collider? Now you have to hit a target which is the size of a nucleus - MUCH smaller than one Angstrom - so you would have to have a MUCH finer tolerance setting! Check out the definition of the "barn" as a unit of area in nuclear physics: http://en.wikipedia.org/wiki/Barn_(unit) - and yes, Mathcad does know how big (small!) 1 barn is!)

http://julianh72.blogspot.com  

@jhardy1:
I'm now indeed more aware of the constraints/limitations of software like this. I just found it out the hard way

Funny thing is, I was thinking about the LHC as well. Imagine 9,99...98 vs 10 being the difference of 'finding a Higgs-Boson particle' vs 'finding some random particle' ... I guess (hope) those scientist are well aware of these 'errors'

It's incumbent on the engineer, or scientist, to understand the nature of his problem, his solution, and his tool.  While Mathcad does its normal numeric processing with double precision, quad precision math has been around and available for decades, and Mathcad's symbolic processor has arbitrary precision.

With Mathcad's symbolic processor, you could crank barns, outhouses, and sheds to the 10th power and beyond all day long.

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

Because of this type of rounding issues, well known in programming, when trimming a number, say X, we use X + epsilon instead.

Mathcad's TOL = 0.001 works as epsilon, and to simplify consider the alternative function
jfloor(X) = floor(abs(X) + TOL)* sign(X)

We keep these sort of things in a standardized include file.