×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Are you an
Engineering professional?
Join Eng-Tips Forums!
• Talk With Other Members
• Be Notified Of Responses
• Keyword Search
Favorite Forums
• Automated Signatures
• Best Of All, It's Free!

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

#### Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

# Decimal Accuracy

## Decimal Accuracy

(OP)
I continue to hang out and mostly lurk in an editor's facebook group. I have been accused of mansplaining during a discussion of metric conversions. I have claimed that 4.5 miles converts to 7.2 km, not 7.24 km.

Is there a documented, formal standard that dictates decimal accuracy?

Is there such a thing as a style guide for engineering calculations?

--
JHG

### RE: Decimal Accuracy

Sure, this is from NIST SP-811

B.7.2 Rounding converted numerical values of quantities
• The use of the factors given in Secs. B.8 and B.9 to convert values of quantities was demonstrated in Sec. B.3. In most cases the product of the unconverted numerical value and the factor will be a numerical value with a number of digits that exceeds the number of significant digits (see Sec. 7.9) of the unconverted numerical value. Proper conversion procedure requires rounding this converted numerical value to the number of significant digits that is consistent with the maximum possible rounding error of the unconverted numerical value.

• Example: To express the value l = 36 ft in meters, use the factor 3.048 E−01 from Sec. B.8 or Sec. B.9 and write
l = 36 ft × 0.3048 m/ft = 10.9728 m = 11.0 m.

• The final result, l = 11.0 m, is based on the following reasoning: The numerical value “36” has two significant digits, and thus a relative maximum possible rounding error (abbreviated RE in this Guide for simplicity) of ± 0.5/36 = ± 1.4 %, because it could have resulted from rounding the number 35.5, 36.5, or any number between 35.5 and 36.5. To be consistent with this RE, the converted numerical value “10.9728” is rounded to 11.0 or three significant digits because the number 11.0 has an RE of ± 0.05/11.0 = ± 0.45 %. Although this ± 0.45 % RE is one-third of the ± 1.4 % RE of the unconverted numerical value “36,” if the converted numerical value “10.9728” had been rounded to 11 or two significant digits, information contained in the unconverted numerical value “36” would have been lost. This is because the RE of the numerical value “11” is ± 0.5/11 = ± 4.5 %, which is three times the ± 1.4 % RE of the unconverted numerical value “36.” This example therefore shows that when selecting the number of digits to retain in the numerical value of a converted quantity, one must often choose between discarding information or providing unwarranted information. Consideration of the end use of the converted value can often help one decide which choice to make.
• Note: Consider that one had been told initially that the value l = 36 ft had been rounded to the nearest inch. Then in this case, since l is known to within ± 1 in, the RE of the numerical value “36” is ± 1 in/(36 ft × 12 in/ft) = ± 0.23 %. Although this is less than the ± 0.45 % RE of the number 11.0, it is comparable to it. Therefore, the result l = 11.0 m is still given as the converted value. (Note that the numerical value “10.97” would give excessive unwarranted information because it has an RE that is one-fifth of ± 0.23 %.)
• TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

### RE: Decimal Accuracy

(OP)
IRstuff,

Thank you. That is exactly what I needed. I would give you are star, but the button is not visible in this forum or on this thread.

Here is an interesting link someone put up: Significant Figure Rules Penn State University.

--
JHG

### RE: Decimal Accuracy

I'd say you're being pedantic ! 4.5 miles is 7.242km ... 7.24kkm is a better translation than 7.2km ... why would you say 7.24km is wrong and 7.2km is preferred ?

if it were a math/engineering forum you could say the input is 4.5 miles +-0.05 miles so the answer is 7.24km +- 0.08km. You might say the input has only 2 significant digits so the output should too. I'd've said the increase in "accuracy" of the output is due to the translation of the units.

quit hanging out where you don't belong !

### RE: Decimal Accuracy

4.5 miles is 7242.0 m

Hard to argue that 7242.0 m is not 7.242 km

Anyway, you are right. You will not convince anyone. It probably won't make any difference to the world.

### RE: Decimal Accuracy

rb1957,

The issue is that the original number, 4.5 is clearly an approximation itself, so could be 4.4 or 4.6, but not 4.55

So the implied error of up to 0.05 miles shouldn't then imply a greater accuracy in the translation. yes 4.5000 miles is 7242m. but then it wasn't 4.5000, it was 4.5. So it could have been 4.5462miles which would then be 7319m (7.3km or 7.32km). so to stay true to the original accuracy 7.2 will be correct.

It's always annoyed me when what is clearly a nominal number in one set of unit is then magically translated as some mega precise number in another set, instead of rounding up or down to make it the same accuracy or the same nominal value. So e.g. 12" NPS becomes 300NPS and not 304.8 NPS.

The numbers might not be accurate, but a sign I read recently said that this area of parkland was 2000 acres (clearly a rough total), but then in brackets it had something like (4064ha). Now 4,000 would have been a much better number IMHO and it's good to find something that expresses this in a scientific manner.

Remember - More details = better answers
Also: If you get a response it's polite to respond to it.

### RE: Decimal Accuracy

It is a bit pedantic, but I'm often accused of over-precision, since I like to typically have 3 significant figures of precision, but that's usually to show that precision has been carried through all the calculations. Nevertheless, using 7.242 or even 7.24 is overly precise, since the rounding error (RE) for the original number is ±0.05, which means that the any precision beyond ±0.08 km is unnecessary.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

### RE: Decimal Accuracy

(OP)
IRstuff,

Pedantry goes both ways.

#### Quote:

Oh no! My garbage is being raided by a 181.5 kg bear!

This does provoke funny letters to the editors of newspapers.

--
JHG

### RE: Decimal Accuracy

drawoh,

Exactly my point - I assume this was a "standard" 400 lb bear, +/- 50 lbs.

So I assume this really now should be a 200 kg bear unless it is rather thinner nowadays and is a 150kg bear...

Remember - More details = better answers
Also: If you get a response it's polite to respond to it.

### RE: Decimal Accuracy

Calculations should never introduce precision that wasn’t there before the calculation. That’s something we lost when the slide rule was replaced with the calculator.

I’ll see your silver lining and raise you two black clouds. - Protection Operations

### RE: Decimal Accuracy

what's the accuracy of 1/8" ... 1/16", yes?

then is 0.125" more accurate than 1/8" ? if its 0.12" then is the error +- 0.05" or 0.06" (closer to 1/16") ?

there's more than enough pedantic-ness to go around ...

### RE: Decimal Accuracy

(OP)

#### Quote (LittleInch)

drawoh,

Exactly my point - I assume this was a "standard" 400 lb bear, +/- 50 lbs.

So I assume this really now should be a 200 kg bear unless it is rather thinner nowadays and is a 150kg bear...

Alternatively, you can grab the bear, wrestle him into submission, and weigh him.

--
JHG

### RE: Decimal Accuracy

I was always told "Never convert to a metric tolerance closer than the imperial tolerances on the drawing.", by my head draughtsman.
B.E.

You are judged not by what you know, but by what you can do.

### RE: Decimal Accuracy

#### Quote (there's more than enough pedantic-ness to go around ...)

Surely you mean "pedantry"?

(Sorry - I couldn't resist the urge to be pedantic - pedant that I am! )

### RE: Decimal Accuracy

> Surely you mean "pedantry"?

I think pedanticism would be a better choice.

=====================================
(2B)+(2B)' ?

### RE: Decimal Accuracy

So how many seconds are in a year?

### RE: Decimal Accuracy

a 400-year Gregorian average year is 31556952.000 seconds, but 31556925.975 (rounding error from somewhere in MC) seconds according to Mathcad, which is the mean Tropical year

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

### RE: Decimal Accuracy

Is that 1 year or 1.000?

### RE: Decimal Accuracy

I'm more interested in the "mansplaining" thing. I had to look up what that is.

Is it a legitimate complaint of being condescending, or is it an ad hominem attack to beat you in the argument?

### RE: Decimal Accuracy

More than legitimate; Cris Collinsworth had to walk back this:

#### Quote (https://www.si.com/extra-mustard/2020/12/03/cris-c...)

"In particular, the ladies that I met. They had really specific questions about the game. I'm like, 'Wow.' You're just blown away by how strong the fans are here in this town."

Possibly just bad phrasing, etc., but he clearly understood the condescension dripping from that impromptu comment. Nevertheless, troglodytes still exist.

Now, I'm an equal opportunity mansplainer, I'll mansplain to men or women

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

### RE: Decimal Accuracy

The Collinsworth example seems pretty silly.

Imagine if I (middle aged male with a buzzcut) walk into a salon to buy a gift for my wife. The employee would instantly realize she's probably not dealing with a guy who pays attention to his hair like Fabio would. She'd have to womansplain to me what the latest and greatest products are. I should be more sensitive so that I can detect when I should be offended.

### RE: Decimal Accuracy

#### Quote (271828)

Imagine if I (middle aged male with a buzzcut) walk into a salon to buy a gift for my wife. The employee would instantly realize she's probably not dealing with a guy who pays attention to his hair like Fabio would. She'd have to womansplain to me what the latest and greatest products are. I should be more sensitive so that I can detect when I should be offended.

The issue at hand, at least when I hear about this, is about the frequency of contexts in which women feel they are talked down to.

So yes- you at the salon is a reasonable example. But for a lot of women they can receive that type of treatment at mechanics, with repair people, at stores selling technology, at hardware stores etc etc etc. It's a real thing.

### RE: Decimal Accuracy

(OP)

#### Quote (271828)

I'm more interested in the "mansplaining" thing. I had to look up what that is.

I am trying to find the Facebook thread. I noted that I am unable to Womansplain. I recall that a woman ("woman here") provided the link to Penn State.

Let's face it, mansplaining is fun. You do need to be correct.

--
JHG

### RE: Decimal Accuracy

My understanding of mansplaining is explaining things to a woman (or another man, for that matter)that the person is already aware of and may even be an expert at.

https://xkcd.com/1716/

### RE: Decimal Accuracy

I guess one would have to be on the other end of mansplaining to get it. Things are always clearer on the receiving end.

### RE: Decimal Accuracy

#### Quote (stevenal)

Is that 1 year or 1.000?

Neither. The question is as clear as it would be on an exam, where you will not be allowed to ask questions; thus, you are forced to actually answer the question. I understand how hard that is for engineers.

42

### RE: Decimal Accuracy

Number of significant digits always implies a corresponding accuracy. If the absolute true value of a quoted number is within + or - 0.5 times the value of decimal place of the last number quoted, it is correct else it is in error.

Pi, accurate to 10 significant figures, is 3.141592653
If you told me pi = 3.14 I would call that correct, because the true value is found between 3.14 + 0.01/2 = 3.145 and 3.14-0.01/2 = 3.135
If you told me pi = 3.1, that would also count true, since pi > 3.05 and < 3.15.
If you told me pi = 3 you would be correct, since pi lies between 2.5 and 3.5
If you told me pi = 3.0, then no, that's wrong, because pi is not found between 2.95 and 3.05

### RE: Decimal Accuracy

1503-44...

I assume you meant: "Number of significant digits always implies a corresponding accuracy precision."

============
"Is it the only lesson of history that mankind is unteachable?"
--Winston S. Churchill

### RE: Decimal Accuracy

Precisely. Thanks.

### RE: Decimal Accuracy

My SwissMicros DM42 calculator gives pi to 33 decimal places, which is a bit more than I need.

============
"Is it the only lesson of history that mankind is unteachable?"
--Winston S. Churchill

### RE: Decimal Accuracy

Yeah, Mathcad is fun for that; it this rendition of pi.

3.141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117067982148086513282306647093844609550582231725359408128481117450284102701938521105559644622948954930381964428810975665933446128475648233786783165271201909

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

### RE: Decimal Accuracy

Back in the 1980s my Casio fx180p often spent its weekends computing PI.

Steve

#### Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

#### Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

#### Resources

Low-Volume Rapid Injection Molding With 3D Printed Molds
Learn methods and guidelines for using stereolithography (SLA) 3D printed molds in the injection molding process to lower costs and lead time. Discover how this hybrid manufacturing process enables on-demand mold fabrication to quickly produce small batches of thermoplastic parts. Download Now
Examine how the principles of DfAM upend many of the long-standing rules around manufacturability - allowing engineers and designers to place a partâ€™s function at the center of their design considerations. Download Now
Taking Control of Engineering Documents
This ebook covers tips for creating and managing workflows, security best practices and protection of intellectual property, Cloud vs. on-premise software solutions, CAD file management, compliance, and more. Download Now

Close Box

# Join Eng-Tips® Today!

Join your peers on the Internet's largest technical engineering professional community.
It's easy to join and it's free.

Here's Why Members Love Eng-Tips Forums:

• Talk To Other Members
• Notification Of Responses To Questions
• Favorite Forums One Click Access
• Keyword Search Of All Posts, And More...

Register now while it's still free!