×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Decimal Accuracy

Decimal Accuracy

Decimal Accuracy

(OP)
I continue to hang out and mostly lurk in an editor's facebook group. I have been accused of mansplaining during a discussion of metric conversions. I have claimed that 4.5 miles converts to 7.2 km, not 7.24 km.

Is there a documented, formal standard that dictates decimal accuracy?

Is there such a thing as a style guide for engineering calculations?

--
JHG

Replies continue below

Recommended for you

RE: Decimal Accuracy

Sure, this is from NIST SP-811

B.7.2 Rounding converted numerical values of quantities
  • The use of the factors given in Secs. B.8 and B.9 to convert values of quantities was demonstrated in Sec. B.3. In most cases the product of the unconverted numerical value and the factor will be a numerical value with a number of digits that exceeds the number of significant digits (see Sec. 7.9) of the unconverted numerical value. Proper conversion procedure requires rounding this converted numerical value to the number of significant digits that is consistent with the maximum possible rounding error of the unconverted numerical value.


  • Example: To express the value l = 36 ft in meters, use the factor 3.048 E−01 from Sec. B.8 or Sec. B.9 and write
    l = 36 ft × 0.3048 m/ft = 10.9728 m = 11.0 m.

  • The final result, l = 11.0 m, is based on the following reasoning: The numerical value “36” has two significant digits, and thus a relative maximum possible rounding error (abbreviated RE in this Guide for simplicity) of ± 0.5/36 = ± 1.4 %, because it could have resulted from rounding the number 35.5, 36.5, or any number between 35.5 and 36.5. To be consistent with this RE, the converted numerical value “10.9728” is rounded to 11.0 or three significant digits because the number 11.0 has an RE of ± 0.05/11.0 = ± 0.45 %. Although this ± 0.45 % RE is one-third of the ± 1.4 % RE of the unconverted numerical value “36,” if the converted numerical value “10.9728” had been rounded to 11 or two significant digits, information contained in the unconverted numerical value “36” would have been lost. This is because the RE of the numerical value “11” is ± 0.5/11 = ± 4.5 %, which is three times the ± 1.4 % RE of the unconverted numerical value “36.” This example therefore shows that when selecting the number of digits to retain in the numerical value of a converted quantity, one must often choose between discarding information or providing unwarranted information. Consideration of the end use of the converted value can often help one decide which choice to make.
  • Note: Consider that one had been told initially that the value l = 36 ft had been rounded to the nearest inch. Then in this case, since l is known to within ± 1 in, the RE of the numerical value “36” is ± 1 in/(36 ft × 12 in/ft) = ± 0.23 %. Although this is less than the ± 0.45 % RE of the number 11.0, it is comparable to it. Therefore, the result l = 11.0 m is still given as the converted value. (Note that the numerical value “10.97” would give excessive unwarranted information because it has an RE that is one-fifth of ± 0.23 %.)
  • TTFN (ta ta for now)
    I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
    FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

    RE: Decimal Accuracy

    (OP)
    IRstuff,

    Thank you. That is exactly what I needed. I would give you are star, but the button is not visible in this forum or on this thread. sad

    Here is an interesting link someone put up: Significant Figure Rules Penn State University.

    --
    JHG

    RE: Decimal Accuracy

    I'd say you're being pedantic ! 4.5 miles is 7.242km ... 7.24kkm is a better translation than 7.2km ... why would you say 7.24km is wrong and 7.2km is preferred ?

    if it were a math/engineering forum you could say the input is 4.5 miles +-0.05 miles so the answer is 7.24km +- 0.08km. You might say the input has only 2 significant digits so the output should too. I'd've said the increase in "accuracy" of the output is due to the translation of the units.

    quit hanging out where you don't belong !

    another day in paradise, or is paradise one day closer ?

    RE: Decimal Accuracy

    4.5 miles is 7242.0 m

    Hard to argue that 7242.0 m is not 7.242 km

    Anyway, you are right. You will not convince anyone. It probably won't make any difference to the world.

    RE: Decimal Accuracy

    rb1957,

    The issue is that the original number, 4.5 is clearly an approximation itself, so could be 4.4 or 4.6, but not 4.55

    So the implied error of up to 0.05 miles shouldn't then imply a greater accuracy in the translation. yes 4.5000 miles is 7242m. but then it wasn't 4.5000, it was 4.5. So it could have been 4.5462miles which would then be 7319m (7.3km or 7.32km). so to stay true to the original accuracy 7.2 will be correct.

    It's always annoyed me when what is clearly a nominal number in one set of unit is then magically translated as some mega precise number in another set, instead of rounding up or down to make it the same accuracy or the same nominal value. So e.g. 12" NPS becomes 300NPS and not 304.8 NPS.

    The numbers might not be accurate, but a sign I read recently said that this area of parkland was 2000 acres (clearly a rough total), but then in brackets it had something like (4064ha). Now 4,000 would have been a much better number IMHO and it's good to find something that expresses this in a scientific manner.

    Remember - More details = better answers
    Also: If you get a response it's polite to respond to it.

    RE: Decimal Accuracy

    It is a bit pedantic, but I'm often accused of over-precision, since I like to typically have 3 significant figures of precision, but that's usually to show that precision has been carried through all the calculations. Nevertheless, using 7.242 or even 7.24 is overly precise, since the rounding error (RE) for the original number is ±0.05, which means that the any precision beyond ±0.08 km is unnecessary.

    TTFN (ta ta for now)
    I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
    FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

    RE: Decimal Accuracy

    (OP)
    IRstuff,

    Pedantry goes both ways.

    Quote:

    Oh no! My garbage is being raided by a 181.5 kg bear!

    This does provoke funny letters to the editors of newspapers.

    --
    JHG

    RE: Decimal Accuracy

    drawoh,

    Exactly my point - I assume this was a "standard" 400 lb bear, +/- 50 lbs.

    So I assume this really now should be a 200 kg bear unless it is rather thinner nowadays and is a 150kg bear...

    Remember - More details = better answers
    Also: If you get a response it's polite to respond to it.

    RE: Decimal Accuracy

    Calculations should never introduce precision that wasn’t there before the calculation. That’s something we lost when the slide rule was replaced with the calculator.

    I’ll see your silver lining and raise you two black clouds. - Protection Operations

    RE: Decimal Accuracy

    what's the accuracy of 1/8" ... 1/16", yes?

    then is 0.125" more accurate than 1/8" ? if its 0.12" then is the error +- 0.05" or 0.06" (closer to 1/16") ?

    there's more than enough pedantic-ness to go around ...

    another day in paradise, or is paradise one day closer ?

    RE: Decimal Accuracy

    (OP)

    Quote (LittleInch)


    drawoh,

    Exactly my point - I assume this was a "standard" 400 lb bear, +/- 50 lbs.

    So I assume this really now should be a 200 kg bear unless it is rather thinner nowadays and is a 150kg bear...

    Alternatively, you can grab the bear, wrestle him into submission, and weigh him.

    --
    JHG

    RE: Decimal Accuracy

    I was always told "Never convert to a metric tolerance closer than the imperial tolerances on the drawing.", by my head draughtsman.
    B.E.

    You are judged not by what you know, but by what you can do.

    RE: Decimal Accuracy

    Quote (there's more than enough pedantic-ness to go around ...)


    Surely you mean "pedantry"?

    (Sorry - I couldn't resist the urge to be pedantic - pedant that I am! smile)

    http://julianh72.blogspot.com

    RE: Decimal Accuracy

    > Surely you mean "pedantry"?

    I think pedanticism would be a better choice.

    =====================================
    (2B)+(2B)' ?

    RE: Decimal Accuracy

    So how many seconds are in a year?

    RE: Decimal Accuracy

    Is that 1 year or 1.000?

    RE: Decimal Accuracy

    I'm more interested in the "mansplaining" thing. I had to look up what that is.

    Is it a legitimate complaint of being condescending, or is it an ad hominem attack to beat you in the argument?

    RE: Decimal Accuracy

    More than legitimate; Cris Collinsworth had to walk back this:

    Quote (https://www.si.com/extra-mustard/2020/12/03/cris-c...)

    "In particular, the ladies that I met. They had really specific questions about the game. I'm like, 'Wow.' You're just blown away by how strong the fans are here in this town."

    Possibly just bad phrasing, etc., but he clearly understood the condescension dripping from that impromptu comment. Nevertheless, troglodytes still exist.

    Now, I'm an equal opportunity mansplainer, I'll mansplain to men or women winky smile

    TTFN (ta ta for now)
    I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
    FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

    RE: Decimal Accuracy

    The Collinsworth example seems pretty silly.

    Imagine if I (middle aged male with a buzzcut) walk into a salon to buy a gift for my wife. The employee would instantly realize she's probably not dealing with a guy who pays attention to his hair like Fabio would. She'd have to womansplain to me what the latest and greatest products are. I should be more sensitive so that I can detect when I should be offended.

    RE: Decimal Accuracy

    Quote (271828)

    Imagine if I (middle aged male with a buzzcut) walk into a salon to buy a gift for my wife. The employee would instantly realize she's probably not dealing with a guy who pays attention to his hair like Fabio would. She'd have to womansplain to me what the latest and greatest products are. I should be more sensitive so that I can detect when I should be offended.

    The issue at hand, at least when I hear about this, is about the frequency of contexts in which women feel they are talked down to.

    So yes- you at the salon is a reasonable example. But for a lot of women they can receive that type of treatment at mechanics, with repair people, at stores selling technology, at hardware stores etc etc etc. It's a real thing.

    RE: Decimal Accuracy

    (OP)

    Quote (271828)


    I'm more interested in the "mansplaining" thing. I had to look up what that is.

    I am trying to find the Facebook thread. I noted that I am unable to Womansplain. I recall that a woman ("woman here") provided the link to Penn State.

    Let's face it, mansplaining is fun. You do need to be correct.

    --
    JHG

    RE: Decimal Accuracy

    My understanding of mansplaining is explaining things to a woman (or another man, for that matter)that the person is already aware of and may even be an expert at.

    https://xkcd.com/1716/

    RE: Decimal Accuracy

    I guess one would have to be on the other end of mansplaining to get it. Things are always clearer on the receiving end.

    RE: Decimal Accuracy

    Quote (stevenal)

    Is that 1 year or 1.000?

    Neither. The question is as clear as it would be on an exam, where you will not be allowed to ask questions; thus, you are forced to actually answer the question. I understand how hard that is for engineers.

    RE: Decimal Accuracy

    42

    RE: Decimal Accuracy

    Number of significant digits always implies a corresponding accuracy. If the absolute true value of a quoted number is within + or - 0.5 times the value of decimal place of the last number quoted, it is correct else it is in error.

    Pi, accurate to 10 significant figures, is 3.141592653
    If you told me pi = 3.14 I would call that correct, because the true value is found between 3.14 + 0.01/2 = 3.145 and 3.14-0.01/2 = 3.135
    If you told me pi = 3.1, that would also count true, since pi > 3.05 and < 3.15.
    If you told me pi = 3 you would be correct, since pi lies between 2.5 and 3.5
    If you told me pi = 3.0, then no, that's wrong, because pi is not found between 2.95 and 3.05

    RE: Decimal Accuracy

    1503-44...

    I assume you meant: "Number of significant digits always implies a corresponding accuracy precision."

    ============
    "Is it the only lesson of history that mankind is unteachable?"
    --Winston S. Churchill

    RE: Decimal Accuracy

    Precisely. Thanks.

    RE: Decimal Accuracy

    My SwissMicros DM42 calculator gives pi to 33 decimal places, which is a bit more than I need. smile

    ============
    "Is it the only lesson of history that mankind is unteachable?"
    --Winston S. Churchill

    RE: Decimal Accuracy

    Yeah, Mathcad is fun for that; it this rendition of pi.

    3.141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117067982148086513282306647093844609550582231725359408128481117450284102701938521105559644622948954930381964428810975665933446128475648233786783165271201909

    TTFN (ta ta for now)
    I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
    FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

    RE: Decimal Accuracy

    Back in the 1980s my Casio fx180p often spent its weekends computing PI.

    Steve

    Red Flag This Post

    Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

    Red Flag Submitted

    Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
    The Eng-Tips staff will check this out and take appropriate action.

    Reply To This Thread

    Posting in the Eng-Tips forums is a member-only feature.

    Click Here to join Eng-Tips and talk with other members! Already a Member? Login



    News


    Close Box

    Join Eng-Tips® Today!

    Join your peers on the Internet's largest technical engineering professional community.
    It's easy to join and it's free.

    Here's Why Members Love Eng-Tips Forums:

    Register now while it's still free!

    Already a member? Close this window and log in.

    Join Us             Close