Contact US

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Measurement data with respect to gage accuracy

Measurement data with respect to gage accuracy

Measurement data with respect to gage accuracy

I have a question regarding how to record inspection data from a measurement device. Recently, we had an ISO inspector here who stated that we're recording too many decimal places because it's more than the gage's accuracy. For example, Mitutoyo calipers have an accuracy of .001 in, but a resolution of .0005 in, so some data was recorded out to four decimal places.

So was the ISO inspector correct? If so, is there a standard to back up his statement (he didn't provide one)?

I've searched the forums, and came across two threads (thread281-168380: Inspection Accuracy VS. Dimensional Accuracy and thread286-122849: Decimal place inspection) that have information close to what I'm looking for, but not exactly.

RE: Measurement data with respect to gage accuracy

Take a look at ASTM E29

ASTM E29 - 06b Standard Practice for Using Significant Digits in Test Data to Determine Conformance with Specifications  

RE: Measurement data with respect to gage accuracy

Does that standard incorporate the manufacturer's accuracy specification for a gage?

RE: Measurement data with respect to gage accuracy

You must have a good system if the inspector is nit picking something like this.  

IMO you are doing exactly what you should be doing.  The inspector is wrong.  His way you would have to be rounding which would increase your uncertainty budget.  That is to say for example.  You have a tolerance of 1.005 to 1.010.  You measure 1.0095, your way says that with .001 uncertianty it could be as much as 1.0105, his way you would round to 1.010 the top of you tolerance, but now the uncertainty says that it could be as much as 1.011.  Both would be out of tolerance but depending on application this could be significant in acquiring devation approval.  On the low end of the tolerance if you measured say 1.0055, then your way it could acutually, with the .001 uncertianty be 1.0045. With the inspector rounding to 1.006 and an uncertainty of .001 you would consider the measurment no smaller than 1.005.  Now you have yours out of tolerance and his in tolerance.  Of course this .001 uncertianty only uses the calipers accuracy and does not take into consideration other factors such as operator repeatability etc. which needs to be added to your budget.

Sorry for the long explanation, but it aggravates me when an inspector cites something and does not back it up with a written requirement.


RE: Measurement data with respect to gage accuracy

Yeah, we do have a pretty good system here. We tend to have very few findings, and they're usually minor.

So the gage's accuracy is the same thing as the gage's uncertainty? If so, then I don't believe the inspector fully understands the concept of uncertainty in measurement (which I have to admit, I'm just trying to learn myself).

And if you go by the inspector's logic, then why bother even having that fourth decimal place on the gage?

And don't apologize for a long explanation. I'm looking for all the info I can get.

RE: Measurement data with respect to gage accuracy

Couldn't an accuracy of .001" equal +/-.0005"?

Believe it if you need it or leave it if you dare. - Robert Hunter

RE: Measurement data with respect to gage accuracy

I would not say that accuracy = uncertainty.  Only that instrument accuracy is part of the uncertainty equation along with other factors such as environment, operator repeatabiliy, and any other factor that may cause a variance in the measurement.  I too am relativley new to all this. I have been out of QC for 15 years and now find myself back in. So I will ask someone here if the may have a better explanation.

"Couldn't an accuracy of .001" equal +/-.0005"?"

On the face of it I would agree.  But then I think accuracy of .001 could also mean to within .001 of the measured reading which would then mean +/- .001.  Again any of the more experienced QC people have any input.

I don't want to hijack the thread so does anyone have any more input on Runnik's original question?


RE: Measurement data with respect to gage accuracy

I was taught to always record data to the precision of the instrument.  Accuracy of the measurement is considered in a seperate step, e.g. when you present the data (how big should my error bars be) or when comparing your number to a standard.  In the case of inspection, one should always try to use an instrument with a precision much greater than the tolerance range (10x if possible, 3x minimum), so that discussions of the accuracy of the tools becomes a non-issue, or at least one that doesn't come up very often.  When it does come up, you check the caliper measurement (1.0095 vs. 1.010 limit) using a more precise tool, like a micrometer or optical comparator.  Or, kick it over to MRB/customer to let them decide, then go back and fix your process.


RE: Measurement data with respect to gage accuracy

"Couldn't an accuracy of .001" equal +/-.0005"?"

It could, but my original post meant to say ±.001". Sorry for the confusion.

edpaq - I wasn't implying that the accuracy reflected the total measurement uncertainty. I was just clarifying that the gage accuracy is equal to the gage uncertainty.

btrueblood - Do you mean the resolution of the instrument? Precision refers to the repeatability and reproducibility of an instrument, which isn't known unless you do a GR&R study. I  think the accuracy of the tools would be an issue quite often here, as we see dimensions at the extremes of our tolerance limits quite a bit (which is a whole other issue).

RE: Measurement data with respect to gage accuracy

Sorry, yes, resolution.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members! Already a Member? Login


Low-Volume Rapid Injection Molding With 3D Printed Molds
Learn methods and guidelines for using stereolithography (SLA) 3D printed molds in the injection molding process to lower costs and lead time. Discover how this hybrid manufacturing process enables on-demand mold fabrication to quickly produce small batches of thermoplastic parts. Download Now
Design for Additive Manufacturing (DfAM)
Examine how the principles of DfAM upend many of the long-standing rules around manufacturability - allowing engineers and designers to place a part’s function at the center of their design considerations. Download Now
Taking Control of Engineering Documents
This ebook covers tips for creating and managing workflows, security best practices and protection of intellectual property, Cloud vs. on-premise software solutions, CAD file management, compliance, and more. Download Now

Close Box

Join Eng-Tips® Today!

Join your peers on the Internet's largest technical engineering professional community.
It's easy to join and it's free.

Here's Why Members Love Eng-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close