Continue to Site

Eng-Tips is the largest engineering community on the Internet

Intelligent Work Forums for Engineering Professionals

  • Congratulations cowski on being selected by the Eng-Tips community for having the most helpful posts in the forums last week. Way to Go!

what tolerance

Status
Not open for further replies.

jumacao

Chemical
Dec 26, 2008
41
what instrument tolerance I need if I want to measure a process up to one decimal point?
 
Replies continue below

Recommended for you

Seriously. Not being versed in metrology I took the following fragment from Wikipedia:
A common convention in science and engineering is to express accuracy and/or precision implicitly by means of significant figures. Here, when not explicitly stated, the margin of error is understood to be one-half the value of the last significant place. For instance, a recording of 843.6 m, or 843.0 m, or 800.0 m would imply a margin of 0.05 m (the last significant place is the tenths place), while a recording of 8436 m would imply a margin of error of 0.5 m (the last significant digits are the units).
========================================================
Thus, accuracy can be said to be the 'correctness' of a measurement, while precision could be identified as the ability to resolve smaller differences.

I'd say the instrument should be able to measure down to the second decimal point.
 
The 10:1 test accuracy ratio (TAR) was previously imposed by MIL-STD-45662, but the current thinking per MIL-HDBK-52 is more like 4:1, so 1/4 of the minimum desired resolution is required.

TTFN

FAQ731-376
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor