Try it while you read
Open the free Gauge Linearity and Bias tool and click Load sample: it reproduces the case study below.
Theory: is the gauge right everywhere?
A Type 1 study checks the gauge at one point. Many gauges, however, are used over a range: a micrometer from 0 to 25 mm, a CMM over the whole part. The linearity study measures several reference parts across the range and regresses the bias (measurement − reference) on the reference value.
If the slope b is significantly different from zero (p < 0.05), the bias depends on the size: the gauge is not linear. If the slope is zero but the intercept is not, there is a constant offset. The confidence band around the regression line must contain the zero line for the gauge to be accurate everywhere.
Key point
Step by step
- Select 5 references covering the operating range, with values known from a better measurement (e.g. the lab).
- Measure each 12 times in random order.
- Enter one row per reference: reference value, then the measurements.
- Enter the process variation (6σ) or the tolerance as base for the percentages.
- Read the verdict, the regression and the per-reference t-tests.
Case study: a gauge used from 2 to 10 mm
Five references (2, 4, 6, 8, 10 mm) are measured 12 times each (the Load sample data, process variation 0.6).
| Reference | Average bias | p |
|---|---|---|
| 2 | −0.0174 | < 0.001 |
| 4 | −0.0073 | 0.013 |
| 6 | −0.0003 | 0.955 |
| 8 | +0.0098 | 0.022 |
| 10 | +0.0232 | < 0.001 |
- Slope 0.00491 (p < 0.001), R² 62 %: the bias grows with size.
- Average bias 0.0016 (p 0.48): on average the gauge looks fine, which is exactly why a one-point check missed the problem.
The gauge was set with a 6 mm master only. The team recalibrates at 2 and 10 mm as well and repeats the study; until then the gauge is restricted to 5–7 mm.
Common mistakes
- Using references that cover only a small part of the range.
- Judging only the average bias.
- Reference values that are not more accurate than the gauge under test.
Continue with a Gage R&R to check precision with several operators.
Frequently asked questions
What is the difference between linearity and bias?
Bias is the systematic difference between measurement and true value at one point. Linearity is how that bias changes across the range. A gauge can have zero average bias but still be wrong at both ends.
What if only some references show a bias?
Look at the per-reference table: if the bias grows steadily with size, recalibrate at several points; if one point is off, check that reference part.