Try it while you read
Open the free Regression and Correlation tool and click Load sample: it reproduces the case study below.
Theory: a straight line with an uncertainty
Least squares regression finds the coefficients that minimise the squared differences between the measured and the fitted values. What matters for an engineer:
- Coefficient b₁: how much Y changes per unit of X, with its confidence interval.
- p-value of each factor: below 0.05 the effect is unlikely to be chance.
- R²: share of the variation of Y explained by the model. Adjusted R² penalises extra factors.
- s: residual standard deviation in the unit of Y, the typical scatter around the line.
- Residuals: must look random. A curve, a funnel or outliers mean the model is wrong or incomplete.
Correlation (r) measures the strength of a linear relation between two variables from −1 to +1; r² equals R² in a simple regression.
Key point
Step by step
- Data: in the regression tool one row per part: Y first, then up to four X. Click a column name to rename it.
- Scatter plot: check the shape, outliers and the range of X.
- Model: read the equation, the coefficient table with p-values and confidence intervals, R², adjusted R² and s.
- Residuals: the residual plots and the list of unusual points (standardized residual beyond ±2).
- Report: the verdict states the effect per unit of X with its confidence interval; download the PDF with the plots and the ANOVA table.
Case study: hardness after heat treatment
The Load sample: 24 parts with hardness (HRC) after hardening and the furnace temperature of their batch, between 823 and 886 °C.
| Result | Value |
|---|---|
| Equation | Hardness = −148.75 + 0.2086 × Temperature |
| Slope, 95 % confidence interval | 0.184 to 0.234 HRC per °C |
| p-value of temperature | < 0.001 |
| R² / adjusted R² | 93.2 % / 92.9 % |
| s (residual standard deviation) | 0.99 HRC |
| Correlation r | 0.97 |
Each 10 °C adds about 2.1 HRC. At 860 °C the model predicts 30.6 HRC. With s close to 1 HRC, single parts scatter by about ±2 HRC around the line, so a hardness tolerance of ±3 HRC needs the temperature held within roughly ±5 °C around the target. The residuals show no pattern and no unusual points.
Result
Common mistakes
- Reading R² without looking at the scatter plot or the residuals.
- Using the equation outside the tested range.
- Too narrow a range of X, which hides a real effect.
- Many factors on few data points: aim for at least 10 rows per factor.
- Treating correlation as proof of cause.
To test causes on purpose use a factorial design; to compare groups use the hypothesis tests.
Frequently asked questions
What is a good R²?
There is no fixed limit. In a controlled process relation (temperature and hardness) R² above 0.8 is common; in noisy data a significant factor with R² of 0.3 can still be useful. Always read R² together with the residual standard deviation s, which is in the unit of Y.
Does a significant regression prove cause and effect?
No. It shows an association in the data. Cause and effect needs process knowledge or a designed experiment where the factor is changed on purpose.
When should I use multiple regression?
When more than one factor influences the result and they vary in the data at the same time. Watch for factors that are correlated with each other: their coefficients become unstable.