Lesson 5 · 30 min

Calibration curves and detection limits

Instruments give signals; concentrations come from a calibration line. Least-squares fitting, the unknown’s confidence interval, residual plots, detection and quantitation limits — and standard addition to beat matrix effects.

In this lesson

Fit a calibration line by least squares and give the uncertainties of its slope and intercept
Find an unknown concentration and its confidence interval from the calibration line
Check linearity with a residual plot and calculate the detection (3s/m) and quantitation (10s/m) limits
Recognise matrix effects and get the right answer by standard addition

A water report says “lead: not detected”. That does not mean there is not a single lead atom; it means the concentration is below the smallest amount the method can reliably tell apart from nothing — its detection limit. Spectrophotometers, atomic-absorption instruments and chromatographs do not report concentrations; they report signals, and a calibration curve built from standards of known concentration turns signal into concentration. This lesson covers how to fit that line, how to give an unknown’s concentration with its uncertainty, and how small a signal can be before it stops meaning anything.

Predict first

A calibration uses six standards from 0 to 5 mg/L. Where must the unknown fall for its concentration, read from the line, to be most precise?

Pick an answer and lock it in before you look. A wrong guess teaches more than no guess.

Near the lowest standard Near the mean of the standards (the middle of the line) Near the highest standard It makes no difference

The least-squares line

Least squares chooses the line $y = mx + b$ that minimises the sum of the squared vertical deviations (the residuals). It assumes that the concentrations x are exact, that all the error is in the signals y, and that it is the same size for every point. Besides the slope and intercept it gives three uncertainties:

s_y = \sqrt{\frac{\sum (y_i - m x_i - b)^2}{n-2}},\quad s_m = \frac{s_y}{\sqrt{\sum (x_i-\bar{x})^2}},\quad s_b = s_y\sqrt{\frac{\sum x_i^2}{n\sum (x_i-\bar{x})^2}}

The divisor is n − 2: the two parameters, m and b, each use up one degree of freedom.

$s_y$: standard deviation of the points about the line
$s_m,\ s_b$: standard deviations of the slope and intercept
$n$: number of calibration points

x_0 = \frac{\bar{y}_0 - b}{m},\qquad s_{x_0} = \frac{s_y}{|m|}\sqrt{\frac{1}{k} + \frac{1}{n} + \frac{(\bar{y}_0 - \bar{y})^2}{m^2\sum (x_i-\bar{x})^2}}

The unknown is read k times with mean signal ȳ₀; its confidence interval is x₀ ± t·sₓ₀, with n − 2 degrees of freedom for t.

$1/k$: scatter of the unknown’s own readings: shrinks with more readings
$1/n$: uncertainty in where the whole line sits
$(\bar{y}_0 - \bar{y})^2$: grows the further you are from the centre of the calibration

Worked example

A four-point calibration

Four standards: (1, 2), (3, 3), (4, 4), (6, 5). Find the least-squares line and its uncertainties; an unknown read once gives y = 3.50 — find its x with a 95 % confidence interval.

Residuals, detection and quantitation limits

After fitting, always look at the residual plot: the residuals should scatter randomly about zero. An arc (negative at both ends and positive in the middle, or the reverse) means the response has curved — stray light in a spectrophotometer reading low at high absorbance, say — and you should shorten the linear range rather than trust a handsome R².

When a signal is close to the blank, ask whether it can be told apart from the blank at all. Measure the blank (or a very low sample) many times and find its standard deviation s:

detection limit LOD = 3s/m: a signal 3s above the blank has only about a 1 % chance of being a false “detected”;
quantitation limit LOQ = 10s/m: above it the relative standard deviation is about 10 %, good enough to report a number.

Results between the two can only be reported as “detected, below the quantitation limit”.

Matrix effects and standard addition

Everything in a sample except the analyte — salts, organic matter, acidity, viscosity — is the matrix. The matrix can suppress or enhance the signal, so the same concentration gives a different slope in the sample than in pure solvent, and an external calibration with standards in pure solvent comes out systematically low or high. Standard addition spikes known amounts of standard into the sample itself, so the calibration happens in the sample’s own matrix: plot the signal against the amount added, extrapolate the line to zero signal, and the magnitude of the x-intercept, $x = b/m$ , is the unknown’s concentration.

External calibration against standard addition · extrapolating to the x-intercept

Standard addition: beating the matrix effect

Matrix effect (slope in sample / slope in pure solvent) · 0.65 Reading noise σ · 0.004 Show the true concentration

External: standards in pure solvent
Standard addition: spiked into the sample

External-calibration result: 2.02 mg/L
Standard-addition result (95 %): 3.09 ± 0.43 mg/L

The matrix cuts the sensitivity to 65 %: the external calibration comes out about a third low, while the standard-addition extrapolation still gives the right concentration. Set the matrix factor to 1.00 (no matrix effect) and the two methods agree again; then raise the noise and watch the extrapolation’s uncertainty grow quickly.

Concept check

The matrix of a river-water sample reduces the analyte’s signal to 70 % of that in pure solvent. What happens if you calibrate with standards in pure solvent?

The result is right; the matrix only affects precision The result is about 30 % low The result is about 30 % high No measurement is possible

Try the calculation

Ten readings of a reagent blank give an absorbance standard deviation of 0.0021; the calibration slope is 0.1988 L/mg. What is the detection limit in mg/L?

Key takeaways

The least-squares line gives m ± sₘ and b ± s_b; residuals should be random, and a curve means the linear range has been exceeded.
The unknown’s uncertainty sₓ₀ is smallest at the centre of the calibration and shrinks with more readings (k); the interval is x₀ ± t·sₓ₀ (ν = n − 2).
Detection limit LOD = 3s/m and quantitation limit LOQ = 10s/m, with s the blank’s standard deviation.
When the matrix changes the sensitivity, external calibration is biased; standard addition calibrates in the sample’s own matrix and reads the concentration from the x-intercept.

Progress is stored only in this browser and never uploaded.

Data and references

D. C. Harris, Quantitative Chemical Analysis, 9th ed., W. H. Freeman (2016), chapter 4 (method of least squares, calibration curves) and chapter 5 (detection limits, standard addition).
L. A. Currie, “Nomenclature in evaluation of analytical methods including detection and quantification capabilities (IUPAC Recommendations 1995)”, Pure and Applied Chemistry 67, 1699–1723 (1995).
J. N. Miller, J. C. Miller, Statistics and Chemometrics for Analytical Chemistry, 6th ed., Pearson (2010), chapter 5.