38  Calibration Curves

A strip reader scans the test line of a rapid antigen test and reports a number, for example 0.50. How much antigen was in the sample? The reader measures light, not antigen, so the number alone does not give the concentration. Converting one into the other requires a calibration curve, measured on samples of known concentration.

Analyte, sensor, and detector

A rapid antigen test is an assay, a procedure that determines what a substance is or how much of it a sample contains. The analyte is the substance to be measured, here the antigen (Figure 38.1). A reagent is a substance added to react with the analyte. The reagent that recognizes the analyte is the sensor: in the strip, the gold-labeled detection antibody, which binds the antigen and carries a red label to the test line. The detector measures the label’s signal: the eye, or an optical reader that images the test line (Koczula and Gallotta 2016).

Four boxes joined by arrows from left to right: analyte (antigen in the sample), sensor (gold-labeled antibody), detector (eye or optical reader), and calibration curve (signal to concentration), which outputs a concentration in ng/mL.
Figure 38.1: The measurement chain of an assay. The sensor, a labeled antibody, binds the analyte, and its label produces a signal, here light; the detector turns the signal into a number \(S\); a calibration curve turns \(S\) into a concentration \(c\).

The detector reports a signal, not a concentration

Suppose the detector counts 10,000 photons from the test line. The count depends on the amount of analyte, and also on how much label each antibody carries, how long the detector collects light, and what fraction of that light it registers. Doubling the collection time doubles the count from the same sample. A detector’s output, written here in arbitrary units (AU), therefore has no fixed relation to concentration.

The relation is found by calibration. The assay is run on standards, samples with known concentrations of the analyte, and the calibration curve is the relation between their signals and concentrations (International Council for Harmonisation (ICH) 2022). It holds only for the reagents and detector settings it was measured with.

The calibration curve

Seven antigen standards, from a blank (0 ng/mL) to 80 ng/mL, give the signals in Table 38.1 (simulated for this illustration).

Table 38.1: Calibration standards: known concentration \(c\) and signal \(S\), simulated from the line \(S = 0.040 + 0.012\,c\) plus random scatter with a standard deviation of 0.03 AU.
\(c\) (ng/mL) 0 5 10 20 40 60 80
\(S\) (AU) 0.050 0.125 0.170 0.241 0.547 0.773 0.984

The signal rises with concentration, and the points lie close to a straight line, so over this range the calibration is modeled as

\[S = a + b\,c ,\]

where the intercept \(a\) (AU) is the background signal of a sample with no analyte and the slope \(b\) (AU per ng/mL) is the signal gained per unit of concentration. If every count, background included, is proportional to the detector’s collection time, changing that time multiplies \(a\) and \(b\) by the same factor.

No line passes through all seven points. The residual of a standard is its measured signal minus the signal the line predicts (Çetinkaya-Rundel and Hardin 2024),

\[e_i = S_i - (a + b\,c_i) .\]

The least-squares line makes the sum of squared residuals, \(\text{SSE} = \sum_i e_i^2\), as small as possible (Illowsky and Dean 2023). Squaring counts every residual as positive and weights large misses most. The slope and intercept of this line are

\[b = \frac{\sum_i (c_i - \bar c)(S_i - \bar S)}{\sum_i (c_i - \bar c)^2} , \qquad a = \bar S - b\,\bar c ,\]

where \(\bar c\) and \(\bar S\) are the mean concentration and mean signal of the standards (Illowsky and Dean 2023). For Table 38.1, \(b = 0.0118\) AU per ng/mL and \(a = 0.049\) AU. The slope differs from the simulating value 0.012 by 0.0002 AU per ng/mL and the intercept from 0.040 by 0.009 AU, because of the scatter.

\(\text{SSE}(a, b) = \sum_i (S_i - a - b\,c_i)^2\) is smallest where its slope is zero in each parameter. The partial derivative of SSE with respect to \(a\), found with \(b\) held constant, is written \(\partial\,\text{SSE}/\partial a\) (partial derivatives), and likewise for \(b\):

\[\frac{\partial\,\text{SSE}}{\partial a} = -2\sum_i (S_i - a - b\,c_i) = 0 \quad\Rightarrow\quad \bar S = a + b\,\bar c ,\]

the same step that makes the mean the least-squares center of a dataset (numerical data). Substituting \(a = \bar S - b\,\bar c\),

\[\frac{\partial\,\text{SSE}}{\partial b} = -2\sum_i c_i\big[(S_i - \bar S) - b\,(c_i - \bar c)\big] = 0 .\]

For \(n\) standards, deviations from a mean sum to zero, \(\sum_i (c_i - \bar c) = \sum_i c_i - n\,\bar c = 0\), and likewise for \(S_i - \bar S\). The bracket therefore sums to zero, so adding or subtracting a constant times its sum does not change the condition on \(b\). For the constant \(\bar c\),

\[\bar c\sum_i \big[(S_i - \bar S) - b\,(c_i - \bar c)\big] = \bar c\Big[\sum_i (S_i - \bar S) - b\sum_i (c_i - \bar c)\Big] = 0 .\]

Dividing the condition on \(b\) by \(-2\) and subtracting this zero replaces \(c_i\) by \(c_i - \bar c\) in front of the bracket:

\[\sum_i (c_i - \bar c)\big[(S_i - \bar S) - b\,(c_i - \bar c)\big] = \sum_i (c_i - \bar c)(S_i - \bar S) - b\sum_i (c_i - \bar c)^2 = 0 ,\]

which solves to the formula for \(b\) above.

Judging the fit

The coefficient of determination compares the scatter left about the line with the total scatter of the signals about their mean, \(\text{SST} = \sum_i (S_i - \bar S)^2\):

\[R^2 = 1 - \frac{\text{SSE}}{\text{SST}} .\]

\(R^2\) is the fraction of the variation in the signal that the line explains, from 0 (none of it) to 1 (every point on the line) (Çetinkaya-Rundel and Hardin 2024). For Table 38.1, \(\text{SSE} = 0.0032\ \text{AU}^2\) and \(\text{SST} = 0.777\ \text{AU}^2\), so \(R^2 = 0.996\).

A high \(R^2\) does not prove that a straight line is the right model. The residuals should also scatter about zero with no pattern, as they do in Figure 38.2 (right). Residuals that are negative at both ends and positive in the middle reveal a curved response, and a straight line should not be fitted over that range (Çetinkaya-Rundel and Hardin 2024).

Reading an unknown

A sample of unknown concentration is run on the same assay, and its signal \(S^*\) is converted by solving \(S^* = a + b\,\hat c\) for the estimated concentration:

\[\hat c = \frac{S^* - a}{b} .\]

An unknown that reads \(S^* = 0.500\) AU gives \(\hat c = (0.500 - 0.049)/0.0118 = 38\) ng/mL. The slope sets how finely the assay resolves concentration. An error \(\Delta S\) in a reading becomes an error \(\Delta S/b\) in \(\hat c\), so a steeper line gives a smaller concentration error for the same signal noise. The fit, the residuals, and the reading take a few lines of code:

import numpy as np

# seven standards: known concentration c (ng/mL) and measured signal S (AU)
c = np.array([0, 5, 10, 20, 40, 60, 80])
S = np.array([0.050, 0.125, 0.170, 0.241, 0.547, 0.773, 0.984])

# least-squares slope and intercept
b = np.sum((c - c.mean()) * (S - S.mean())) / np.sum((c - c.mean())**2)
a = S.mean() - b * c.mean()

# residuals and the coefficient of determination
e = S - (a + b * c)
R2 = 1 - np.sum(e**2) / np.sum((S - S.mean())**2)

# read an unknown sample
S_star = 0.500
c_hat = (S_star - a) / b

print(f"b = {b:.4f} AU per ng/mL,  a = {a:.4f} AU,  R^2 = {R2:.3f}")
print("residuals (AU):", np.round(e, 3))
print(f"unknown: S = {S_star:.3f} AU  ->  c = {c_hat:.1f} ng/mL")
b = 0.0118 AU per ng/mL,  a = 0.0493 AU,  R^2 = 0.996
residuals (AU): [ 0.001  0.017  0.002 -0.045  0.024  0.013 -0.012]
unknown: S = 0.500 AU  ->  c = 38.1 ng/mL
Left: seven blue points lie close to a rising gold line of signal against concentration from 0 to 80 ng/mL, and a dashed guide converts a reading of 0.50 AU to 38 ng/mL. Right: the seven residuals, all within 0.05 AU of zero, scatter above and below it with no trend.
Figure 38.2: Calibration with the standards of Table 38.1. Left: the least-squares line (gold) through the standards (blue); the unknown reading of 0.500 AU (vermilion) converts to 38 ng/mL. Right: the residuals scatter about zero with no trend.

The working range

The standards of Table 38.1 span 0 to 80 ng/mL. This span, from the lowest to the highest standard, is the calibrated range (International Council for Harmonisation (ICH) 2022). Converting a signal outside it is extrapolation, which assumes the line continues where it was never measured (Çetinkaya-Rundel and Hardin 2024). The working range is the part of the calibrated range where the line still fits: its upper end is the highest standard or the point where the response stops being linear, whichever is lower, and its lower end is set by the noise of a blank.

Above about 100 ng/mL the response bends below the line and levels off (Figure 38.3) as the assay enters the capture-limited regime (Case B, defined in antibody binding), in which the capture antibody is the scarce reagent and caps the signal. An assay with capture antibody in excess (Case A, defined there) is not capped this way; its signal levels off instead when the antigen uses up the gold-labeled antibody, the onset of the hook effect described below. Binding assays calibrated up to this limit are usually fitted with a curved model instead of a line (International Council for Harmonisation (ICH) 2022). At higher concentration the signal of a sandwich test can fall. This is the high-dose hook effect (Ross et al. 2020), explained in antibody binding. On the curve of Figure 38.3, samples of 106 and 405 ng/mL then give the same signal, 1.30 AU, and the line reports both as 106 ng/mL (Table 38.2).

Signal against concentration from 0 to 600 ng/mL: points follow a straight line inside a shaded band from 0 to 80 ng/mL, fall below the dashed extension of the line above about 100 ng/mL, peak near 200 ng/mL, and decline at higher concentration, so a dotted line at 1.30 AU crosses the curve at 106 and 405 ng/mL.
Figure 38.3: Illustrative simulated response (grey curve) and standards (blue) up to 540 ng/mL for the same assay. Inside the calibrated range (shaded) the standards follow the line. Above it the response saturates and then falls (the hook effect), so the extrapolated line (dashed) overestimates the signal and a reading of 1.30 AU corresponds to two concentrations.
Table 38.2: Two samples with the same signal, 1.30 AU, on the curve of Figure 38.3, converted with the line fitted to Table 38.1.
Sample concentration (ng/mL) Concentration the line reports (ng/mL)
106 106
405 106

A signal above the highest standard is not extrapolated. The sample is diluted by a known factor into the calibrated range and measured again, and the result is multiplied by the dilution factor (International Council for Harmonisation (ICH) 2022). A sample far above the calibrated range can also read inside it: the 540 ng/mL standard in Figure 38.3 gives 0.83 AU, which the line reports as 66 ng/mL. Dilution does not catch this case, because the reading is already inside the range. Method validation therefore dilutes samples above the calibrated range to confirm that the hook effect does not suppress their readings (International Council for Harmonisation (ICH) 2022). At the low end, the signal approaches the background \(a\) and is lost in the scatter of blank readings, which sets the lowest detectable concentration (limit of detection).

Exercises

TipProblem 1

Four standards give the following signals.

\(c\) (ng/mL) 0 20 40 60
\(S\) (AU) 0.06 0.28 0.55 0.79
  1. Compute the least-squares slope \(b\) and intercept \(a\).
  2. Compute the residuals, SSE, SST, and \(R^2\).
  3. An unknown reads \(S^* = 0.40\) AU. Estimate its concentration.
  4. Repeated readings of one sample have a standard deviation of 0.010 AU. What is the standard deviation of \(\hat c\)?

\(\bar c = 30\) ng/mL and \(\bar S = 1.68/4 = 0.42\) AU.

  1. The deviations are \(c_i - \bar c = (-30, -10, 10, 30)\) and \(S_i - \bar S = (-0.36, -0.14, 0.13, 0.37)\). The numerator is \(10.8 + 1.4 + 1.3 + 11.1 = 24.6\) and the denominator is \(900 + 100 + 100 + 900 = 2000\), so \(b = 24.6/2000 = \mathbf{0.0123}\) AU per ng/mL and \(a = 0.42 - 0.0123 \times 30 = \mathbf{0.051}\) AU.

  2. The line predicts 0.051, 0.297, 0.543, and 0.789 AU, so the residuals are 0.009, −0.017, 0.007, and 0.001 AU. \(\text{SSE} = 0.00042\ \text{AU}^2\) and \(\text{SST} = 0.303\ \text{AU}^2\), so \(R^2 = 1 - 0.00042/0.303 = \mathbf{0.9986}\).

  3. \(\hat c = (0.40 - 0.051)/0.0123 = \mathbf{28.4}\) ng/mL.

  4. An error \(\Delta S\) in a reading becomes \(\Delta S/b\) in \(\hat c\), so the standard deviation of \(\hat c\) is \(0.010/0.0123 = \mathbf{0.81}\) ng/mL.

TipProblem 2

A strip reader counts photons from the test line during a 0.5 s exposure. Its calibration over 0 to 50 ng/mL is \(N = 400 + 120\,c\), with \(N\) in counts and \(c\) in ng/mL. Assume every count, background included, is proportional to the exposure time.

  1. A sample gives 3400 counts. What is its concentration?
  2. A technician doubles the exposure to 1.0 s, measures the same sample, and reads the result with the old line. What concentration is reported?
  3. What calibration line applies at 1.0 s?
  1. \(\hat c = (3400 - 400)/120 = \mathbf{25}\) ng/mL.

  2. The count doubles to 6800, and the old line gives \((6800 - 400)/120 = \mathbf{53.3}\) ng/mL, more than twice the correct 25 ng/mL and above the calibrated range.

  3. Every count is proportional to the exposure, so the intercept and the slope both double: \(N = \mathbf{800 + 240\,c}\), which returns \((6800 - 800)/240 = 25\) ng/mL.

TipProblem 3

A student calibrates an assay with seven standards.

\(c\) (ng/mL) 0 25 50 75 100 150 200
\(S\) (AU) 0.05 0.30 0.56 0.80 1.02 1.32 1.45
  1. Fit a line to all seven standards (adapt the code above). Report \(a\), \(b\), \(R^2\), and the residuals.
  2. Describe the pattern of the residuals. What does it show?
  3. Refit using only the standards from 0 to 100 ng/mL. Read an unknown at \(S^* = 0.90\) AU with both lines.
  1. \(a = 0.170\) AU, \(b = 0.00718\) AU per ng/mL, and \(R^2 = 0.954\). The residuals are −0.120, −0.050, 0.031, 0.091, 0.132, 0.073, and −0.156 AU.

  2. The residuals are negative at both ends and positive in the middle, so the response is curved: it bends over above about 100 ng/mL. A straight line is the wrong model from 0 to 200 ng/mL, even with \(R^2 = 0.954\).

  3. The 0 to 100 ng/mL line has \(a = 0.058\) AU, \(b = 0.00976\) AU per ng/mL, and \(R^2 = 0.999\), with no residual larger than 0.014 AU. It gives \(\hat c = (0.90 - 0.058)/0.00976 = \mathbf{86}\) ng/mL. The seven-point line gives \((0.90 - 0.170)/0.00718 = \mathbf{102}\) ng/mL, about 18% too high.

TipProblem 4

The calibration is the line \(S = 0.04 + 0.012\,c\) used to simulate the standards of Table 38.1, which span 0 to 80 ng/mL (\(S\) in AU, \(c\) in ng/mL).

  1. A sample reads 1.30 AU. Invert the line. Why should the result not be reported?
  2. The sample is diluted fourfold (to one quarter of its concentration) and reads 0.34 AU. Find the concentration of the original sample.
  3. Samples from a second group of patients are expected to contain 400 to 600 ng/mL. Of a fivefold, tenfold, or twentyfold dilution, which is the least dilution that places all of them inside the calibrated range?
  1. \(\hat c = (1.30 - 0.04)/0.012 = 105\) ng/mL. This lies above the highest standard, so it is an extrapolation into a region where the response may have bent away from the line (Figure 38.3). The line fitted to Table 38.1 gives 106 ng/mL (Table 38.2).

  2. The diluted sample gives \((0.34 - 0.04)/0.012 = 25\) ng/mL, inside the range, so the original contained \(4 \times 25 = \mathbf{100}\) ng/mL.

  3. Fivefold gives 80 to 120 ng/mL, partly above the range. Tenfold gives 40 to 60 ng/mL, inside it. Twentyfold (20 to 30 ng/mL) also works but dilutes more than needed.

References

Çetinkaya-Rundel, Mine, and Johanna Hardin. 2024. Introduction to Modern Statistics. 2nd ed. OpenIntro. https://openintro-ims.netlify.app/.
Illowsky, Barbara, and Susan Dean. 2023. Introductory Statistics 2e. OpenStax, Rice University. https://openstax.org/books/introductory-statistics-2e/pages/12-3-the-regression-equation.
International Council for Harmonisation (ICH). 2022. M10 Bioanalytical Method Validation and Study Sample Analysis: Guidance for Industry. Issued by the U.S. Food and Drug Administration, Center for Drug Evaluation and Research (CDER) and Center for Biologics Evaluation and Research (CBER). https://www.fda.gov/media/162903/download.
Koczula, Katarzyna M., and Andrea Gallotta. 2016. “Lateral Flow Assays.” Essays in Biochemistry 60 (1): 111–20. https://doi.org/10.1042/EBC20150012.
Ross, Georgina M. S., Daniel Filippini, Michel W. F. Nielen, and Gert IJ. Salentijn. 2020. “Unraveling the Hook Effect: A Comprehensive Study of High Antigen Concentration Effects in Sandwich Lateral Flow Immunoassays.” Analytical Chemistry 92 (23): 15587–95. https://doi.org/10.1021/acs.analchem.0c03740.