Skip to content

Linear and Nonlinear Curve Fitting and the Chi-Square Test

CSIR NET Physical Sciences: weighted least squares, slope and intercept errors, linearisation, nonlinear fits, chi-square goodness of fit, practice.

  • 21 min read
  • Updated
On this page · 14 sections
  1. Prerequisites
  2. The Hook: Is That Line Any Good?
  3. The Principle of Least Squares
  4. Straight-Line Fit: \( y = mx + c \)
  5. Nonlinear Models
  6. Propagation of Errors
  7. The Chi-Square Statistic and Its Distribution
  8. Memory Tricks
  9. Don’t Confuse
  10. Common Mistakes
  11. Practice
  12. Quick Revision Table
  13. Key Takeaways
  14. What’s Next

Prerequisites

The Hook: Is That Line Any Good?

The Principle of Least Squares

Suppose you measure \( y_i \) at points \( x_i \) (\( i = 1, \ldots, N \)), each \( y_i \) with standard deviation \( \sigma_i \), and the model is \( y = f(x; a_1, \ldots, a_p) \) with \( p \) parameters. If each \( y_i \) is Gaussian about the true \( f(x_i) \), the probability of the observed data is proportional to

$$\prod_i \exp\left[-\frac{(y_i - f(x_i))^2}{2\sigma_i^2}\right] = \exp\left(-\tfrac{1}{2}\chi^2\right), \qquad \chi^2 = \sum_{i=1}^N \frac{\left[y_i - f(x_i; \mathbf a)\right]^2}{\sigma_i^2}.$$

Maximising that likelihood is the same as minimising \( \chi^2 \). That is the whole justification of least squares: it is the maximum-likelihood estimate for Gaussian errors. When every \( \sigma_i \) equals the same \( \sigma \), it reduces to minimising the plain sum of squared residuals.

Straight-Line Fit: \( y = mx + c \)

Equal errors

Setting \( \partial\chi^2/\partial m = \partial\chi^2/\partial c = 0 \) gives the normal equations

$$m\sum x_i^2 + c\sum x_i = \sum x_iy_i, \qquad m\sum x_i + cN = \sum y_i.$$

With \( \Delta = N\sum x_i^2 - \left(\sum x_i\right)^2 \):

$$\boxed{m = \frac{N\sum x_iy_i - \sum x_i\sum y_i}{\Delta}, \qquad c = \frac{\sum x_i^2\sum y_i - \sum x_i\sum x_iy_i}{\Delta} = \bar y - m\bar x}$$

The best line always passes through the centroid \( (\bar x, \bar y) \). The parameter uncertainties follow from propagating \( \sigma \) through these linear formulas:

$$\boxed{\sigma_m^2 = \frac{N\sigma^2}{\Delta}, \qquad \sigma_c^2 = \frac{\sigma^2\sum x_i^2}{\Delta}}$$

If \( \sigma \) is not known in advance, estimate it from the scatter itself, using \( N - 2 \) (not \( N \)) because two parameters were fitted:

$$\sigma^2 \approx \frac{1}{N - 2}\sum_i\left(y_i - mx_i - c\right)^2.$$

Unequal errors: weighted least squares

Give each point weight \( w_i = 1/\sigma_i^2 \) and replace every sum: \( N \to \sum w_i \), \( \sum x_i \to \sum w_ix_i \), and so on. Then \( \sigma_m^2 = \sum w_i/\Delta_w \) and \( \sigma_c^2 = \sum w_ix_i^2/\Delta_w \) with \( \Delta_w = \sum w_i\sum w_ix_i^2 - (\sum w_ix_i)^2 \). A point with half the error counts four times as much.

The simplest weighted fit is a constant, \( y = \mu \): the weighted mean

$$\boxed{\bar x_w = \frac{\sum x_i/\sigma_i^2}{\sum 1/\sigma_i^2}, \qquad \sigma_{\bar x_w} = \left(\sum\frac{1}{\sigma_i^2}\right)^{-1/2}}$$

which is how independent measurements of the same quantity are combined. For \( N \) equal errors it reduces to the familiar \( \sigma/\sqrt N \).

Correlation between slope and intercept

Slope and intercept are generally correlated: \( \text{cov}(m, c) = -\sigma^2\sum x_i/\Delta \). If all \( x_i > 0 \), tilting the line up lowers the intercept, so the covariance is negative. Shifting the origin to \( \bar x \) (fitting \( y = m(x - \bar x) + c' \)) makes the two estimates independent, which is why \( c' = \bar y \) has the simple error \( \sigma/\sqrt N \).

The linear correlation coefficient

$$r = \frac{\sum(x_i - \bar x)(y_i - \bar y)}{\sqrt{\sum(x_i - \bar x)^2\sum(y_i - \bar y)^2}}, \qquad -1 \leq r \leq 1.$$

\( r^2 \) is the fraction of the variance in \( y \) explained by the line. A high \( r \) does not prove the model is right: a gentle parabola sampled over a short range gives \( r > 0.99 \). Judge a fit by its residuals and its \( \chi^2 \), not by \( r \).

Nonlinear Models

Linearisation

Many physics laws become straight lines after a transformation:

LawPlotSlopeIntercept
\( y = Ae^{bx} \) (decay, Arrhenius-type, charging)\( \ln y \) vs \( x \)\( b \)\( \ln A \)
\( y = Ax^n \) (power law, Kepler, scaling)\( \ln y \) vs \( \ln x \)\( n \)\( \ln A \)
\( y = A/(B + x) \)\( 1/y \) vs \( x \)\( 1/A \)\( B/A \)
\( R = R_0e^{E_a/k_BT} \) (thermally activated)\( \ln R \) vs \( 1/T \)\( E_a/k_B \)\( \ln R_0 \)
\( y = a + bx^2 \)\( y \) vs \( x^2 \)\( b \)\( a \)

Polynomial and other linear-in-parameter fits

“Linear least squares” means linear in the parameters, not in \( x \). The model \( y = a_0 + a_1x + a_2x^2 \) or \( y = a\sin x + b\cos x \) is still linear: the normal equations are a \( p \times p \) linear system \( \mathbf{(A^TWA)\,a = A^TWy} \), where \( \mathbf A \) is the design matrix \( A_{ij} = \phi_j(x_i) \) and \( \mathbf W \) the diagonal weight matrix. The parameter covariance matrix is \( (\mathbf{A^TWA})^{-1} \). One exact solve, no iteration.

Truly nonlinear fits

A model such as \( y = A\exp[-(x - x_0)^2/2s^2] + B \) is nonlinear in \( x_0 \) and \( s \). Minimising \( \chi^2 \) then needs iteration from a starting guess:

  • Gauss–Newton: linearise \( f \) about the current parameters, solve the linear least-squares problem for the step, repeat. Fast near the minimum, can overshoot far from it.
  • Gradient (steepest) descent: step down \( -\nabla\chi^2 \). Robust far from the minimum, slow near it.
  • Levenberg–Marquardt: interpolates between the two with a damping parameter; the standard choice in fitting software.

Practical consequences that Part C can test: a nonlinear fit can converge to a local minimum, so starting values matter; it needs at least as many points as parameters (in practice many more); and the curvature of \( \chi^2 \) at the minimum gives the parameter errors.

Reading errors from the chi-square surface

Near its minimum, \( \chi^2 \) is a paraboloid in the parameters. For one parameter \( a \),

$$\chi^2(a) \approx \chi^2_{\min} + \frac{(a - \hat a)^2}{\sigma_a^2}, \qquad \sigma_a^{-2} = \frac{1}{2}\frac{\partial^2\chi^2}{\partial a^2}.$$

So the \( 1\sigma \) interval is where \( \chi^2 \) rises by 1 above its minimum (with the other parameters re-optimised at each step). This “\( \Delta\chi^2 = 1 \)” rule works for linear and nonlinear fits alike.

Propagation of Errors

For a result \( q = f(x, y, \ldots) \) with independent errors,

$$\sigma_q^2 = \left(\frac{\partial f}{\partial x}\right)^2\sigma_x^2 + \left(\frac{\partial f}{\partial y}\right)^2\sigma_y^2 + \cdots$$
FormRule
\( q = x \pm y \)absolute errors add in quadrature
\( q = xy \) or \( x/y \)fractional errors add in quadrature
\( q = x^n \)\( \sigma_q/q = \lvert n\rvert\,\sigma_x/x \)
\( q = \ln x \)\( \sigma_q = \sigma_x/x \)
\( q = e^x \)\( \sigma_q/q = \sigma_x \)

Correlated variables need the covariance term \( 2\frac{\partial f}{\partial x}\frac{\partial f}{\partial y}\text{cov}(x, y) \); this is where the slope–intercept correlation matters if you compute, say, the \( x \)-intercept \( -c/m \).

The Chi-Square Statistic and Its Distribution

If the model is right and the \( \sigma_i \) are right, each term \( (y_i - f_i)/\sigma_i \) is a standard normal variable. After fitting \( p \) parameters from \( N \) points, \( \chi^2 \) follows the chi-square distribution with

$$\boxed{\nu = N - p \text{ degrees of freedom}, \qquad \langle\chi^2\rangle = \nu, \qquad \text{Var}(\chi^2) = 2\nu}$$

so a good fit has \( \chi^2 \approx \nu \pm \sqrt{2\nu} \). The reduced chi-square \( \chi^2_\nu = \chi^2/\nu \) should be close to 1.

\( \chi^2_\nu \)MeaningWhat to check
\( \approx 1 \)Fit consistent with the errorsnothing; quote the parameters
\( \gg 1 \)Scatter larger than the error bars allowwrong model, underestimated \( \sigma_i \), outliers, unaccounted systematic error
\( \ll 1 \)Scatter smaller than the error bars implyoverestimated \( \sigma_i \), too many parameters (over-fitting), correlated or fabricated data

The p-value is the probability that a correct model would give a \( \chi^2 \) at least as large as observed. A p-value below 0.05 (or 0.01) is the usual signal to reject the model. Upper critical values at the 5% level, worth remembering for the first few \( \nu \):

\( \nu \)12345610
\( \chi^2_{0.05} \)3.845.997.819.4911.0712.5918.31

For large \( \nu \) the distribution is close to Gaussian with mean \( \nu \) and standard deviation \( \sqrt{2\nu} \), so \( \chi^2 > \nu + 2\sqrt{2\nu} \) is suspicious.

Pearson’s chi-square test for counts

For data in \( k \) bins with observed counts \( O_j \) and expected counts \( E_j \) (from a hypothesis such as “the die is fair” or “the counts are Poisson”):

$$\boxed{\chi^2 = \sum_{j=1}^k\frac{(O_j - E_j)^2}{E_j}}$$

This is the same statistic with \( \sigma_j^2 = E_j \) (Poisson variance). Degrees of freedom: \( \nu = k - 1 - (\text{number of parameters estimated from the data}) \). The \( -1 \) is for the constraint that the total count is fixed. The test needs expected counts of about 5 or more per bin; merge sparse bins first.

Memory Tricks

Don’t Confuse

ThisvsThat
Linear least squares: linear in the parameters (\( a + bx + cx^2 \) qualifies)Nonlinear: parameters inside functions (\( Ae^{-x/\tau} \) in \( \tau \)); needs iteration
\( \nu = N - p \) for a curve fit\( \nu = k - 1 - p \) for a binned Pearson test
\( \chi^2_\nu \gg 1 \): bad model or errors too small\( \chi^2_\nu \ll 1 \): errors too large or over-fitting
Correlation coefficient \( r \): strength of linear association\( \chi^2 \): agreement with the error bars
Standard deviation of data \( \sigma \)Standard error of the mean \( \sigma/\sqrt N \)
Precision (small random scatter)Accuracy (small systematic offset); a fit cannot remove a systematic error

Common Mistakes

Practice

All questions below are original practice questions written for this page. They are not from any CSIR NET paper.

Level 1: Foundation

  1. A straight line is fitted to 12 data points. How many degrees of freedom does the \( \chi^2 \) have?
  2. Which plot turns \( y = Ax^n \) into a straight line, and what is its slope?
  3. Three measurements of the same quantity have equal errors \( \sigma \). What is the error of their mean?
Answers
  1. 10. \( \nu = N - p = 12 - 2 \).
  2. \( \ln y \) against \( \ln x \), slope \( n \), intercept \( \ln A \).
  3. \( \sigma/\sqrt 3 \). The weighted-mean formula with equal weights.

Level 2: Part B Style

Practice Q1. The least-squares straight line through the points \( (0, 1) \), \( (1, 3) \), \( (2, 4) \), \( (3, 6) \) is:

(a) \( y = 1.5x + 1.2 \) (b) \( y = 1.6x + 1.1 \) (c) \( y = 1.7x + 1.0 \) (d) \( y = 1.6x + 1.0 \)

Answer: (b) y = 1.6x + 1.1

\( N = 4 \), \( \sum x = 6 \), \( \sum y = 14 \), \( \sum x^2 = 14 \), \( \sum xy = 0 + 3 + 8 + 18 = 29 \). \( \Delta = 4(14) - 36 = 20 \).

\( m = (4 \times 29 - 6 \times 14)/20 = 32/20 = 1.6 \). \( c = \bar y - m\bar x = 3.5 - 1.6 \times 1.5 = 1.1 \).

Check: the line passes through the centroid \( (1.5, 3.5) \). Option (d) fails that check.

Practice Q2. A straight line fitted to 20 points with quoted errors gives \( \chi^2 = 45 \). The most reasonable conclusion is:

(a) the fit is excellent (b) the errors are overestimated (c) the errors are underestimated or the model is inadequate (d) more parameters are needed to reduce \( \nu \)

Answer: (c)

\( \nu = 18 \), so \( \chi^2_\nu = 2.5 \). For \( \nu = 18 \), \( \chi^2 \) should lie near \( 18 \pm 6 \); 45 is about 4.5 standard deviations high. Either the error bars are too small or a straight line is the wrong model. Option (b) is the reading for \( \chi^2_\nu \ll 1 \).

Practice Q3. A die is rolled 60 times, giving counts 8, 12, 9, 15, 6, 10 for faces 1 to 6. Testing the hypothesis that the die is fair, the chi-square value and the degrees of freedom are:

(a) 5.0, 5 (b) 5.0, 6 (c) 0.5, 5 (d) 50, 5

Answer: (a) 5.0 and 5

Deviations from \( E = 10 \): \( 8 - 10 = -2 \), \( 12 - 10 = 2 \), \( 9 - 10 = -1 \), \( 15 - 10 = 5 \), \( 6 - 10 = -4 \), \( 10 - 10 = 0 \). Squares: 4, 4, 1, 25, 16, 0, total 50. So \( \chi^2 = 50/10 = 5.0 \).

\( \nu = 6 - 1 = 5 \) (one constraint: the total is fixed at 60). Since \( 5.0 < 11.07 \), the die is consistent with fair. Option (d) forgets to divide by \( E \); option (b) forgets the constraint.

Practice Q4. Values \( y_i \) with constant absolute error \( \sigma \) are to be fitted to \( y = Ae^{-x/\tau} \) by plotting \( \ln y \) against \( x \). The appropriate weight for point \( i \) in the straight-line fit is proportional to:

(a) 1 (b) \( y_i \) (c) \( y_i^2 \) (d) \( 1/y_i^2 \)

Answer: (c) y²

\( \sigma_{\ln y} = \sigma/y \), so \( w = 1/\sigma_{\ln y}^2 = y^2/\sigma^2 \propto y^2 \). The large-\( y \) points, which are the most precise on a log scale, dominate. Option (b) is the weight for Poisson counts (\( \sigma_y = \sqrt y \)).

Level 3: Part C Style

Practice Q5. A short-lived source is counted for equal short intervals starting at \( t = 0, 10, 20, 30 \) min, giving (background-corrected) 1000, 610, 370 and 225 counts. A straight-line fit of \( \ln N \) against \( t \) gives a half-life closest to:

(a) 10 min (b) 14 min (c) 20 min (d) 28 min

Answer: (b) 14 min

\( \ln N = 6.908, 6.413, 5.914, 5.416 \). The successive differences are \( -0.494, -0.500, -0.497 \) per 10 min, so the slope is \( -\lambda \approx -0.0497 \) min\( ^{-1} \) (a full least-squares fit gives \( -0.0497 \)).

\( t_{1/2} = \ln 2/\lambda = 0.693/0.0497 \approx 13.9 \) min.

Two checks: the counts fall by roughly 0.61 every 10 min, and \( 0.61^{1.4} \approx 0.5 \), consistent with 14 min. With counts of order 1000, Poisson errors are about 3%, so the near-perfect line is reasonable; a proper fit would weight each point by \( N_i \).

Practice Q6. Three laboratories report measurements of the same quantity: \( 10.2 \pm 0.2 \), \( 9.8 \pm 0.2 \) and \( 10.5 \pm 0.5 \) (arbitrary units). Consider:

  1. The weighted mean is about 10.04.
  2. The error of the weighted mean is about 0.14.
  3. The \( \chi^2 \) of the three values about the weighted mean is about 2.9 with 2 degrees of freedom, so they are mutually consistent at the 5% level.
  4. The unweighted mean, 10.17, is a better estimate because it treats all laboratories equally.

Which are correct?

(a) 1 and 2 only (b) 1, 2 and 3 only (c) 3 and 4 only (d) 1, 2, 3 and 4

Answer: (b) 1, 2 and 3 only

Weights \( 1/\sigma^2 = 25, 25, 4 \), total 54.

Statement 1: \( \bar x_w = (25 \times 10.2 + 25 \times 9.8 + 4 \times 10.5)/54 = 542/54 = 10.037 \). True.

Statement 2: \( \sigma = 1/\sqrt{54} = 0.136 \). True.

Statement 3: \( \chi^2 = \left(\frac{0.163}{0.2}\right)^2 + \left(\frac{-0.237}{0.2}\right)^2 + \left(\frac{0.463}{0.5}\right)^2 = 0.66 + 1.41 + 0.86 = 2.93 \). One parameter (the mean) was fitted, so \( \nu = 2 \); critical value 5.99. Consistent. True.

Statement 4: the unweighted mean gives the least precise result as much influence as the best ones; it is unbiased but has a larger error. False.

Practice Q7. Five points with \( \sigma = 0.2 \) each are fitted by \( y = mx + c \), giving \( m = 1.97 \pm 0.06 \), \( c = 0.09 \pm 0.21 \), \( \chi^2 = 2.3 \). A student then fits a quartic polynomial (five parameters) to the same five points. Consider:

  1. The quartic passes through every point, so its \( \chi^2 = 0 \).
  2. The quartic has zero degrees of freedom, so its \( \chi^2 \) says nothing about the quality of the model.
  3. The straight-line fit, with \( \chi^2_\nu \approx 0.76 \), is acceptable.
  4. Because \( c \) is within one standard deviation of zero, the data are consistent with a line through the origin.

Which are correct?

(a) 1 and 3 only (b) 1, 2 and 3 only (c) 1, 2, 3 and 4 (d) 2 and 4 only

Answer: (c) 1, 2, 3 and 4

Statement 1: five parameters through five points with distinct \( x \) interpolate exactly. True.

Statement 2: \( \nu = 5 - 5 = 0 \); a zero \( \chi^2 \) with no degrees of freedom carries no information. True. This is over-fitting in its extreme form.

Statement 3: \( \nu = 3 \), \( \chi^2_\nu = 2.3/3 \approx 0.76 \), well inside the expected range \( 3 \pm 2.4 \). True.

Statement 4: \( 0.09/0.21 \approx 0.4\sigma \). True. Refitting \( y = mx \) would then give a slope with a smaller error, since one fewer parameter is estimated.

Quick Revision Table

ItemResult
Least squaresmaximum likelihood for Gaussian errors: minimise \( \chi^2 = \sum(y_i - f_i)^2/\sigma_i^2 \)
Line slope\( m = (N\sum xy - \sum x\sum y)/\Delta \), \( \Delta = N\sum x^2 - (\sum x)^2 \)
Line intercept\( c = \bar y - m\bar x \); line passes through centroid
Errors\( \sigma_m^2 = N\sigma^2/\Delta \), \( \sigma_c^2 = \sigma^2\sum x^2/\Delta \)
\( \sigma \) from scatter\( \sum r_i^2/(N - 2) \)
Weighted mean\( \sum(x_i/\sigma_i^2)/\sum(1/\sigma_i^2) \), error \( (\sum 1/\sigma_i^2)^{-1/2} \)
Linearised log fitweight \( y_i^2/\sigma_{y_i}^2 \)
Degrees of freedom\( N - p \) (fit); \( k - 1 - p \) (binned test)
\( \chi^2 \) distributionmean \( \nu \), variance \( 2\nu \)
5% critical values3.84 (1), 5.99 (2), 7.81 (3), 9.49 (4), 11.07 (5)
Parameter error from surface\( \Delta\chi^2 = 1 \)
Pearson test\( \sum(O - E)^2/E \), need \( E \gtrsim 5 \) per bin

Key Takeaways

What’s Next