Mathematical Sciences / Linear Models, Regression and Multivariate Analysis
/ Topic 7 of 7 · 7/7 Linear Models and Multivariate Analysis Formula Sheet and Short Notes One-page CSIR NET revision: Gauss–Markov and BLUE, F tests, ANOVA and EMS, regression diagnostics, multivariate normal, Wishart, correlation, PCA and LDA.
7 min read Updated 16 Nov 2026
Share
Copy link
Print · PDF
Save Share this topic
csirnetnotes.com/mathematical-sciences/linear-models-multivariate-analysis/quick-revision
Copy link Ctrl C Pre-written message Linear Models and Multivariate Analysis Formula Sheet and Short Notes — notes, formulas and practice, free. csirnetnotes.com/mathematical-sciences/linear-models-multivariate-analysis/quick-revision
On this page · 9 sections▾ The paper in one table Gauss–Markov model and linear hypotheses ANOVA and ANCOVA Regression Multivariate normal and quadratic forms Partial and multiple correlation PCA, discriminant, cluster, canonical Traps in one table Where to go next Every formula, rule and trap from Gauss–Markov Models, BLUE and Linear Hypotheses , ANOVA, ANCOVA and Fixed, Random and Mixed Effects , Simple, Multiple and Logistic Regression , Multivariate Normal, Wishart and Quadratic Forms , Partial and Multiple Correlation and PCA, Discriminant, Cluster and Canonical Correlation Analysis , on one page.
The paper in one table Per the NTA information bulletin for December 2026 (Mathematical Sciences, code 704):
Part Given Attempt Correct Wrong Maximum B 40 25 +3 −0.75 75 C 60 20 +4.75 0 95
Part C: one or more options may be correct; marks only if you mark all correct options and no wrong one. No partial credit, no negative marking. Details on the Mathematical Sciences overview .
Gauss–Markov model and linear hypotheses $$Y = X\beta + \varepsilon, \quad E\varepsilon = 0, \quad \operatorname{Var}\varepsilon = \sigma^2I, \quad \operatorname{rank}X = r.$$
Item Fact Normal equations \( X^TX\beta = X^TY \), always consistent Hat matrix \( P = X(X^TX)^-X^T \): symmetric, idempotent, \( \operatorname{tr} = r \), g-inverse free Estimable \( \lambda^T\beta \) \( \lambda^T \in \) row space of \( X \); \( r \) independent ones One-way effects \( \mu + \alpha_i \) and contrasts (\( \sum c_i = 0 \)) estimable; \( \mu \), \( \alpha_i \) alone not BLUE \( \lambda^T\hat\beta \), \( \operatorname{Var} = \sigma^2\lambda^T(X^TX)^-\lambda \); needs no normality GLS \( (X^TV^{-1}X)^{-1}X^TV^{-1}Y \) if \( \operatorname{Var} = \sigma^2V \) \( \sigma^2 \) \( s^2 = \frac{Y^T(I - P)Y}{n - r} \) unbiased; MLE divides by \( n \) Normal theory \( \text{SSE}/\sigma^2 \sim \chi^2_{n - r} \), independent of \( \hat Y \) F test, \( H_0: L\beta = c \) \( \frac{(\text{SSE}_H - \text{SSE})/q}{\text{SSE}/(n - r)} \sim F_{q, n - r} \)
ANOVA and ANCOVA Design df one-way, \( k \) groups, \( N \) obs treatments \( k - 1 \), error \( N - k \) two-way, one per cell A \( a - 1 \), B \( b - 1 \), error \( (a - 1)(b - 1) \) two-way, \( n \) per cell A \( a - 1 \), B \( b - 1 \), AB \( (a - 1)(b - 1) \), error \( ab(n - 1) \) ANCOVA one-way error \( N - k - 1 \)
Model F denominators all fixed MSE for A, B, AB all random A and B over MSAB; AB over MSE A fixed, B random (restricted) A over MSAB; B over MSE; AB over MSE A fixed, B random (unrestricted) A over MSAB; B over MSAB; AB over MSE
Random one-way: \( E(\text{MSB}) = \sigma^2 + n\sigma_a^2 \); \( \hat\sigma_a^2 = \frac{\text{MSB} - \text{MSE}}{n} \) (can be negative); intraclass \( \rho_I = \frac{\sigma_a^2}{\sigma_a^2 + \sigma^2} \). ANCOVA: \( \hat\beta = \frac{E_{xy}}{E_{xx}} \); adjusted error SS \( E_{yy} - \frac{E_{xy}^2}{E_{xx}} \); adjusted mean \( \bar y_i - \hat\beta(\bar x_i - \bar x) \); assumes equal slopes and a covariate unaffected by treatment. Regression Item Fact Simple \( \hat\beta_1 = \frac{S_{xy}}{S_{xx}} \), \( \hat\beta_0 = \bar y - \hat\beta_1\bar x \), \( \operatorname{Var}\hat\beta_1 = \frac{\sigma^2}{S_{xx}} \), df \( n - 2 \) \( R^2 \) (simple) \( r^2 \); slope \( t^2 = F \) Intervals at \( x_0 \) mean: \( s\sqrt{\frac{1}{n} + \frac{(x_0 - \bar x)^2}{S_{xx}}} \); new obs: add 1 under the root Multiple \( \hat\beta = (X^TX)^{-1}X^TY \), df \( n - p \) (\( p \) includes intercept) Adjusted \( R^2 \) \( 1 - (1 - R^2)\frac{n - 1}{n - p} \) Overall F \( \frac{R^2/(p - 1)}{(1 - R^2)/(n - p)} \) Leverage \( \sum h_{ii} = p \), \( \frac{1}{n} \le h_{ii} \le 1 \), flag \( > \frac{2p}{n} \); simple: \( \frac{1}{n} + \frac{(x_i - \bar x)^2}{S_{xx}} \) Residuals \( \operatorname{Var}e_i = \sigma^2(1 - h_{ii}) \); correlated; sum to 0 with intercept Cook’s distance \( \frac{r_i^2}{p}\cdot\frac{h_{ii}}{1 - h_{ii}} \) VIF \( \frac{1}{1 - R_j^2} \) Durbin–Watson \( d \approx 2(1 - \hat\rho) \in [0, 4] \) Logistic \( \ln\frac{p}{1 - p} = x^T\beta \); \( e^{\beta_j} \) = odds ratio; MLE by IRLS; score \( X^T(y - \hat p) = 0 \)
Item Fact Definition every \( a^TX \) normal Linear map \( AX + b \sim N(A\mu + b, A\Sigma A^T) \) Independence jointly normal: \( \Sigma_{12} = 0 \iff \) independent Conditional \( N(\mu_1 + \Sigma_{12}\Sigma_{22}^{-1}(x_2 - \mu_2), \; \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}) \) Bivariate mean \( \mu_1 + \rho\frac{\sigma_1}{\sigma_2}(x_2 - \mu_2) \), variance \( \sigma_1^2(1 - \rho^2) \) Conditional independence \( (\Sigma^{-1})_{ij} = 0 \) Mahalanobis \( (X - \mu)^T\Sigma^{-1}(X - \mu) \sim \chi^2_p \) Mean of a form \( E(X^TAX) = \operatorname{tr}(A\Sigma) + \mu^TA\mu \) (any distribution) Chi-square \( A\Sigma \) idempotent; df \( = \operatorname{rank}A \); non-centrality \( \mu^TA\mu \) (or half, by convention) Independence of forms \( A\Sigma B = 0 \); linear \( LX \) and \( X^TAX \): \( L\Sigma A = 0 \) Cochran \( \sum A_i = I \), ranks add to \( n \) \( \iff \) independent \( \chi^2_{r_i} \) Wishart \( \sum Z_iZ_i^T \sim W_p(n, \Sigma) \); \( E = n\Sigma \); \( (n - 1)S \sim W_p(n - 1, \Sigma) \) Wishart extras \( \frac{a^TWa}{a^T\Sigma a} \sim \chi^2_n \); invertible iff \( n \ge p \); \( E(W^{-1}) = \frac{\Sigma^{-1}}{n - p - 1} \) Hotelling \( \frac{n - p}{p(n - 1)}\, n(\bar X - \mu_0)^TS^{-1}(\bar X - \mu_0) \sim F_{p, n - p} \)
Partial and multiple correlation $$r_{12\cdot3} = \frac{r_{12} - r_{13}r_{23}}{\sqrt{(1 - r_{13}^2)(1 - r_{23}^2)}}, \qquad R^2_{1\cdot23} = \frac{r_{12}^2 + r_{13}^2 - 2r_{12}r_{13}r_{23}}{1 - r_{23}^2}.$$
\( 1 - R^2_{1\cdot23} = (1 - r_{12}^2)(1 - r_{13\cdot2}^2) \). From \( P = R^{-1} \): \( r_{ij\cdot\text{rest}} = -\frac{p^{ij}}{\sqrt{p^{ii}p^{jj}}} \), \( R^2_{i\cdot\text{rest}} = 1 - \frac{1}{p^{ii}} \). \( 0 \le R \le 1 \); \( R_{1\cdot23} \ge \lvert r_{12} \rvert, \lvert r_{13} \rvert \); \( R = 0 \iff r_{12} = r_{13} = 0 \). Feasible iff \( 1 - r_{12}^2 - r_{13}^2 - r_{23}^2 + 2r_{12}r_{13}r_{23} \ge 0 \). Equicorrelation \( \rho \): \( r_{12\cdot3} = \frac{\rho}{1 + \rho} \), \( R^2_{1\cdot23} = \frac{2\rho^2}{1 + \rho} \). Tests: partial of order \( k \): \( t = \frac{r\sqrt{n - 2 - k}}{\sqrt{1 - r^2}} \sim t_{n - 2 - k} \); multiple with \( k \) predictors: \( F_{k, n - k - 1} \). PCA, discriminant, cluster, canonical Technique Key fact PCA \( Y_j = e_j^T(X - \mu) \), \( \operatorname{Var} = \lambda_j \); share \( \frac{\lambda_j}{\operatorname{tr}\Sigma} \); depends on units \( 2 \times 2 \) correlation eigenvalues \( 1 \pm \rho \), PC1 \( \propto Z_1 + Z_2 \) Equicorrelation \( 1 + (p - 1)\rho \) and \( 1 - \rho \) (\( p - 1 \) times) Fisher LDF \( a = \Sigma^{-1}(\mu_1 - \mu_2) \); to \( \pi_1 \) if \( a^Tx \ge \frac{1}{2}a^T(\mu_1 + \mu_2) \) Errors \( \Phi(-\Delta/2) \), \( \Delta^2 = (\mu_1 - \mu_2)^T\Sigma^{-1}(\mu_1 - \mu_2) \) Priors, costs cut-off shifts by \( \ln\frac{c(1 \mid 2)p_2}{c(2 \mid 1)p_1} \) Several groups eigenvectors of \( W^{-1}B \), at most \( \min(g - 1, p) \) Linkages single min (chaining), complete max (compact), average mean, Ward WSS k-means minimises WSS, local optimum, \( k \) fixed in advance Canonical \( \rho_k^2 \) = eigenvalues of \( \Sigma_{11}^{-1}\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} \); \( \min(p, q) \) pairs; invariant within sets \( q = 1 \) canonical correlation = multiple correlation
Traps in one table Statement Verdict Gauss–Markov needs normal errors false Every \( \beta_j \) is estimable when \( r < p \) false \( s^2 = \text{SSE}/(n - p) \) in a rank-deficient model false: \( n - r \) Residuals are independent with variance \( \sigma^2 \) false Adding a predictor can lower \( R^2 \) false (adjusted \( R^2 \) can fall) \( e^{\beta_j} \) in logistic regression is a ratio of probabilities false: odds ratio Normal and uncorrelated implies independent false without joint normality \( \Sigma_{13} = 0 \) implies \( X_1 \perp X_3 \mid X_2 \) false: look at \( \Sigma^{-1} \) \( \hat\sigma_a^2 \) is always non-negative false Fixed factor crossed with a random one is tested against MSE false: MSAB PCA is unchanged by rescaling variables false (true for canonical correlations) \( R_{1\cdot23} \) can be smaller than \( \lvert r_{12} \rvert \) false
Where to go next Advertisement Advertisement