Linear Models, Regression and Multivariate Analysis
CSIR NET Mathematical Sciences Unit 4: Gauss–Markov models, ANOVA, regression, multivariate normal and Wishart, correlation, PCA and discriminant analysis.
- 6
- Topics
- 107
- Min read
- 01 Gauss–Markov Models, BLUE and Linear HypothesesStart here CSIR NET statistics: the Gauss–Markov linear model, normal equations, estimability, BLUE, estimating sigma squared and the F test for linear hypotheses.
- 02 ANOVA, ANCOVA and Fixed, Random and Mixed Effects CSIR NET statistics: one-way and two-way ANOVA, expected mean squares, fixed, random and mixed effects models, variance components and ANCOVA.
- 03 Simple, Multiple and Logistic Regression CSIR NET statistics: simple and multiple regression, R-squared, t and F tests, residual diagnostics, leverage, Cook's distance, VIF and logistic regression.
- 04 Multivariate Normal, Wishart and Quadratic Forms CSIR NET statistics: multivariate normal properties, marginals and conditionals, quadratic forms and chi-square, Cochran's theorem, Wishart and Hotelling T².
- 05 Partial and Multiple Correlation CSIR NET statistics: partial and multiple correlation coefficients, three-variable formulas, the inverse correlation matrix, bounds, and t and F tests.
- 06 PCA, Discriminant, Cluster and Canonical Correlation Analysis CSIR NET statistics: principal components, Fisher's linear discriminant, hierarchical and k-means clustering, and canonical correlations, with worked numbers.
- Linear Models and Multivariate Analysis Formula Sheet and Short Notes Revise The whole chapter on one page.
Chapter Overview
This chapter covers the linear-model and multivariate part of Unit 4 (statistics) of the CSIR NET Mathematical Sciences syllabus. It is where statistics meets Linear Algebra: least squares is an orthogonal projection, degrees of freedom are ranks, sums of squares are quadratic forms, and principal components, discriminant directions and canonical correlations are eigenvectors.
The notes assume an MSc course in linear models and multivariate analysis. They do not re-derive every theorem. They collect the exact statements (what Gauss–Markov does and does not assume, when a quadratic form is chi-square, which mean square goes in the F denominator), the small computations Part B rewards, and the counter-examples that settle multiple-correct Part C options.
What You’ll Learn
By the end of this chapter you will be able to:
- Decide whether a parametric function is estimable, write its BLUE and variance, and test a linear hypothesis with the F statistic
- Build one-way and two-way ANOVA tables, choose the right F denominator for fixed, random and mixed effects, estimate variance components, and adjust means by ANCOVA
- Fit simple and multiple regressions, compute \( R^2 \), adjusted \( R^2 \) and t and F tests, read leverage, Cook’s distance, VIF and Durbin–Watson, and interpret a logistic coefficient as an odds ratio
- Use the marginal and conditional distributions of the multivariate normal, decide when quadratic forms are chi-square and independent, and use the Wishart and Hotelling \( T^2 \) distributions
- Compute partial and multiple correlations from simple correlations or from the inverse correlation matrix, and check whether a correlation matrix is possible
- Find principal components, Fisher’s discriminant direction, dendrogram heights and canonical correlations in small cases
Prerequisites
- Linear Algebra, especially Quadratic Forms and Inner Product Spaces and Orthonormal Bases
- Correlation and the two regression lines from Descriptive Statistics and Exploratory Data Analysis
- The normal, chi-square, t and F distributions from a first course in probability
Learning Path
graph TD
A[1. Gauss-Markov Models, BLUE and Linear Hypotheses] --> B[2. ANOVA, ANCOVA and Effects Models]
A --> C[3. Simple, Multiple and Logistic Regression]
A --> D[4. Multivariate Normal, Wishart and Quadratic Forms]
D --> B
C --> E[5. Partial and Multiple Correlation]
D --> E
E --> F[6. PCA, Discriminant, Cluster and Canonical Correlation]
B --> G[Quick Revision]
F --> GPages 1 to 3 are the linear-model track and page 4 supplies the distribution theory behind their F tests. Pages 5 and 6 are the multivariate track. If you prepare Unit 4 selectively, pages 1, 3 and 4 carry the most reusable ideas.
Topics in This Chapter
| # | Topic | What it covers | Time |
|---|---|---|---|
| 1 | Gauss–Markov Models, BLUE and Linear Hypotheses | normal equations, hat matrix, estimability, Gauss–Markov theorem, GLS, \( s^2 \), F test, restricted least squares | 35 min |
| 2 | ANOVA, ANCOVA and Fixed, Random and Mixed Effects | one-way and two-way ANOVA, expected mean squares, variance components, intraclass correlation, ANCOVA adjustment | 35 min |
| 3 | Simple, Multiple and Logistic Regression | slope and intercept, \( R^2 \), adjusted \( R^2 \), t and F tests, leverage, Cook’s distance, VIF, Durbin–Watson, logistic regression | 35 min |
| 4 | Multivariate Normal, Wishart and Quadratic Forms | properties, conditionals, precision matrix, \( E(X^TAX) \), chi-square and independence of forms, Cochran, Wishart, Hotelling \( T^2 \) | 35 min |
| 5 | Partial and Multiple Correlation | three-variable formulas, recursion, inverse correlation matrix, feasibility, tests | 30 min |
| 6 | PCA, Discriminant, Cluster and Canonical Correlation Analysis | principal components, Fisher’s discriminant, linkages and k-means, canonical correlations | 35 min |
| Quick Revision | every formula and trap on one page | 10 min |
Official syllabus lines and where they are covered
| Official topic line | Covered in |
|---|---|
| Gauss-Markov models, BLUE, linear hypotheses | Gauss–Markov Models, BLUE and Linear Hypotheses |
| ANOVA and ANCOVA; fixed, random, mixed effects | ANOVA, ANCOVA and Fixed, Random and Mixed Effects |
| Simple, multiple and logistic regression; diagnostics | Simple, Multiple and Logistic Regression |
| Multivariate normal and Wishart distributions, quadratic forms | Multivariate Normal, Wishart and Quadratic Forms |
| Partial and multiple correlation | Partial and Multiple Correlation |
| PCA, discriminant, cluster, canonical correlation | PCA, Discriminant, Cluster and Canonical Correlation Analysis |
How CSIR NET Asks It
Per the NTA information bulletin for December 2026, the Mathematical Sciences paper (code 704) is marked as follows:
| Part B | Part C | |
|---|---|---|
| Questions given | 40 | 60 |
| Questions to attempt | 25 | 20 |
| Marks per correct answer | 3 | 4.75 |
| Marks per wrong answer | −0.75 | 0 |
| Maximum | 75 | 95 |
NTA publishes no topic-wise weightage, so we make no claim about how many questions come from this chapter. What the format tells you is how to prepare:
- Part B (single correct) rewards short computations: a BLUE and its variance, \( E(\text{SSE}) \), an F ratio or variance component from a small table, a regression slope or adjusted \( R^2 \), a conditional normal mean, \( E(X^TAX) \), a partial correlation, the share of variance of the first principal component.
- Part C (one or more correct) rewards exact statements. A typical stem fixes a model or a covariance matrix and lists four claims: “every parameter is estimable”, “the residuals are independent”, “\( X_1 \) and \( X_3 \) are conditionally independent given \( X_2 \)”, “principal components are scale invariant”. Each is settled by one theorem or one small example. Every topic page ends with Level 3 practice in exactly this format, with every option justified.
For the marking scheme and the other units, see the Mathematical Sciences overview. Part A (General Aptitude) is common to all subjects; its Experimental Design, Sampling and Data page covers the design ideas behind ANOVA at Part A depth. Found an error? See About for how to report it.
Start Learning
Begin with the first topic: