Short-, Medium- and Long-Range Prediction
CSIR NET meteorology: forecast ranges, predictability and chaos, error growth, ensemble prediction, seasonal monsoon forecasts and verification scores.
- 19 min
On this page · 16 sections
- Prerequisites
- The Hook: Three Questions, One Atmosphere
- Forecast Ranges
- Predictability: Why Forecasts Fail
- Short-Range Prediction
- Medium-Range Prediction
- Extended-Range Prediction
- Long-Range (Seasonal) Prediction
- Forecast Verification
- Memory Tricks
- Don’t Confuse
- Common Mistakes
- Practice
- Quick Revision Table
- Key Takeaways
- What’s Next
Prerequisites
- The models used at each range: Barotropic, Baroclinic and Primitive-Equation Models.
- ENSO, the Indian Ocean Dipole and the monsoon, which supply long-range predictability: Monsoon, Cyclones and ENSO and Air Masses, Monsoon, Jets, Cyclones and ENSO.
- Correlation, regression and probability for verification scores: Distributions and Statistical Inference and Part A data interpretation in General Aptitude.
- Chapter map: Numerical Weather Prediction.
The Hook: Three Questions, One Atmosphere
Forecast Ranges
The World Meteorological Organization’s manual on the global data-processing and forecasting system defines the ranges by lead time:
| Range | Lead time | Typical tools |
|---|---|---|
| Nowcasting | up to about 2 hours | radar and satellite extrapolation, very-short-range high-resolution NWP |
| Very short-range | up to 12 hours | high-resolution regional NWP, rapid-update assimilation |
| Short-range | beyond 12 hours, up to 72 hours (3 days) | regional and global NWP, statistical post-processing |
| Medium-range | beyond 3 days, up to 10 days | global NWP and ensemble prediction |
| Extended-range | beyond 10 days, up to 30 days | ensembles, intraseasonal signals (MJO, monsoon intraseasonal oscillation) |
| Long-range | from 30 days up to 2 years (monthly and seasonal) | coupled ocean–atmosphere ensembles, statistical models |
| Climate forecasting | beyond 2 years | coupled climate models, scenarios |
The India Meteorological Department uses the same broad split in its products: short-range forecasts (up to 3 days), medium-range forecasts (3–10 days), extended-range forecasts (weekly outlooks to about four weeks), and long-range forecasts (the seasonal monsoon rainfall forecast).
Predictability: Why Forecasts Fail
Chaos and sensitive dependence
In 1963 Edward Lorenz showed that a three-variable model of convection, fully deterministic, had solutions that diverged from each other however close they started. Small differences in initial conditions grow roughly exponentially until the two solutions are as different as two randomly chosen states. This sensitive dependence on initial conditions is the defining feature of chaos. In a later study (1969) he argued that, because errors at small scales grow fastest and spread to larger scales, the atmosphere has a finite intrinsic predictability limit for day-to-day weather of the order of two weeks, which better observations and models cannot remove.
Two separate limits are worth naming:
| Kind | Meaning |
|---|---|
| Intrinsic predictability | The limit set by the atmosphere’s own dynamics, even with a perfect model and almost perfect initial data |
| Practical predictability | The limit reached with today’s observations, assimilation and models; it improves with time towards the intrinsic limit |
Error growth
If an initial error \( E_0 \) grows exponentially with doubling time \( \tau_d \), then
$$E(t) = E_0\,2^{t/\tau_d}, \qquad t = \tau_d\log_2\frac{E}{E_0}.$$Growth slows and stops when the error reaches saturation (the size of the difference between two random states, about \( \sqrt{2} \) times the climatological standard deviation). A forecast loses usefulness well before saturation.
Two consequences follow directly from the logarithm:
- Halving the initial error gains only one doubling time of useful lead.
- Reducing the initial error tenfold gains \( \log_2 10 \approx 3.3 \) doubling times.
Error growth also depends on scale. Errors in convective-scale features grow within hours; synoptic-scale errors double in about a day or two; planetary-scale patterns hold longer. That is why a thunderstorm’s position can be forecast only hours ahead, a cyclone’s track several days ahead, and a blocking pattern sometimes beyond a week.
Predictability of the first and second kind
Lorenz also distinguished two sources of forecast skill:
| First kind | Second kind | |
|---|---|---|
| Depends on | initial conditions | boundary conditions and slow forcings |
| Forecasts | the actual weather sequence | statistics (averages, probabilities) of weather |
| Dominant at | short and medium range | long range (seasonal and beyond) |
| Lost or kept | lost after about two weeks | can persist for seasons where forcing is strong |
Slow forcings that make long-range forecasts possible:
- Sea surface temperature: ENSO in the tropical Pacific is the strongest single source; the Indian Ocean Dipole and Atlantic SSTs also matter for the Indian monsoon.
- Soil moisture and snow cover (for example Eurasian and Himalayan snow): they change surface heating and evaporation for weeks to months.
- Sea ice and stratospheric conditions.
- Greenhouse gases and aerosols over longer periods.
Seasonal predictability is highest in the tropics, where the atmosphere responds strongly to SST, and lower in the mid-latitudes, where internal chaotic variability dominates.
Short-Range Prediction
Short-range skill comes almost entirely from the initial state, so the main tools are dense observations, frequent data assimilation and high resolution:
- Regional (limited-area) models nested in global models, at kilometre-scale grid spacing where deep convection is resolved explicitly (“convection-permitting”); they must be non-hydrostatic.
- Rapid-update cycles: assimilation every hour or few hours, using radar reflectivity and radial winds, surface networks, aircraft data and geostationary satellite images.
- Nowcasting: for the first one or two hours, extrapolation of radar echoes and satellite cloud features often beats NWP, because the model needs time to “spin up” convection from the initial state.
- Statistical post-processing: raw model output has systematic biases (for example 2 m temperature over complex terrain). Model Output Statistics (MOS) fits regression equations between past model forecasts and observations at each station, correcting the model’s own biases. The older perfect prog method fits observed predictors to observed weather and then applies the equations to model forecasts as if they were perfect; it does not correct model bias but does not need retraining when the model changes.
Typical short-range products: quantitative precipitation forecasts (QPF), heavy-rainfall and thunderstorm warnings, cyclone track and landfall, fog, heatwave and cold-wave outlooks.
Medium-Range Prediction
By day 3–10, initial errors have grown to the synoptic scale, so a single deterministic run is only one of many plausible futures.
Deterministic skill
The standard measure is the anomaly correlation coefficient (ACC) of the 500 hPa geopotential height: the correlation between forecast and observed anomalies from climatology. By convention, a forecast with ACC above 60% is regarded as having useful synoptic skill, and above about 80% as accurate in its broad pattern. The lead time at which the ACC falls to 60% is a common headline measure of model progress; it has increased steadily over the decades, often summarised as about a day per decade, with the northern hemisphere ahead of the southern hemisphere for most of that history (the gap closed as satellite data assimilation improved).
Ensemble prediction systems (EPS)
An ensemble runs the model many times from slightly different initial states (and often with perturbed physics), to sample the uncertainty.
| Ingredient | Purpose | Examples of methods |
|---|---|---|
| Initial perturbations | represent analysis uncertainty, aimed at the fastest-growing directions | singular vectors (fastest growth over a chosen interval), bred vectors (breeding growing modes in the assimilation cycle), perturbations from an ensemble of data assimilations |
| Model perturbations | represent model error | stochastic physics, multiple parameterization schemes, multi-model ensembles |
| Control run | unperturbed member |
Ensemble products:
- Ensemble mean: usually more skilful than any single member at medium range, because unpredictable details average out (it is smoother than reality).
- Spread (standard deviation of members): an estimate of forecast uncertainty. In a well-calibrated ensemble, the spread matches the error of the ensemble mean on average (the spread–skill relationship).
- Probabilities: the fraction of members exceeding a threshold (for example, rain above 64.5 mm in 24 hours).
- Spaghetti plots (one contour from every member), plumes (time series at one place) and cluster analysis of scenarios; cyclone track strike-probability maps.
Extended-Range Prediction
Between about 10 and 30 days, initial-condition skill for daily weather is nearly gone, but some intraseasonal signals remain predictable:
- The Madden–Julian Oscillation (MJO): an eastward-moving envelope of tropical convection with a period of roughly 30–60 days, which modulates rainfall over India, cyclogenesis and the monsoon onset.
- The monsoon intraseasonal oscillation: northward-propagating active and break spells over India.
- Persistent anomalies of soil moisture and SST, and some stratospheric events.
Extended-range forecasts are issued as weekly averages or anomalies (for example week 2, week 3, week 3–4) from ensembles, often as probabilities of above- or below-normal conditions.
Long-Range (Seasonal) Prediction
What is forecast
A seasonal forecast predicts the statistics of the season, usually as an anomaly from climatology and as probabilities of three equally likely categories (terciles: below normal, near normal, above normal). It does not forecast individual weather events.
Statistical and dynamical methods
| Statistical (empirical) | Dynamical | |
|---|---|---|
| Basis | historical relationships between predictors (SST indices, snow cover, pressure patterns) and seasonal rainfall or temperature | coupled ocean–atmosphere (and land, sea-ice) model ensembles started from observed states |
| Strength | cheap, uses long records | physically consistent, can respond to unprecedented states |
| Weakness | relationships can weaken or change; risk of overfitting with many predictors | model biases (for example in monsoon rainfall), need large ensembles and hindcasts |
Statistical methods include multiple linear regression, principal-component regression and ensembles of statistical models. Dynamical systems are run as hindcasts (retrospective forecasts) over many past years to measure skill and correct bias, and are often combined into multi-model ensembles, which are usually more reliable than any single model.
Seasonal forecasting of the Indian summer monsoon
IMD issues an operational long-range forecast of the June–September (southwest monsoon) all-India rainfall, first in April with an update around the end of May or early June, and adds monthly and regional outlooks. The forecast is expressed as a percentage of the long period average (LPA) and as probabilities for five categories:
| Category | Rainfall (% of LPA) |
|---|---|
| Deficient | below 90 |
| Below normal | 90–95 |
| Normal | 96–104 |
| Above normal | 105–110 |
| Excess | above 110 |
IMD now combines statistical ensemble methods with coupled dynamical model forecasts. The main known drivers are ENSO (El Niño years tend towards deficient monsoons, though not always), the Indian Ocean Dipole and Eurasian snow cover; much of the year-to-year variability is internal and unpredictable, which keeps monsoon seasonal skill modest.
Forecast Verification
Continuous variables
| Score | Formula | Perfect | Notes |
|---|---|---|---|
| Mean error (bias) | \( \frac{1}{N}\sum (F_i - O_i) \) | 0 | systematic over- or under-forecasting |
| Mean absolute error | \( \frac{1}{N}\sum \lvert F_i - O_i\rvert \) | 0 | |
| Root-mean-square error | \( \sqrt{\frac{1}{N}\sum (F_i - O_i)^2} \) | 0 | penalises large errors |
| Anomaly correlation | \( \dfrac{\sum f'_i o'_i}{\sqrt{\sum f'^2_i \sum o'^2_i}} \) | 1 | anomalies \( f', o' \) from climatology; pattern skill, insensitive to amplitude bias |
| Skill score | \( SS = 1 - S/S_{\text{ref}} \) (for error scores) | 1 | positive means better than the reference (climatology or persistence) |
(The anomaly correlation is sometimes computed after removing the mean anomaly over the domain, the “centred” form; the uncentred form above is the one used in the practice questions.)
Yes/no events: the contingency table
| Observed yes | Observed no | |
|---|---|---|
| Forecast yes | hits \( a \) | false alarms \( b \) |
| Forecast no | misses \( c \) | correct negatives \( d \) |
| Score | Formula | Perfect |
|---|---|---|
| Probability of detection (hit rate), POD | \( a/(a + c) \) | 1 |
| False alarm ratio, FAR | \( b/(a + b) \) | 0 |
| Critical success index (threat score), CSI | \( a/(a + b + c) \) | 1 |
| Frequency bias | \( (a + b)/(a + c) \) | 1 (> 1 over-forecasts the event) |
| Proportion correct | \( (a + d)/n \) | 1 (misleading for rare events) |
| Equitable threat score, ETS | \( (a - a_r)/(a + b + c - a_r) \), \( a_r = (a + b)(a + c)/n \) | 1 (0 for a random forecast) |
The false alarm ratio (fraction of “yes” forecasts that were wrong) is not the false alarm rate \( b/(b + d) \) (fraction of non-events forecast as events), which is used in ROC curves.
Probability forecasts
- Brier score: \( BS = \frac{1}{N}\sum (p_i - o_i)^2 \), with \( p_i \) the forecast probability and \( o_i = 1 \) or 0. Perfect = 0; it can be decomposed into reliability, resolution and uncertainty.
- Reliability diagram: observed frequency against forecast probability; a reliable system lies on the diagonal (when it says 30%, the event happens 30% of the time).
- ROC curve: hit rate against false alarm rate for different probability thresholds; area under the curve above 0.5 shows discrimination.
Memory Tricks
Don’t Confuse
| This | vs | That |
|---|---|---|
| Predictability of the first kind (initial value) | Second kind (boundary forcing: SST, soil moisture, snow) | |
| Intrinsic predictability (set by the atmosphere) | Practical predictability (set by today’s models and data) | |
| Medium-range: 3–10 days | Extended-range: 10–30 days | |
| Ensemble mean: best estimate, smoothed | Ensemble spread: uncertainty | |
| MOS: corrects the model’s own bias | Perfect prog: assumes the model is perfect | |
| False alarm ratio \( b/(a + b) \) | False alarm rate \( b/(b + d) \) | |
| Frequency bias (count ratio) | Mean error bias (average difference) | |
| Seasonal forecast of mean rainfall | Forecast of the weather on a given day of the season |
Common Mistakes
Practice
All questions below are original practice questions written for this page. They are not from any CSIR NET paper.
Level 1: Foundation
- In the WMO classification, which range covers a forecast for day 5?
- Which source of predictability dominates a seasonal forecast: initial conditions or boundary forcing?
- What does a Brier score of zero mean?
Answers
- Medium-range (beyond 3 days up to 10 days).
- Boundary forcing (predictability of the second kind: SST, soil moisture, snow cover).
- A perfect probability forecast: every forecast probability was 1 for events that happened and 0 for events that did not.
Level 2: Part B Style
Practice Q1. A model’s 500 hPa error starts at 0.2 units and doubles every 2 days. If the error is considered saturated at 6.4 units, the time to saturation is:
(a) 5 days (b) 6.4 days (c) 10 days (d) 32 days
Answer: (c) 10 days
\( 6.4/0.2 = 32 = 2^5 \): five doublings × 2 days = 10 days. Option (d) confuses the ratio with days; option (a) counts doublings instead of days.
Practice Q2. Forecast anomalies at five grid points are 2, −1, 3, 0, −2 and the observed anomalies are 1, −2, 2, 1, −1 (same units). The uncentred anomaly correlation coefficient is about:
(a) 0.60 (b) 0.72 (c) 0.85 (d) 0.95
Answer: (c) about 0.85
\( \sum f'o' = 2 + 2 + 6 + 0 + 2 = 12 \); \( \sum f'^2 = 4 + 1 + 9 + 0 + 4 = 18 \); \( \sum o'^2 = 1 + 4 + 4 + 1 + 1 = 11 \).
ACC \( = 12/\sqrt{18 \times 11} = 12/14.07 = 0.853 \). Above 0.6, so this would count as a useful forecast of the pattern.
Practice Q3. Four probability forecasts of rain were 0.9, 0.7, 0.2 and 0.1; rain occurred on the first and fourth occasions only. The Brier score is:
(a) 0.16 (b) 0.25 (c) 0.34 (d) 0.68
Answer: (c) about 0.34
\( (0.9 - 1)^2 + (0.7 - 0)^2 + (0.2 - 0)^2 + (0.1 - 1)^2 = 0.01 + 0.49 + 0.04 + 0.81 = 1.35 \); divided by 4 gives 0.3375. The costly terms are the confident wrong forecasts (0.7 with no rain, 0.1 with rain).
Practice Q4. IMD forecasts the season’s all-India monsoon rainfall at 103% of the LPA. In IMD’s five-category scheme this falls in:
(a) Below normal (b) Normal (c) Above normal (d) Excess
Answer: (b) Normal
Normal is 96–104% of LPA. Above normal is 105–110%; excess is above 110%.
Level 3: Part C Style
Practice Q5. Over 200 days, heavy-rain forecasts for a district give 40 hits, 20 false alarms, 10 misses and 130 correct negatives. Consider:
- POD = 0.80 and FAR ≈ 0.33.
- The critical success index is about 0.57.
- The frequency bias is 1.2, so the event is over-forecast.
- The equitable threat score is about 0.45.
Which are correct?
(a) 1 and 2 only (b) 1, 2 and 3 only (c) 2, 3 and 4 only (d) 1, 2, 3 and 4
Answer: (d) 1, 2, 3 and 4
\( a = 40, b = 20, c = 10, d = 130, n = 200 \).
- POD \( = 40/50 = 0.80 \); FAR \( = 20/60 = 0.333 \). True.
- CSI \( = 40/70 = 0.571 \). True.
- Bias \( = 60/50 = 1.2 > 1 \). True.
- \( a_r = (60)(50)/200 = 15 \); ETS \( = (40 - 15)/(70 - 15) = 25/55 = 0.455 \). True.
Proportion correct is \( 170/200 = 0.85 \), which looks better than any of these and is the least informative.
Practice Q6. A five-member ensemble forecasts 24-hour rainfall at a station of 12, 18, 25, 30 and 45 mm. Which statement is correct?
(a) Ensemble mean 26 mm; population spread about 11.3 mm; probability of more than 20 mm is 0.6 (b) Ensemble mean 25 mm; spread about 12.6 mm; probability of more than 20 mm is 0.4 (c) Ensemble mean 26 mm; spread 0, because the members are all forecasts of the same day (d) Ensemble mean 45 mm, since the warning should be based on the wettest member
Answer: (a)
Mean \( = 130/5 = 26 \) mm. Deviations: −14, −8, −1, 4, 19; squares sum to \( 196 + 64 + 1 + 16 + 361 = 638 \); population variance \( 638/5 = 127.6 \); spread \( \approx 11.3 \) mm (the sample standard deviation, dividing by 4, would be about 12.6 mm). Members above 20 mm: 25, 30, 45, so probability \( 3/5 = 0.6 \).
(b) uses the median (25) and the wrong count. The wettest member (d) tells you what is possible, not what to expect.
Practice Q7. Synoptic-scale forecast errors double every 1.5 days. Which statements are correct?
- Reducing the initial error by a factor of 10 extends the time to reach a given error level by about 5 days.
- Halving the initial error extends that time by 1.5 days.
- Indefinite improvement in initial data would extend useful daily-weather forecasts indefinitely.
- A seasonal-mean forecast can show skill beyond this limit because it relies on predictability of the second kind.
(a) 1 and 2 only (b) 1, 2 and 4 only (c) 3 and 4 only (d) 1, 2, 3 and 4
Answer: (b) 1, 2 and 4 only
- \( \tau_d\log_2 10 = 1.5 \times 3.32 = 4.98 \approx 5 \) days. True.
- One doubling time = 1.5 days. True.
- False. Errors at small scales grow ever faster and spread upscale, so there is an intrinsic limit of the order of two weeks for daily weather; and the exponential model itself fails as errors saturate.
- True. Seasonal means draw on slowly varying forcing (SST, soil moisture, snow).
Quick Revision Table
| Idea | Formula or fact |
|---|---|
| WMO ranges | nowcast ≤ 2 h; very short ≤ 12 h; short 12–72 h; medium 3–10 d; extended 10–30 d; long 30 d–2 yr; climate > 2 yr |
| Chaos | Lorenz 1963: sensitive dependence on initial conditions |
| Intrinsic limit | of the order of two weeks for daily weather |
| Error growth | \( E = E_0 2^{t/\tau_d} \); gain \( \tau_d\log_2(\text{factor}) \) |
| First / second kind | initial conditions / boundary forcing |
| Long-range sources | SST (ENSO, IOD), soil moisture, snow cover, sea ice |
| Short range | high-resolution regional models, rapid-update DA, nowcasting, MOS |
| Medium range | global models, EPS; ACC 60% = useful |
| EPS perturbations | singular vectors, bred vectors, ensemble DA; stochastic physics; multi-model |
| Extended range | MJO (about 30–60 days), monsoon intraseasonal oscillation; weekly anomalies |
| IMD monsoon categories | < 90 deficient; 90–95 below normal; 96–104 normal; 105–110 above normal; > 110 excess (% LPA) |
| POD, FAR, CSI | \( a/(a+c) \), \( b/(a+b) \), \( a/(a+b+c) \) |
| Bias, ETS | \( (a+b)/(a+c) \); \( (a-a_r)/(a+b+c-a_r) \) |
| Brier score | \( \frac1N\sum(p - o)^2 \); 0 is perfect |
| Skill score | \( 1 - S/S_{\text{ref}} \) |
Key Takeaways
What’s Next
- Objective Analysis, Initialization, Data Assimilation and Satellite Data: how the initial state that all these forecasts depend on is built.
- Barotropic, Baroclinic and Primitive-Equation Models: the models behind each range.
- Long-range drivers in the Part B context: Climate, Sea Level, Pollution and Ozone.
- Numerical Weather Prediction: Quick Revision.