Picard, Euler, Modified Euler and Runge-Kutta
CSIR NET numerical analysis: Picard iteration, Euler and modified Euler methods, Runge-Kutta of orders 2 and 4, local and global error, stability for IVPs.
- 20 min
On this page · 18 sections
- Prerequisites
- The Hook: Three Answers to One Question
- The Initial Value Problem
- Picard’s Method of Successive Approximations
- One-Step Methods: The Vocabulary
- Euler’s Method
- Modified Euler Methods (Second Order)
- The Classical Fourth-Order Runge-Kutta Method
- Systems and Higher-Order Equations
- Stability: the Test Equation
- Comparison Table
- Memory Tricks
- Don’t Confuse
- Common Mistakes
- Practice
- Quick Revision Table
- Key Takeaways
- What’s Next
Prerequisites
- Numerical Differentiation and Integration: each one-step method is a quadrature rule applied to \( y(x_{n+1}) = y(x_n) + \int_{x_n}^{x_{n+1}} f(x, y(x))\,dx \).
- Iteration, Newton-Raphson and Rate of Convergence: Picard’s method is a fixed-point iteration on functions, and implicit methods need a fixed-point or Newton solve at each step.
- Sequences and Series of Functions: Picard iterates converge uniformly, by a Weierstrass M-test argument.
- First-order ODEs: separable and linear equations, to check answers against exact solutions.
The Hook: Three Answers to One Question
The Initial Value Problem
$$y' = f(x, y), \qquad y(x_0) = y_0.$$Integrating, the IVP is equivalent (for continuous \( f \)) to the integral equation
$$y(x) = y_0 + \int_{x_0}^{x} f(t, y(t))\,dt.$$A bounded \( \frac{\partial f}{\partial y} \) on \( R \) gives a Lipschitz constant. Without the Lipschitz condition, solutions exist (Peano) but need not be unique: \( y' = 3y^{2/3} \), \( y(0) = 0 \) has both \( y \equiv 0 \) and \( y = x^3 \).
Picard’s Method of Successive Approximations
$$y_0(x) \equiv y_0, \qquad y_{n+1}(x) = y_0 + \int_{x_0}^{x} f\big(t, y_n(t)\big)\,dt.$$This is the fixed-point iteration for the integral equation. Under the hypotheses of the theorem,
$$\lvert y(x) - y_n(x) \rvert \le \frac{M L^n \lvert x - x_0 \rvert^{n+1}}{(n + 1)!},$$a bound that goes to 0 faster than any geometric sequence.
More examples worth knowing:
| IVP | Iterates | Exact solution |
|---|---|---|
| \( y' = y \), \( y(0) = 1 \) | \( y_n = 1 + x + \dots + \frac{x^n}{n!} \) | \( e^x \) |
| \( y' = 2x(1 + y) \), \( y(0) = 0 \) | \( y_1 = x^2 \), \( y_2 = x^2 + \frac{x^4}{2} \), \( y_3 = x^2 + \frac{x^4}{2} + \frac{x^6}{6} \) | \( e^{x^2} - 1 \) |
| \( y' = x^2 + y^2 \), \( y(0) = 0 \) | \( y_1 = \frac{x^3}{3} \), \( y_2 = \frac{x^3}{3} + \frac{x^7}{63} \) | not elementary |
Strengths and limits. Picard gives a formula, not a table, and proves existence. But each step needs an integral in closed form, which quickly becomes impossible (try \( y' = \sin y \)). In practice it is used for theory and for a few terms of a series; for numbers, we use step-by-step methods.
Taylor series method. Differentiate the ODE to get \( y'', y''', \dots \) at \( x_0 \) and sum \( y(x_0 + h) \approx \sum_{k=0}^{p} \frac{h^k}{k!}y^{(k)}(x_0) \). It has order \( p \) but needs derivatives of \( f \); Runge-Kutta methods get the same order from values of \( f \) only.
One-Step Methods: The Vocabulary
Grid \( x_n = x_0 + nh \); \( y_n \approx y(x_n) \). A one-step method is \( y_{n+1} = y_n + h\,\Phi(x_n, y_n; h) \).
- Local truncation error \( \tau_{n+1} \): the error made in one step when starting from the exact value, \( y(x_{n+1}) - y(x_n) - h\Phi(x_n, y(x_n); h) \).
- Order \( p \): \( \tau = O(h^{p+1}) \). Then (for Lipschitz \( \Phi \)) the global error \( y(x_n) - y_n \) at a fixed \( x \) is \( O(h^p) \): about \( \frac{x - x_0}{h} \) steps, each adding \( O(h^{p+1}) \).
- A method has order \( p \) if its expansion in \( h \) agrees with the Taylor series of the true solution through the \( h^p \) term.
Euler’s Method
Follow the tangent line for one step:
$$\boxed{y_{n+1} = y_n + h\,f(x_n, y_n)}$$It is the left rectangle rule applied to \( \int_{x_n}^{x_{n+1}} f\,dx \).
- Local truncation error: \( \frac{h^2}{2}y''(\xi) \), so order 1.
- Global error bound: if \( \lvert y'' \rvert \le M \) and \( f \) is Lipschitz with constant \( L \), then \( \lvert y(x_n) - y_n \rvert \le \frac{hM}{2L}\left(e^{L(x_n - x_0)} - 1\right) \). Halving \( h \) roughly halves the error.
Backward (implicit) Euler \( y_{n+1} = y_n + hf(x_{n+1}, y_{n+1}) \) is also order 1, but needs an equation solved at each step; its reward is stability (below).
Modified Euler Methods (Second Order)
Heun’s method (Euler predictor, trapezoidal corrector)
$$y^{*}_{n+1} = y_n + hf(x_n, y_n), \qquad y_{n+1} = y_n + \frac{h}{2}\Big[f(x_n, y_n) + f(x_{n+1}, y^{*}_{n+1})\Big].$$It is the trapezoidal rule with the unknown endpoint value predicted by Euler. Some texts iterate the corrector until two successive values agree; the limit is the solution of the implicit trapezoidal equation \( y_{n+1} = y_n + \frac{h}{2}[f(x_n, y_n) + f(x_{n+1}, y_{n+1})] \), which is also second order.
Midpoint method
$$y_{n+1} = y_n + h\,f\Big(x_n + \frac{h}{2},\ y_n + \frac{h}{2}f(x_n, y_n)\Big).$$It is the midpoint rule with the midpoint value predicted by a half Euler step.
The general two-stage Runge-Kutta family
$$k_1 = f(x_n, y_n), \quad k_2 = f(x_n + \alpha h,\ y_n + \beta h k_1), \quad y_{n+1} = y_n + h(a k_1 + b k_2).$$Matching the Taylor series through \( h^2 \) gives the order-2 conditions
$$a + b = 1, \qquad b\alpha = \frac{1}{2}, \qquad b\beta = \frac{1}{2}.$$| Method | \( a \) | \( b \) | \( \alpha = \beta \) |
|---|---|---|---|
| Heun (trapezoidal) | \( \frac{1}{2} \) | \( \frac{1}{2} \) | 1 |
| Midpoint | 0 | 1 | \( \frac{1}{2} \) |
| Ralston | \( \frac{1}{4} \) | \( \frac{3}{4} \) | \( \frac{2}{3} \) |
A one-parameter family: every choice of \( b \ne 0 \) gives a second-order method, and no choice gives order 3 for all \( f \). With \( b = 0 \) the scheme is Euler.
The Classical Fourth-Order Runge-Kutta Method
$$\begin{aligned} k_1 &= h\,f(x_n, y_n), \\ k_2 &= h\,f\big(x_n + \tfrac{h}{2},\ y_n + \tfrac{k_1}{2}\big), \\ k_3 &= h\,f\big(x_n + \tfrac{h}{2},\ y_n + \tfrac{k_2}{2}\big), \\ k_4 &= h\,f\big(x_n + h,\ y_n + k_3\big), \\ y_{n+1} &= y_n + \tfrac{1}{6}\big(k_1 + 2k_2 + 2k_3 + k_4\big). \end{aligned}$$(Here the \( k_i \) include the factor \( h \); some books put \( h \) outside instead. Watch which convention the question uses.)
- Four evaluations of \( f \) per step; local error \( O(h^5) \), global \( O(h^4) \): halving \( h \) divides the error by about 16.
- If \( f = f(x) \) only, then \( k_2 = k_3 \) and the step is \( \frac{h}{6}[f(x_n) + 4f(x_n + \frac{h}{2}) + f(x_n + h)] \): Simpson’s rule on \( [x_n, x_{n+1}] \).
- For \( y' = \lambda y \), one step multiplies \( y \) by \( 1 + z + \frac{z^2}{2} + \frac{z^3}{6} + \frac{z^4}{24} \), \( z = h\lambda \): the Taylor polynomial of \( e^z \) through \( z^4 \).
Systems and Higher-Order Equations
Every method above works component-wise for a system \( \mathbf{y}' = \mathbf{f}(x, \mathbf{y}) \). A higher-order equation is first reduced to a system: for \( y'' = g(x, y, y') \), set \( u = y \), \( v = y' \):
$$u' = v, \qquad v' = g(x, u, v), \qquad u(x_0) = y_0, \quad v(x_0) = y_0'.$$In RK4 for a system, compute \( k_1 \) for all components before any \( k_2 \), and so on.
Stability: the Test Equation
Apply a method to \( y' = \lambda y \) with \( \operatorname{Re}\lambda < 0 \) (exact solution decays). A one-step method gives \( y_{n+1} = R(h\lambda)\,y_n \), and the numerical solution decays iff \( \lvert R(h\lambda) \rvert < 1 \). The set of such \( z = h\lambda \) is the region of absolute stability.
| Method | \( R(z) \) | Real stability interval (\( \lambda < 0 \)) |
|---|---|---|
| Euler | \( 1 + z \) | \( -2 < z < 0 \), i.e. \( h < \frac{2}{\lvert \lambda \rvert} \) |
| Heun / midpoint (any 2-stage order 2) | \( 1 + z + \frac{z^2}{2} \) | \( -2 < z < 0 \) |
| Classical RK4 | \( 1 + z + \frac{z^2}{2} + \frac{z^3}{6} + \frac{z^4}{24} \) | about \( -2.785 < z < 0 \) |
| Backward Euler | \( \frac{1}{1 - z} \) | all \( z < 0 \) (whole left half-plane) |
| Implicit trapezoidal | \( \frac{1 + z/2}{1 - z/2} \) | all \( z < 0 \) (whole left half-plane) |
Example: for \( y' = -20y \), Euler is stable only for \( h < 0.1 \). With \( h = 0.15 \), \( R = 1 - 3 = -2 \) and the numerical solution oscillates with growing amplitude, although the true solution decays. Problems with widely separated time scales (stiff problems) force tiny steps on explicit methods; implicit methods avoid this.
Comparison Table
| Method | Type | \( f \)-evaluations per step | Local error | Global order |
|---|---|---|---|---|
| Picard | successive approximation (analytic) | integrals, not evaluations | \( \frac{ML^n h^{n+1}}{(n+1)!} \) after \( n \) iterations | (not a stepping method) |
| Euler | explicit, 1 stage | 1 | \( O(h^2) \) | 1 |
| Backward Euler | implicit | solve each step | \( O(h^2) \) | 1 |
| Heun / modified Euler | explicit, 2 stages | 2 | \( O(h^3) \) | 2 |
| Midpoint | explicit, 2 stages | 2 | \( O(h^3) \) | 2 |
| Classical RK4 | explicit, 4 stages | 4 | \( O(h^5) \) | 4 |
Memory Tricks
Don’t Confuse
| This | vs | That |
|---|---|---|
| Local truncation error \( O(h^{p+1}) \) | global error \( O(h^p) \) | |
| Heun: slope at the start and at the predicted end, averaged | midpoint: one slope at the predicted midpoint | |
| Picard iterate \( y_n(x) \): a function | Euler value \( y_n \): a number at \( x_n \) | |
| Explicit methods: conditionally stable | backward Euler, implicit trapezoid: stable for every \( h \) when \( \operatorname{Re}\lambda < 0 \) | |
| RK4 \( k_i \) with the \( h \) inside | \( k_i \) as slopes, with \( y_{n+1} = y_n + \frac{h}{6}(\cdots) \) | |
| Order of accuracy (how fast error \( \to 0 \)) | stability (whether errors stay bounded at a given \( h \)) |
Common Mistakes
Practice
All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.
Level 1: Foundation
- Write the Picard iteration for \( y' = f(x, y) \), \( y(x_0) = y_0 \).
- State the local and global orders of Euler’s method and of classical RK4.
- For \( y' = \lambda y \) with \( \lambda < 0 \), what is the largest step for which Euler’s method is stable?
Answers
- \( y_{n+1}(x) = y_0 + \int_{x_0}^x f(t, y_n(t))\,dt \), starting from \( y_0(x) \equiv y_0 \).
- Euler: local \( O(h^2) \), global \( O(h) \). RK4: local \( O(h^5) \), global \( O(h^4) \).
- \( h < \frac{2}{\lvert \lambda \rvert} \) (from \( \lvert 1 + h\lambda \rvert < 1 \)).
Level 2: Part B Style
Practice Q1. For \( y' = 2x(1 + y) \), \( y(0) = 0 \), the second Picard iterate (starting from \( y_0 \equiv 0 \)) is
(a) \( x^2 \) (b) \( x^2 + \frac{x^4}{2} \) (c) \( x^2 + \frac{x^4}{4} \) (d) \( x^2 + x^4 \)
Answer: (b)
\( y_1 = \int_0^x 2t\,dt = x^2 \). \( y_2 = \int_0^x 2t(1 + t^2)\,dt = x^2 + \frac{x^4}{2} \). These are partial sums of the exact solution \( e^{x^2} - 1 = x^2 + \frac{x^4}{2} + \frac{x^6}{6} + \cdots \).
Practice Q2. Euler’s method with \( h = 0.1 \) for \( y' = y^2 \), \( y(0) = 1 \) gives \( y(0.2) \approx \)
(a) 1.2 (b) 1.21 (c) 1.221 (d) 1.25
Answer: (c)
\( y_1 = 1 + 0.1(1)^2 = 1.1 \); \( y_2 = 1.1 + 0.1(1.21) = 1.221 \). (Exact: \( y = \frac{1}{1 - x} \), \( y(0.2) = 1.25 \).)
Practice Q3. Heun’s method (Euler predictor, trapezoidal corrector) with \( h = 0.2 \) for \( y' = -2xy^2 \), \( y(0) = 1 \) gives \( y(0.2) \approx \)
(a) 0.92 (b) 0.96 (c) 0.9615 (d) 1.0
Answer: (b)
\( f(0, 1) = 0 \), so the predictor is \( y^* = 1 \). \( f(0.2, 1) = -0.4 \). Corrector: \( y_1 = 1 + 0.1(0 - 0.4) = 0.96 \). (Exact: \( \frac{1}{1 + x^2} \), \( y(0.2) \approx 0.9615 \); option (d) is Euler’s value.)
Practice Q4. For \( y' = -20y \), the explicit Euler method produces a decaying numerical solution exactly when the step \( h \) satisfies
(a) \( h < 0.05 \) (b) \( h < 0.1 \) (c) \( h < 0.2 \) (d) every \( h > 0 \)
Answer: (b)
\( y_{n+1} = (1 - 20h)y_n \); decay needs \( \lvert 1 - 20h \rvert < 1 \), i.e. \( 0 < h < 0.1 \). For \( 0.05 < h < 0.1 \) the iterates alternate in sign but still decay; (a) is the condition for monotone decay. Option (d) describes backward Euler.
Level 3: Part C Style
Practice Q5. Consider \( y_{n+1} = y_n + h(ak_1 + bk_2) \) with \( k_1 = f(x_n, y_n) \), \( k_2 = f(x_n + \alpha h, y_n + \beta h k_1) \). Which statements are true?
(a) \( a = b = \frac{1}{2} \), \( \alpha = \beta = 1 \) gives a second-order method. (b) \( a = 0 \), \( b = 1 \), \( \alpha = \beta = \frac{1}{2} \) gives a second-order method. (c) \( a = \frac{1}{3} \), \( b = \frac{2}{3} \), \( \alpha = \beta = \frac{3}{4} \) gives a second-order method. (d) Some choice of \( a, b, \alpha, \beta \) gives third order for every smooth \( f \).
Answer: (a), (b) and (c)
Order-2 conditions: \( a + b = 1 \), \( b\alpha = \frac{1}{2} \), \( b\beta = \frac{1}{2} \).
- (a) True (Heun): \( 1 \), \( \frac{1}{2} \), \( \frac{1}{2} \).
- (b) True (midpoint): \( 1 \), \( \frac{1}{2} \), \( \frac{1}{2} \).
- (c) True: \( \frac{1}{3} + \frac{2}{3} = 1 \), \( \frac{2}{3}\cdot\frac{3}{4} = \frac{1}{2} \).
- (d) False. With two stages, the \( h^3 \) term of the Taylor expansion contains \( f_y(f_x + f f_y) \) type terms that \( k_2 \) cannot produce; test on \( y' = y \): the step multiplies \( y \) by \( 1 + h + b\beta h^2 = 1 + h + \frac{h^2}{2} \), missing \( \frac{h^3}{6} \) whatever the parameters.
Practice Q6. RK4 (with \( k_i \) including the factor \( h \)) is applied to \( y' = y - x \), \( y(0) = 2 \), \( h = 0.1 \). The exact solution is \( y = e^x + x + 1 \). Which statements are true?
(a) \( k_1 = 0.2 \) and \( k_2 = 0.205 \). (b) \( k_3 = 0.20525 \) and \( k_4 = 0.210525 \). (c) \( y_1 \approx 2.2051708 \), within \( 10^{-7} \) of \( y(0.1) \). (d) Halving \( h \) would reduce the global error at \( x = 1 \) by a factor of about 4.
Answer: (a), (b) and (c)
- (a) True. \( k_1 = 0.1(2 - 0) = 0.2 \); \( k_2 = 0.1(2.1 - 0.05) = 0.205 \).
- (b) True. \( k_3 = 0.1(2.1025 - 0.05) = 0.20525 \); \( k_4 = 0.1(2.20525 - 0.1) = 0.210525 \).
- (c) True. \( y_1 = 2 + \frac{0.2 + 0.41 + 0.4105 + 0.210525}{6} = 2 + \frac{1.231025}{6} \approx 2.2051708 \); exact \( e^{0.1} + 1.1 \approx 2.2051709 \), a difference of about \( 8.5 \times 10^{-8} \).
- (d) False. RK4 is fourth order: halving \( h \) divides the global error by about \( 2^4 = 16 \).
Practice Q7. For the IVP \( y' = \lambda y \), \( y(0) = 1 \), with real \( \lambda < 0 \), which statements are true?
(a) One step of classical RK4 multiplies \( y \) by \( 1 + h\lambda + \frac{(h\lambda)^2}{2} + \frac{(h\lambda)^3}{6} + \frac{(h\lambda)^4}{24} \). (b) Heun’s method is stable for every \( h\lambda \in (-2, 0) \). (c) Classical RK4 is stable for \( h\lambda = -2.5 \), while Euler’s method is not. (d) The Picard iterates for this IVP are \( y_n(x) = \sum_{k=0}^{n}\frac{(\lambda x)^k}{k!} \).
Answer: all four, (a), (b), (c) and (d)
- (a) True. Compute the stages: \( k_1 = h\lambda y \), \( k_2 = h\lambda(1 + \frac{z}{2})y \), \( k_3 = h\lambda(1 + \frac{z}{2} + \frac{z^2}{4})y \), \( k_4 = h\lambda(1 + z + \frac{z^2}{2} + \frac{z^3}{4})y \) with \( z = h\lambda \); the weighted sum gives the stated factor.
- (b) True. \( R(z) = 1 + z + \frac{z^2}{2} = \frac{1}{2}\left((z + 1)^2 + 1\right) \), which lies in \( [\frac{1}{2}, 1) \) for \( -2 < z < 0 \).
- (c) True. \( R(-2.5) = 1 - 2.5 + 3.125 - 2.6042 + 1.6276 \approx 0.648 \), so \( \lvert R \rvert < 1 \); Euler gives \( \lvert 1 - 2.5 \rvert = 1.5 > 1 \).
- (d) True. \( y_{n+1} = 1 + \int_0^x \lambda y_n(t)\,dt \) adds the next term of the exponential series.
Quick Revision Table
| Idea | Fact |
|---|---|
| Picard-Lindelöf | continuous + Lipschitz in \( y \) \( \Rightarrow \) unique solution on \( \lvert x - x_0 \rvert \le \min(a, \frac{b}{M}) \) |
| Picard iteration | \( y_{n+1} = y_0 + \int_{x_0}^x f(t, y_n(t))\,dt \); error \( \le \frac{ML^n\lvert x - x_0 \rvert^{n+1}}{(n+1)!} \) |
| Euler | \( y_{n+1} = y_n + hf(x_n, y_n) \); order 1 |
| Heun (modified Euler) | predictor \( y^* = y_n + hf_n \); \( y_{n+1} = y_n + \frac{h}{2}[f_n + f(x_{n+1}, y^*)] \); order 2 |
| Midpoint | \( y_{n+1} = y_n + hf(x_n + \frac{h}{2}, y_n + \frac{h}{2}f_n) \); order 2 |
| RK2 conditions | \( a + b = 1 \), \( b\alpha = b\beta = \frac{1}{2} \) |
| RK4 | \( y_{n+1} = y_n + \frac{1}{6}(k_1 + 2k_2 + 2k_3 + k_4) \); order 4; Simpson when \( f = f(x) \) |
| Order \( p \) | local \( O(h^{p+1}) \), global \( O(h^p) \) |
| Stability (real \( h\lambda \)) | Euler and RK2: \( (-2, 0) \); RK4: about \( (-2.785, 0) \); backward Euler: all negative |
Key Takeaways
What’s Next
- Numerical Analysis: Quick Revision: every formula, order and error term of the chapter on one page.
- Numerical Differentiation and Integration: revisit the quadrature rules hiding inside each method.
- The Mathematical Sciences overview lists the ODE and PDE chapter of Unit 3, where existence and uniqueness are treated in full.