Taylor expansions for the moments of functions of random variables

In probability theory, it is possible to approximate the moments of a function f of a random variable X using Taylor expansions, provided that f is sufficiently differentiable and that the moments of X are finite.

First moment

${\displaystyle {\begin{aligned}\operatorname {E} \left[f(X)\right]&{}=\operatorname {E} \left[f\left(\mu _{X}+\left(X-\mu _{X}\right)\right)\right]\\&{}\approx \operatorname {E} \left[f(\mu _{X})+f'(\mu _{X})\left(X-\mu _{X}\right)+{\frac {1}{2}}f''(\mu _{X})\left(X-\mu _{X}\right)^{2}\right].\end{aligned}}}$

Since ${\displaystyle E[X-\mu _{X}]=0,}$ the second term disappears. Also ${\displaystyle E[(X-\mu _{X})^{2}]}$ is ${\displaystyle \sigma _{X}^{2}}$. Therefore,

${\displaystyle \operatorname {E} \left[f(X)\right]\approx f(\mu _{X})+{\frac {f''(\mu _{X})}{2}}\sigma _{X}^{2}}$

where ${\displaystyle \mu _{X}}$ and ${\displaystyle \sigma _{X}^{2}}$ are the mean and variance of X respectively.[1]

It is possible to generalize this to functions of more than one variable using multivariate Taylor expansions. For example,

${\displaystyle \operatorname {E} \left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]}}-{\frac {\operatorname {cov} \left[X,Y\right]}{\operatorname {E} \left[Y\right]^{2}}}+{\frac {\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]^{3}}}\operatorname {var} \left[Y\right]}$

Second moment

Similarly,[1]

${\displaystyle \operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}}$

The above is using a first order approximation unlike for the method used in estimating the first moment. It will be a poor approximation in cases where ${\displaystyle f(X)}$ is highly non-linear. This is a special case of the delta method. For example,

${\displaystyle \operatorname {var} \left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {var} \left[X\right]}{\operatorname {E} \left[Y\right]^{2}}}-{\frac {2\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]^{3}}}\operatorname {cov} \left[X,Y\right]+{\frac {\operatorname {E} \left[X\right]^{2}}{\operatorname {E} \left[Y\right]^{4}}}\operatorname {var} \left[Y\right].}$

The second order approximation, when X follows a normal distribution, is:[2]

${\displaystyle \operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]+{\frac {\left(f''(\operatorname {E} \left[X\right])\right)^{2}}{2}}\left(\operatorname {var} \left[X\right]\right)^{2}=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}+{\frac {1}{2}}\left(f''(\mu _{X})\right)^{2}\sigma _{X}^{4}+\left(f'(\mu _{X})\right)\left(f'''(\mu _{X})\right)\sigma _{X}^{4}}$

See also

• Propagation of uncertainty
• WKB approximation
• Delta method

Notes

1. Haym Benaroya, Seon Mi Han, and Mark Nagurka. Probability Models in Engineering and Science. CRC Press, 2005.
2. Hendeby, Gustaf; Gustafsson, Fredrik. "ON NONLINEAR TRANSFORMATIONS OF GAUSSIAN DISTRIBUTIONS" (PDF). Retrieved 5 October 2017.

Further reading

• Wolter, Kirk M. (1985). "Taylor Series Methods". Introduction to Variance Estimation. New York: Springer. pp. 221–247. ISBN 0-387-96119-4.