Optimal instruments

In statistics and econometrics, optimal instruments are a technique for improving the efficiency of estimators in conditional moment models, a class of semiparametric models that generate conditional expectation functions. To estimate parameters of a conditional moment model, the statistician can derive an expectation function (defining "moment conditions") and use the generalized method of moments (GMM). However, there are infinitely many moment conditions that can be generated from a single model; optimal instruments provide the most efficient moment conditions.

As an example, consider the nonlinear regression model

${\displaystyle y=f(x,\theta )+u}$

${\displaystyle E[u\mid x]=0}$

where y is a scalar (one-dimensional) random variable, x is a random vector with dimension k, and θ is a k-dimensional parameter. The conditional moment restriction ${\displaystyle E[u\mid x]=0}$ is consistent with infinitely many moment conditions. For example:

${\displaystyle E[ux]=E[ux^{2}]=E[ux^{3}]=\dots =0}$

More generally, for any vector-valued function z of x, it will be the case that

${\displaystyle E[z(x)(y-f(x,\theta ))]=0}$.

That is, z defines a finite set of orthogonality conditions.

A natural question to ask, then, is whether an asymptotically efficient set of conditions is available, in the sense that no other set of conditions achieves lower asymptotic variance.[1] Both econometricians[2][3] and statisticians[4] have extensively studied this subject.

The answer to this question is generally that this finite set exists and have been proven for a wide range of estimators. Takeshi Amemiya was one of the first to work on this problem and show the optimal number of instruments for nonlinear simultaneous equation models with homoskedastic and serially uncorrelated errors.[5] The form of the optimal instruments was characterized by Lars Peter Hansen,[6] and results for nonparametric estimation of optimal instruments are provided by Newey.[7] A result for nearest neighbor estimators was provided by Robinson.[8]

In linear regression

The technique of optimal instruments can be used to show that, in a conditional moment linear regression model with iid data, the optimal GMM estimator is generalized least squares. Consider the model

${\displaystyle y=x^{\mathrm {T} }\theta +u}$

${\displaystyle E[u\mid x]=0}$

where y is a scalar random variable, x is a k-dimensional random vector, and θ is a k-dimensional parameter vector. As above, the moment conditions are

${\displaystyle E[z(x)(y-x^{\mathrm {T} }\theta )]=0}$

where z = z(x) is an instrument set of dimension p (p ≥ k). The task is to choose z to minimize the asymptotic variance of the resulting GMM estimator. If the data are iid, the asymptotic variance of the GMM estimator is

${\displaystyle (E[xz^{\mathrm {T} }]^{\mathrm {T} }E[\sigma ^{2}(x)zz^{\mathrm {T} }]^{-1}E[zx^{\mathrm {T} }])^{-1}}$

where ${\displaystyle \sigma ^{2}(x)\equiv E[u^{2}\mid x]}$.

The optimal instruments are given by

${\displaystyle z^{*}(x)={\frac {x}{\sigma ^{2}(x)}}}$

which produces the asymptotic variance matrix

${\displaystyle \left(E\left[{\frac {xx^{\mathrm {T} }}{\sigma ^{2}(x)}}\right]\right)^{-1}}$.

These are the optimal instruments because for any other z, the matrix

${\displaystyle \left(E\left[{\frac {xx^{\mathrm {T} }}{\sigma ^{2}(x)}}\right]\right)^{-1}-(E[xz^{\mathrm {T} }]^{\mathrm {T} }E[\sigma ^{2}(x)zz^{\mathrm {T} }]^{-1}E[zx^{\mathrm {T} }])^{-1}}$

is positive semidefinite.

Given iid data ${\displaystyle (y_{1},x_{1}),\dots ,(y_{N},x_{N})}$, the GMM estimator corresponding to ${\displaystyle z^{*}(x)}$ is

${\displaystyle {\widetilde {\theta }}=\left(\sum _{i=1}^{N}{\frac {x_{i}x_{i}^{\mathrm {T} }}{\sigma ^{2}(x_{i})}}\right)^{-1}\sum _{i=1}^{N}{\frac {x_{i}y_{i}}{\sigma ^{2}(x_{i})}}}$

which is the generalized least squares estimator. (It is unfeasible because σ²(·) is unknown.)[1]

References

1. Arellano, M. (2009). "Generalized Method of Moments and Optimal Instruments" (PDF). Class notes.
2. Chamberlain, G. (1987). "Asymptotic Efficiency in Estimation with Conditional Moment Restrictions". Journal of Econometrics. 34 (3): 305–334. doi:10.1016/0304-4076(87)90015-7.
3. Newey, W. K. (1988). "Adaptive Estimation of Regression Models via Moment Restrictions". Journal of Econometrics. 38 (3): 301–339. doi:10.1016/0304-4076(88)90048-6.
4. Liang, K-Y.; Zeger, S. L. (1986). "Longitudinal Data Analysis using Generalized Linear Models". Biometrika. 73 (1): 13–22. doi:10.1093/biomet/73.1.13.
5. Amemiya, T. (1977). "The Maximum Likelihood and the Nonlinear Three-Stage Least Squares Estimator in the General Nonlinear Simultaneous Equation Model". Econometrica. 45 (4): 955–968. doi:10.2307/1912684. JSTOR 1912684.
6. Hansen, L. P. (1985). "A Method of Calculating Bounds on the Asymptotic Covariance Matrices of Generalized Method of Moments Estimators". Journal of Econometrics. 30 (1–2): 203–238. doi:10.1016/0304-4076(85)90138-1.
7. Newey, W. K. (1990). "Efficient Instrumental Variables Estimation of Nonlinear Models". Econometrica. 58 (4): 809–837. doi:10.2307/2938351. JSTOR 2938351.
8. Robinson, P. (1987). "Asymptotically Efficient Estimation in the Presence of Heteroskedasticity of Unknown Form". Econometrica. 55 (4): 875–891. doi:10.2307/1911033. JSTOR 1911033.

Further reading

• Tsiatis, A. A. (2006). Semiparametric Theory and Missing Data. Springer Series in Statistics. New York: Springer. ISBN 0-387-32448-8.