## Estimating parameters by maximum likelihood and method of moments using mlexp and gmm

\(\newcommand{\epsilonb}{\boldsymbol{\epsilon}}

\newcommand{\ebi}{\boldsymbol{\epsilon}_i}

\newcommand{\Sigmab}{\boldsymbol{\Sigma}}

\newcommand{\Omegab}{\boldsymbol{\Omega}}

\newcommand{\Lambdab}{\boldsymbol{\Lambda}}

\newcommand{\betab}{\boldsymbol{\beta}}

\newcommand{\gammab}{\boldsymbol{\gamma}}

\newcommand{\Gammab}{\boldsymbol{\Gamma}}

\newcommand{\deltab}{\boldsymbol{\delta}}

\newcommand{\xib}{\boldsymbol{\xi}}

\newcommand{\iotab}{\boldsymbol{\iota}}

\newcommand{\xb}{{\bf x}}

\newcommand{\xbit}{{\bf x}_{it}}

\newcommand{\xbi}{{\bf x}_{i}}

\newcommand{\zb}{{\bf z}}

\newcommand{\zbi}{{\bf z}_i}

\newcommand{\wb}{{\bf w}}

\newcommand{\yb}{{\bf y}}

\newcommand{\ub}{{\bf u}}

\newcommand{\Gb}{{\bf G}}

\newcommand{\Hb}{{\bf H}}

\newcommand{\thetab}{\boldsymbol{\theta}}

\newcommand{\XBI}{{\bf x}_{i1},\ldots,{\bf x}_{iT}}

\newcommand{\Sb}{{\bf S}} \newcommand{\Xb}{{\bf X}}

\newcommand{\Xtb}{\tilde{\bf X}}

\newcommand{\Wb}{{\bf W}}

\newcommand{\Ab}{{\bf A}}

\newcommand{\Bb}{{\bf B}}

\newcommand{\Zb}{{\bf Z}}

\newcommand{\Eb}{{\bf E}}\) This post was written jointly with Joerg Luedicke, Senior Social Scientist and Statistician, StataCorp.

**Overview**

We provide an introduction to parameter estimation by maximum likelihood and method of moments using **mlexp** and **gmm**, respectively (see **[R] mlexp** and **[R] gmm**). We include some background about these estimation techniques; see Pawitan (2001, Casella and Berger (2002), Cameron and Trivedi (2005), and Wooldridge (2010) for more details.

Maximum likelihood (ML) estimation finds the parameter values that make the observed data most probable. The parameters maximize the log of the likelihood function that specifies the probability of observing a particular set of data given a model.

Method of moments (MM) estimators specify population moment conditions and find the parameters that solve the equivalent sample moment conditions. MM estimators usually place fewer restrictions on the model than ML estimators, which implies that MM estimators are less efficient but more robust than ML estimators.

**Using mlexp to estimate probit model parameters**

A probit model for the binary dependent variable \(y\) conditional on covariates \(\xb\) with coefficients \(\betab\) is

\[\begin{equation}

y = \begin{cases}

1 & \mbox{ if } \xb\betab’ + \epsilon > 0\\

0 & \mbox{ otherwise }

\end{cases}

\end{equation}\]

where \(\epsilon\) has a standard normal distribution. The log-likelihood function for the probit model is

\[\begin{equation}\label{E:b1}

\ln\{L(\betab;\xb,y)\}= \sum_{i=1}^N y_i \ln\Phi(\xb_{i}\betab’)

+ (1-y_i) \ln\Phi(-\xb_{i}\betab’)

\end{equation}\]

where \(\Phi\) denotes the cumulative standard normal.

We now use **mlexp** to estimate the coefficients of a probit model. We have data on whether an individual belongs to a union (**union**), the individual’s age (**age**), and the highest grade completed (**grade**).

. webuse union (NLS Women 14-24 in 1968) . mlexp ( union*lnnormal({b1}*age + {b2}*grade + {b0}) /// > + (1-union)*lnnormal(-({b1}*age + {b2}*grade + {b0})) ) initial: log likelihood = -18160.456 alternative: log likelihood = -1524604.4 rescale: log likelihood = -14097.135 rescale eq: log likelihood = -14063.38 Iteration 0: log likelihood = -14063.38 Iteration 1: log likelihood = -13796.715 Iteration 2: log likelihood = -13796.336 Iteration 3: log likelihood = -13796.336 Maximum likelihood estimation Log likelihood = -13796.336 Number of obs = 26,200 ------------------------------------------------------------------------------ | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- /b1 | .0051821 .0013471 3.85 0.000 .0025418 .0078224 /b2 | .0373899 .0035814 10.44 0.000 .0303706 .0444092 /b0 | -1.404697 .0587797 -23.90 0.000 -1.519903 -1.289491 ------------------------------------------------------------------------------

Defining a linear combination of the covariates makes it easier to specify the model and to read the output:

. mlexp ( union*lnnormal({xb:age grade _cons}) + (1-union)*lnnormal(-{xb:}) ) initial: log likelihood = -18160.456 alternative: log likelihood = -14355.672 rescale: log likelihood = -14220.454 Iteration 0: log likelihood = -14220.454 Iteration 1: log likelihood = -13797.767 Iteration 2: log likelihood = -13796.336 Iteration 3: log likelihood = -13796.336 Maximum likelihood estimation Log likelihood = -13796.336 Number of obs = 26,200 ------------------------------------------------------------------------------ | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | .0051821 .0013471 3.85 0.000 .0025418 .0078224 grade | .0373899 .0035814 10.44 0.000 .0303706 .0444092 _cons | -1.404697 .0587797 -23.90 0.000 -1.519903 -1.289491 ------------------------------------------------------------------------------

**Using gmm to estimate parameters by MM**

ML specifies a functional form for the distribution of \(y\) conditional on \(\xb\). Specifying \(\Eb[y|\xb]=\Phi(\xb\betab’)\) is less restrictive because it imposes structure only on the first conditional moment instead of on all the conditional moments. Under correct model specification, the ML estimator is more efficient than the MM

estimator because it correctly specifies the conditional mean and all other conditional moments.

The model assumption \(\Eb[y|\xb]=\Phi(\xb\betab’)\) implies the moment conditions \(\Eb[\{y-\Phi(\xb\betab’)\}\xb] = {\bf 0}\). The sample moment equivalent is

\[\sum_{i=1}^N [\{y_i-\Phi(\xb_i\betab’)\}\xb_i] = {\bf 0}\]

In the **gmm** command below, we specify the residuals \(y_i-\Phi(\xb_i\betab’)\) inside the parentheses and the variables that multiply them, known as instruments, in the option **instruments()**.

. gmm ( union - normal({xb:age grade _cons}) ), instruments(age grade) onestep Step 1 Iteration 0: GMM criterion Q(b) = .07831137 Iteration 1: GMM criterion Q(b) = .00004813 Iteration 2: GMM criterion Q(b) = 5.333e-09 Iteration 3: GMM criterion Q(b) = 5.789e-17 note: model is exactly identified GMM estimation Number of parameters = 3 Number of moments = 3 Initial weight matrix: Unadjusted Number of obs = 26,200 ------------------------------------------------------------------------------ | Robust | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | .0051436 .0013349 3.85 0.000 .0025272 .00776 grade | .0383185 .0038331 10.00 0.000 .0308058 .0458312 _cons | -1.415623 .0609043 -23.24 0.000 -1.534994 -1.296253 ------------------------------------------------------------------------------ Instruments for equation 1: age grade _cons

The point estimates are similar to the ML estimates because both estimators are consistent.

**Using gmm to estimate parameters by ML**

When we maximize a log-likelihood function, we find the parameters that set the first derivative to 0. For example, setting the first derivative of the probit log-likelihood function with respect to \(\betab\) to 0 in the sample yields

\[\begin{equation}\label{E:b2}

\frac{\partial \ln\{L(\beta;\xb,y)\}}{\partial \betab} =

\sum_{i=1}^N \left\{y_i \frac{\phi(\xb_{i}\betab’)}{\Phi(\xb_{i}\betab’)}

– (1-y_i) \frac{\phi(-\xb_{i}\betab’)}{\Phi(-\xb_{i}\betab’)}\right\}

\xb_{i} = {\bf 0}

\end{equation}\]

Below, we use **gmm** to find the parameters that solve these sample moment conditions:

. gmm ( union*normalden({xb:age grade _cons})/normal({xb:}) /// > -(1-union)*normalden(-{xb:})/normal(-{xb:}) ), /// > instruments(age grade) onestep Step 1 Iteration 0: GMM criterion Q(b) = .19941827 Iteration 1: GMM criterion Q(b) = .00012506 Iteration 2: GMM criterion Q(b) = 2.260e-09 Iteration 3: GMM criterion Q(b) = 7.369e-19 note: model is exactly identified GMM estimation Number of parameters = 3 Number of moments = 3 Initial weight matrix: Unadjusted Number of obs = 26,200 ------------------------------------------------------------------------------ | Robust | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | .0051821 .001339 3.87 0.000 .0025577 .0078065 grade | .0373899 .0037435 9.99 0.000 .0300528 .044727 _cons | -1.404697 .0601135 -23.37 0.000 -1.522517 -1.286876 ------------------------------------------------------------------------------ Instruments for equation 1: age grade _cons

The point estimates match those reported by **mlexp**. The standard errors differ because **gmm** reports robust standard errors.

**Summary**

We showed how to easily estimate the probit model parameters by ML and by MM using **mlexp** and **gmm**, respectively. We also showed that you can estimate these parameters using restrictions imposed by conditional distributions or using weaker conditional moment restrictions. Finally, we illustrated that the equations imposed by the conditional distributions can be viewed as sample moment restrictions.

**References**

Cameron, A. C., and P. K. Trivedi. 2005. *Microeconometrics Methods and Applications*. 1st ed. New York: Cambridge University Press.

Casella, G., and R. L. Berger. 2002. *Statistical Inference*. 2nd ed. Pacific Grove, CA: Duxbury.

Pawitan, Y. 2001. *In All Likelihood: Statistical Modelling and Inference Using Likelihood*. Oxford: Oxford University Press.

Wooldridge, J. M. 2010. *Econometric Analysis of Cross Section and Panel Data*. 2nd ed. MIT Press.