## Group comparisons in structural equation models: Testing measurement invariance

When fitting almost any model, we may be interested in investigating whether parameters differ across groups such as time periods, age groups, gender, or school attended. In other words, we may wish to perform tests of moderation when the moderator variable is categorical. For regression models, this can be as simple as including group indicators in the model and interacting them with other predictors.

We naturally have hypotheses regarding differences in parameters across groups when fitting structural equation models as well. When these models involve latent variables and the corresponding observed measurements, we can test whether those measurements are invariant across groups. Evaluation of measurement invariance typically involves a series of tests for equality of measurement coefficients (factor loadings), equality of intercepts, and equality of error variances across groups.

In this post, I demonstrate how to use the **sem** command’s **group()** and **ginvariant()** options as well as the postestimation command **estat ginvariant** to easily perform tests of measurement invariance.

**Measurement invariance example**

I use data from Holzinger and Swineford (1939), which records students’ scores on a number of exams designed to measure different types of abilities. The students in this dataset came from two different schools, the Pasteur school and the Grant-White school, and I want to test for differences across schools. Here I focus on three exams that were intended to measure spatial abilities. I will fit the confirmatory factor model corresponding to the following path diagram and perform a series of tests for measurement invariance. Although this example uses the **sem** command, I could have equivalently drawn this diagram in the Builder and selected group analysis to fit all the models discussed below.

To begin, I fit a model with all parameters estimated separately across groups. There are various ways to set the required identifying constraints that provide a scale and location for the latent variable. Here I set the mean of the Spatial latent variable to 0 and the variance to 1 in both groups.

. sem (Spatial -> visual cubes paper), > variance(Spatial@1) mean(Spatial@0) ginvariant(none) group(school) Endogenous variables Measurement: visual cubes paper Exogenous variables Latent: Spatial Fitting target model: Iteration 0: log likelihood = -2603.5782 Iteration 1: log likelihood = -2603.5782 Structural equation model Number of obs = 301 Grouping variable = school Number of groups = 2 Estimation method = ml Log likelihood = -2603.5782 ( 1) [var(Spatial)]1bn.school = 1 ( 2) [mean(Spatial)]1bn.school = 0 ( 3) [var(Spatial)]2.school = 1 ( 4) [mean(Spatial)]2.school = 0 ------------------------------------------------------------------------------- | OIM | Coef. Std. Err. z P>|z| [95% Conf. Interval] --------------+---------------------------------------------------------------- Measurement | visual <- | Spatial | Pasteur | 4.264065 .8600633 4.96 0.000 2.578372 5.949759 Grant-Wh~e | 5.49895 1.190435 4.62 0.000 3.165739 7.83216 _cons | Pasteur | 29.64744 .5674293 52.25 0.000 28.53529 30.75958 Grant-Wh~e | 29.57931 .5721785 51.70 0.000 28.45786 30.70076 ------------+---------------------------------------------------------------- cubes <- | Spatial | Pasteur | 2.26321 .5214501 4.34 0.000 1.241187 3.285234 Grant-Wh~e | 1.808245 .5031516 3.59 0.000 .8220861 2.794404 _cons | Pasteur | 23.9359 .3927222 60.95 0.000 23.16618 24.70562 Grant-Wh~e | 24.8 .3678649 67.42 0.000 24.079 25.521 ------------+---------------------------------------------------------------- paper <- | Spatial | Pasteur | 1.695466 .3429472 4.94 0.000 1.023302 2.36763 Grant-Wh~e | 1.311235 .3413206 3.84 0.000 .6422592 1.980211 _cons | Pasteur | 14.16026 .227089 62.36 0.000 13.71517 14.60534 Grant-Wh~e | 14.30345 .2335324 61.25 0.000 13.84573 14.76116 --------------+---------------------------------------------------------------- mean(Spatial)| [*] | 0 (constrained) --------------+---------------------------------------------------------------- var(e.visual)| Pasteur | 32.04601 6.912718 20.9971 48.90898 Grant-White | 17.23285 12.18676 4.309258 68.91467 var(e.cubes)| Pasteur | 18.93787 2.710244 14.30585 25.06967 Grant-White | 16.35232 2.318816 12.38443 21.59149 var(e.paper)| Pasteur | 5.170226 1.09911 3.408453 7.842631 Grant-White | 6.188581 .9975804 4.512114 8.487938 var(Spatial)| [*] | 1 (constrained) ------------------------------------------------------------------------------- Note: [*] identifies parameter estimates constrained to be equal across groups. LR test of model vs. saturated: chi2(0) = 0.00, Prob > chi2 = .

Glancing through this output, we see that many of the parameter estimates are very similar for the two schools. The **estat ginvariant** command provides tests of invariance across groups.

. estat ginvariant, showpclass(mcoef) class Tests for group invariance of parameters ------------------------------------------------------------------------------ | Wald Test Score Test | chi2 df p>chi2 chi2 df p>chi2 -------------+---------------------------------------------------------------- Measurement | visual <- | Spatial | 0.707 1 0.4004 . . . -----------+---------------------------------------------------------------- cubes <- | Spatial | 0.394 1 0.5301 . . . -----------+---------------------------------------------------------------- paper <- | Spatial | 0.631 1 0.4271 . . . ------------------------------------------------------------------------------ Joint tests for each parameter class ------------------------------------------------------------------------------ | Wald Test Score Test | chi2 df p>chi2 chi2 df p>chi2 -------------+---------------------------------------------------------------- mcoef | 1.097 3 0.7778 . . . ------------------------------------------------------------------------------

The **showpclass(mcoef)** and **class** options restricted the results to tests regarding measurement coefficients and requested a joint test for the hypothesis that all measurement coefficients are equal across groups. The first table in the output reports separate tests of equality of the measurement coefficients across groups. My focus now, however, is on the joint Wald test shown in the second table, and we fail to reject the hypothesis of equality across groups for all measurement coefficients.

I now include the **ginvariant(mcoef)** option in order to fit a model with the measurement coefficients constrained to be equal across groups by typing

. sem (Spatial -> visual cubes paper), variance(Spatial@1) /// mean(Spatial@0) ginvariant(mcoef) group(school)

and then test whether the intercepts can be constrained:

. estat ginvariant, showpclass(mcons) class Tests for group invariance of parameters ------------------------------------------------------------------------------ | Wald Test Score Test | chi2 df p>chi2 chi2 df p>chi2 -------------+---------------------------------------------------------------- Measurement | visual <- | _cons | 0.007 1 0.9326 . . . -----------+---------------------------------------------------------------- cubes <- | _cons | 2.580 1 0.1082 . . . -----------+---------------------------------------------------------------- paper <- | _cons | 0.193 1 0.6605 . . . ------------------------------------------------------------------------------ Joint tests for each parameter class ------------------------------------------------------------------------------ | Wald Test Score Test | chi2 df p>chi2 chi2 df p>chi2 -------------+---------------------------------------------------------------- mcons | 3.011 3 0.3900 . . . ------------------------------------------------------------------------------

We fail to reject the null hypothesis that all intercepts are equal across groups, so I fit the model with those equality constraints by specifying the **ginvariant(mcoef mcons)** option.

. sem (Spatial -> visual cubes paper), variance(Spatial@1) /// mean(Spatial@0) ginvariant(mcoef mcons) group(school)

Then, I test the equality of the error variances.

. estat ginvariant, showpclass(merrvar) class Tests for group invariance of parameters ------------------------------------------------------------------------------ | Wald Test Score Test | chi2 df p>chi2 chi2 df p>chi2 -------------+---------------------------------------------------------------- var(e.visual)| 0.359 1 0.5493 . . . var(e.cubes)| 1.413 1 0.2345 . . . var(e.paper)| 0.014 1 0.9052 . . . ------------------------------------------------------------------------------ Joint tests for each parameter class ------------------------------------------------------------------------------ | Wald Test Score Test | chi2 df p>chi2 chi2 df p>chi2 -------------+---------------------------------------------------------------- merrvar | 1.857 3 0.6027 . . . ------------------------------------------------------------------------------

Once again, we fail to reject the null hypothesis of invariance across groups. I now impose constraints on the coefficients, intercepts, and error variances while allowing the mean and variance of the latent variable to differ across groups. To do this, I remove the **mean(Spatial@0)** option and replace the **variance(Spatial@1)** with **variance(1:Spatial@1)**. With this change, the mean and variance of Spatial will be set to 0 and 1, respectively, in the first group but estimated freely in the second group.

. sem (Spatial -> visual cubes paper), > variance(1:Spatial@1) ginvariant(mcoef mcons merrvar) group(school) Endogenous variables Measurement: visual cubes paper Exogenous variables Latent: Spatial Fitting target model: Iteration 0: log likelihood = -5357.6935 (not concave) Iteration 1: log likelihood = -4792.5814 (not concave) Iteration 2: log likelihood = -4316.3827 (not concave) Iteration 3: log likelihood = -2769.069 (not concave) Iteration 4: log likelihood = -2662.2605 Iteration 5: log likelihood = -2645.7652 Iteration 6: log likelihood = -2629.1987 Iteration 7: log likelihood = -2622.83 (not concave) Iteration 8: log likelihood = -2622.3555 Iteration 9: log likelihood = -2622.3227 Iteration 10: log likelihood = -2621.9007 Iteration 11: log likelihood = -2621.8931 Iteration 12: log likelihood = -2621.893 Structural equation model Number of obs = 301 Grouping variable = school Number of groups = 2 Estimation method = ml Log likelihood = -2621.893 ( 1) [cubes]1bn.school#c.Spatial - [cubes]2.school#c.Spatial = 0 ( 2) [paper]1bn.school#c.Spatial - [paper]2.school#c.Spatial = 0 ( 3) [var(e.visual)]1bn.school - [var(e.visual)]2.school = 0 ( 4) [var(e.cubes)]1bn.school - [var(e.cubes)]2.school = 0 ( 5) [var(e.paper)]1bn.school - [var(e.paper)]2.school = 0 ( 6) [var(Spatial)]1bn.school = 1 ( 7) [visual]1bn.school - [visual]2.school = 0 ( 8) [cubes]1bn.school - [cubes]2.school = 0 ( 9) [paper]1bn.school - [paper]2.school = 0 (10) [visual]2.school#c.Spatial = 1 (11) [mean(Spatial)]1bn.school = 0 ------------------------------------------------------------------------------- | OIM | Coef. Std. Err. z P>|z| [95% Conf. Interval] --------------+---------------------------------------------------------------- Measurement | visual <- | Spatial | Pasteur | 5.472561 1.129916 4.84 0.000 3.257966 7.687156 Grant-Wh~e | 1 (constrained) _cons | [*] | 29.32102 .4932735 59.44 0.000 28.35422 30.28782 ------------+---------------------------------------------------------------- cubes <- | Spatial | [*] | .3968564 .1833049 2.17 0.030 .0375854 .7561274 _cons | [*] | 24.26618 .2890016 83.97 0.000 23.69975 24.83262 ------------+---------------------------------------------------------------- paper <- | Spatial | [*] | .2953686 .137265 2.15 0.031 .0263341 .5644031 _cons | [*] | 14.16525 .1786194 79.30 0.000 13.81516 14.51533 --------------+---------------------------------------------------------------- mean(Spatial)| Pasteur | 0 (constrained) Grant-White | .4140109 .6928933 0.60 0.550 -.9440351 1.772057 --------------+---------------------------------------------------------------- var(e.visual)| [*] | 19.50062 12.09195 5.784095 65.74481 var(e.cubes)| [*] | 20.08682 1.784905 16.87617 23.90829 var(e.paper)| [*] | 6.864085 .691005 5.634982 8.361281 var(Spatial)| Pasteur | 1 (constrained) Grant-White | 25.44848 15.33031 7.814351 82.87636 ------------------------------------------------------------------------------- Note: [*] identifies parameter estimates constrained to be equal across groups. LR test of model vs. saturated: chi2(7) = 36.63, Prob > chi2 = 0.0000

The mean of 0.414 for Spatial in the Grant-White school represents the difference in means of this latent variable across schools, and we find the difference in means across schools is not significantly different from 0.

**Summary**

Tests of hypotheses regarding the equality of parameters across groups are easily performed using the **sem** command and **estat ginvariant**. While there are minor variations throughout structural equation modeling literature in recommendations for setting identifying constraints and for the order of tests for invariance, the tools that I have demonstrated can be adapted to accommodate any form of tests for measurement invariance. These same tools can also be used to test for parameter invariance across groups in other types of structural equation models.

**Reference**

Holzinger, K.~J., and F. Swineford. 1939. A study in factor analysis: The stability of a bi-factor solution. *Supplementary Educational Monographs, 48*. Chicago, IL: University of Chicago.