Home > Statistics > Group comparisons in structural equation models: Testing measurement invariance

Group comparisons in structural equation models: Testing measurement invariance

When fitting almost any model, we may be interested in investigating whether parameters differ across groups such as time periods, age groups, gender, or school attended. In other words, we may wish to perform tests of moderation when the moderator variable is categorical. For regression models, this can be as simple as including group indicators in the model and interacting them with other predictors.

We naturally have hypotheses regarding differences in parameters across groups when fitting structural equation models as well. When these models involve latent variables and the corresponding observed measurements, we can test whether those measurements are invariant across groups. Evaluation of measurement invariance typically involves a series of tests for equality of measurement coefficients (factor loadings), equality of intercepts, and equality of error variances across groups.

In this post, I demonstrate how to use the sem command’s group() and ginvariant() options as well as the postestimation command estat ginvariant to easily perform tests of measurement invariance.

Measurement invariance example

I use data from Holzinger and Swineford (1939), which records students’ scores on a number of exams designed to measure different types of abilities. The students in this dataset came from two different schools, the Pasteur school and the Grant-White school, and I want to test for differences across schools. Here I focus on three exams that were intended to measure spatial abilities. I will fit the confirmatory factor model corresponding to the following path diagram and perform a series of tests for measurement invariance. Although this example uses the sem command, I could have equivalently drawn this diagram in the Builder and selected group analysis to fit all the models discussed below.

graph1

To begin, I fit a model with all parameters estimated separately across groups. There are various ways to set the required identifying constraints that provide a scale and location for the latent variable. Here I set the mean of the Spatial latent variable to 0 and the variance to 1 in both groups.

. sem (Spatial -> visual cubes paper), 
> variance(Spatial@1) mean(Spatial@0) ginvariant(none) group(school)

Endogenous variables

Measurement:  visual cubes paper

Exogenous variables

Latent:       Spatial

Fitting target model:

Iteration 0:   log likelihood = -2603.5782
Iteration 1:   log likelihood = -2603.5782

Structural equation model                       Number of obs     =        301
Grouping variable  = school                     Number of groups  =          2
Estimation method  = ml
Log likelihood     = -2603.5782

 ( 1)  [var(Spatial)]1bn.school = 1
 ( 2)  [mean(Spatial)]1bn.school = 0
 ( 3)  [var(Spatial)]2.school = 1
 ( 4)  [mean(Spatial)]2.school = 0
-------------------------------------------------------------------------------
              |                 OIM
              |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
Measurement   |
  visual <-   |
      Spatial |
     Pasteur  |   4.264065   .8600633     4.96   0.000     2.578372    5.949759
  Grant-Wh~e  |    5.49895   1.190435     4.62   0.000     3.165739     7.83216
        _cons |
     Pasteur  |   29.64744   .5674293    52.25   0.000     28.53529    30.75958
  Grant-Wh~e  |   29.57931   .5721785    51.70   0.000     28.45786    30.70076
  ------------+----------------------------------------------------------------
  cubes <-    |
      Spatial |
     Pasteur  |    2.26321   .5214501     4.34   0.000     1.241187    3.285234
  Grant-Wh~e  |   1.808245   .5031516     3.59   0.000     .8220861    2.794404
        _cons |
     Pasteur  |    23.9359   .3927222    60.95   0.000     23.16618    24.70562
  Grant-Wh~e  |       24.8   .3678649    67.42   0.000       24.079      25.521
  ------------+----------------------------------------------------------------
  paper <-    |
      Spatial |
     Pasteur  |   1.695466   .3429472     4.94   0.000     1.023302     2.36763
  Grant-Wh~e  |   1.311235   .3413206     3.84   0.000     .6422592    1.980211
        _cons |
     Pasteur  |   14.16026    .227089    62.36   0.000     13.71517    14.60534
  Grant-Wh~e  |   14.30345   .2335324    61.25   0.000     13.84573    14.76116
--------------+----------------------------------------------------------------
 mean(Spatial)|
          [*] |          0  (constrained)
--------------+----------------------------------------------------------------
 var(e.visual)|
     Pasteur  |   32.04601   6.912718                       20.9971    48.90898
 Grant-White  |   17.23285   12.18676                      4.309258    68.91467
  var(e.cubes)|
     Pasteur  |   18.93787   2.710244                      14.30585    25.06967
 Grant-White  |   16.35232   2.318816                      12.38443    21.59149
  var(e.paper)|
     Pasteur  |   5.170226    1.09911                      3.408453    7.842631
 Grant-White  |   6.188581   .9975804                      4.512114    8.487938
  var(Spatial)|
          [*] |          1  (constrained)
-------------------------------------------------------------------------------
Note: [*] identifies parameter estimates constrained to be equal across groups.
LR test of model vs. saturated: chi2(0)   =      0.00, Prob > chi2 =      .

Glancing through this output, we see that many of the parameter estimates are very similar for the two schools. The estat ginvariant command provides tests of invariance across groups.

. estat ginvariant, showpclass(mcoef) class

Tests for group invariance of parameters

------------------------------------------------------------------------------
             |            Wald Test                       Score Test
             |      chi2         df    p>chi2       chi2          df    p>chi2
-------------+----------------------------------------------------------------
Measurement  |
  visual <-  |
     Spatial |     0.707          1    0.4004          .           .         .
  -----------+----------------------------------------------------------------
  cubes <-   |
     Spatial |     0.394          1    0.5301          .           .         .
  -----------+----------------------------------------------------------------
  paper <-   |
     Spatial |     0.631          1    0.4271          .           .         .
------------------------------------------------------------------------------

Joint tests for each parameter class

------------------------------------------------------------------------------
             |            Wald Test                       Score Test
             |      chi2         df    p>chi2       chi2          df    p>chi2
-------------+----------------------------------------------------------------
       mcoef |     1.097          3    0.7778          .           .         .
------------------------------------------------------------------------------

The showpclass(mcoef) and class options restricted the results to tests regarding measurement coefficients and requested a joint test for the hypothesis that all measurement coefficients are equal across groups. The first table in the output reports separate tests of equality of the measurement coefficients across groups. My focus now, however, is on the joint Wald test shown in the second table, and we fail to reject the hypothesis of equality across groups for all measurement coefficients.

I now include the ginvariant(mcoef) option in order to fit a model with the measurement coefficients constrained to be equal across groups by typing

. sem (Spatial -> visual cubes paper), variance(Spatial@1) /// 
       mean(Spatial@0) ginvariant(mcoef) group(school)

and then test whether the intercepts can be constrained:

. estat ginvariant, showpclass(mcons) class

Tests for group invariance of parameters

------------------------------------------------------------------------------
             |            Wald Test                       Score Test
             |      chi2         df    p>chi2       chi2          df    p>chi2
-------------+----------------------------------------------------------------
Measurement  |
  visual <-  |
       _cons |     0.007          1    0.9326          .           .         .
  -----------+----------------------------------------------------------------
  cubes <-   |
       _cons |     2.580          1    0.1082          .           .         .
  -----------+----------------------------------------------------------------
  paper <-   |
       _cons |     0.193          1    0.6605          .           .         .
------------------------------------------------------------------------------

Joint tests for each parameter class

------------------------------------------------------------------------------
             |            Wald Test                       Score Test
             |      chi2         df    p>chi2       chi2          df    p>chi2
-------------+----------------------------------------------------------------
       mcons |     3.011          3    0.3900          .           .         .
------------------------------------------------------------------------------

We fail to reject the null hypothesis that all intercepts are equal across groups, so I fit the model with those equality constraints by specifying the ginvariant(mcoef mcons) option.

. sem (Spatial -> visual cubes paper), variance(Spatial@1) /// 
      mean(Spatial@0) ginvariant(mcoef mcons) group(school)

Then, I test the equality of the error variances.

. estat ginvariant, showpclass(merrvar) class

Tests for group invariance of parameters

------------------------------------------------------------------------------
             |            Wald Test                       Score Test
             |      chi2         df    p>chi2       chi2          df    p>chi2
-------------+----------------------------------------------------------------
var(e.visual)|     0.359          1    0.5493          .           .         .
 var(e.cubes)|     1.413          1    0.2345          .           .         .
 var(e.paper)|     0.014          1    0.9052          .           .         .
------------------------------------------------------------------------------

Joint tests for each parameter class

------------------------------------------------------------------------------
             |            Wald Test                       Score Test
             |      chi2         df    p>chi2       chi2          df    p>chi2
-------------+----------------------------------------------------------------
     merrvar |     1.857          3    0.6027          .           .         .
------------------------------------------------------------------------------

Once again, we fail to reject the null hypothesis of invariance across groups. I now impose constraints on the coefficients, intercepts, and error variances while allowing the mean and variance of the latent variable to differ across groups. To do this, I remove the mean(Spatial@0) option and replace the variance(Spatial@1) with variance(1:Spatial@1). With this change, the mean and variance of Spatial will be set to 0 and 1, respectively, in the first group but estimated freely in the second group.

. sem (Spatial -> visual cubes paper), 
> variance(1:Spatial@1) ginvariant(mcoef mcons merrvar) group(school)

Endogenous variables

Measurement:  visual cubes paper

Exogenous variables

Latent:       Spatial

Fitting target model:

Iteration 0:   log likelihood = -5357.6935  (not concave)
Iteration 1:   log likelihood = -4792.5814  (not concave)
Iteration 2:   log likelihood = -4316.3827  (not concave)
Iteration 3:   log likelihood =  -2769.069  (not concave)
Iteration 4:   log likelihood = -2662.2605
Iteration 5:   log likelihood = -2645.7652
Iteration 6:   log likelihood = -2629.1987
Iteration 7:   log likelihood =   -2622.83  (not concave)
Iteration 8:   log likelihood = -2622.3555
Iteration 9:   log likelihood = -2622.3227
Iteration 10:  log likelihood = -2621.9007
Iteration 11:  log likelihood = -2621.8931
Iteration 12:  log likelihood =  -2621.893

Structural equation model                       Number of obs     =        301
Grouping variable  = school                     Number of groups  =          2
Estimation method  = ml
Log likelihood     =  -2621.893

 ( 1)  [cubes]1bn.school#c.Spatial - [cubes]2.school#c.Spatial = 0
 ( 2)  [paper]1bn.school#c.Spatial - [paper]2.school#c.Spatial = 0
 ( 3)  [var(e.visual)]1bn.school - [var(e.visual)]2.school = 0
 ( 4)  [var(e.cubes)]1bn.school - [var(e.cubes)]2.school = 0
 ( 5)  [var(e.paper)]1bn.school - [var(e.paper)]2.school = 0
 ( 6)  [var(Spatial)]1bn.school = 1
 ( 7)  [visual]1bn.school - [visual]2.school = 0
 ( 8)  [cubes]1bn.school - [cubes]2.school = 0
 ( 9)  [paper]1bn.school - [paper]2.school = 0
 (10)  [visual]2.school#c.Spatial = 1
 (11)  [mean(Spatial)]1bn.school = 0
-------------------------------------------------------------------------------
              |                 OIM
              |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
Measurement   |
  visual <-   |
      Spatial |
     Pasteur  |   5.472561   1.129916     4.84   0.000     3.257966    7.687156
  Grant-Wh~e  |          1  (constrained)
        _cons |
          [*] |   29.32102   .4932735    59.44   0.000     28.35422    30.28782
  ------------+----------------------------------------------------------------
  cubes <-    |
      Spatial |
          [*] |   .3968564   .1833049     2.17   0.030     .0375854    .7561274
        _cons |
          [*] |   24.26618   .2890016    83.97   0.000     23.69975    24.83262
  ------------+----------------------------------------------------------------
  paper <-    |
      Spatial |
          [*] |   .2953686    .137265     2.15   0.031     .0263341    .5644031
        _cons |
          [*] |   14.16525   .1786194    79.30   0.000     13.81516    14.51533
--------------+----------------------------------------------------------------
 mean(Spatial)|
     Pasteur  |          0  (constrained)
 Grant-White  |   .4140109   .6928933     0.60   0.550    -.9440351    1.772057
--------------+----------------------------------------------------------------
 var(e.visual)|
          [*] |   19.50062   12.09195                      5.784095    65.74481
  var(e.cubes)|
          [*] |   20.08682   1.784905                      16.87617    23.90829
  var(e.paper)|
          [*] |   6.864085    .691005                      5.634982    8.361281
  var(Spatial)|
     Pasteur  |          1  (constrained)
 Grant-White  |   25.44848   15.33031                      7.814351    82.87636
-------------------------------------------------------------------------------
Note: [*] identifies parameter estimates constrained to be equal across groups.
LR test of model vs. saturated: chi2(7)   =     36.63, Prob > chi2 = 0.0000

The mean of 0.414 for Spatial in the Grant-White school represents the difference in means of this latent variable across schools, and we find the difference in means across schools is not significantly different from 0.

Summary

Tests of hypotheses regarding the equality of parameters across groups are easily performed using the sem command and estat ginvariant. While there are minor variations throughout structural equation modeling literature in recommendations for setting identifying constraints and for the order of tests for invariance, the tools that I have demonstrated can be adapted to accommodate any form of tests for measurement invariance. These same tools can also be used to test for parameter invariance across groups in other types of structural equation models.

Reference

Holzinger, K.~J., and F. Swineford. 1939. A study in factor analysis: The stability of a bi-factor solution. Supplementary Educational Monographs, 48. Chicago, IL: University of Chicago.

  • Patrick

    Hello Kristin,

    Thank you for this helpful blog.
    I am trying to examine longitudinal measurement invariance. I have a large number of individuals that have responded to the same questionnaire at 15 different cycles. I want to know if the questionnaire has measured the same thing at those 15 different cycles. I am trying to see if I can adapt your code to fit my needs. My data is in long format and when I use the command group(cycle), Stata gives me an error r(2000) no observations. Which makes me think that the group() command is not meant for data in long format (Am I right?). Can you point me to another resource that might be helpful?
    Thank you for your time