Fit Check: A Simulation Study on the Sensitivity of Fit Indices in Scalar Invariance Assessment

Scalar invariance is essential for ensuring the validity of latent construct comparisons across different groups. Fit indices are statistics used to assess the goodness of fit, evaluating the equivalency of constructs across groups under invariance constraints on loadings and intercepts for scalar invariance. The focus of this simulation study was on the sensitivity of the change in commonly used fit indices which are Chi-square Goodness of Fit (Δ𝜒2), Root Mean Square Error of Approximation (ΔRMSEA), and Comparative Fit Index (ΔCFI) in assessing scalar invariance through Multi-group Confirmatory Factor Analysis (MG-CFA). The aim was to identify the impact of sample size, magnitude levels of noninvariance in intercept values, and ratios of noninvariant items and the interactions of these conditions on the outcomes yielded by these fit indices. Results from multi-way analysis of variance revealed significant effects for all variables and their interactions on change in Chi-square...Show moreScalar invariance is essential for ensuring the validity of latent construct comparisons across different groups. Fit indices are statistics used to assess the goodness of fit, evaluating the equivalency of constructs across groups under invariance constraints on loadings and intercepts for scalar invariance. The focus of this simulation study was on the sensitivity of the change in commonly used fit indices which are Chi-square Goodness of Fit (Δ𝜒2), Root Mean Square Error of Approximation (ΔRMSEA), and Comparative Fit Index (ΔCFI) in assessing scalar invariance through Multi-group Confirmatory Factor Analysis (MG-CFA). The aim was to identify the impact of sample size, magnitude levels of noninvariance in intercept values, and ratios of noninvariant items and the interactions of these conditions on the outcomes yielded by these fit indices. Results from multi-way analysis of variance revealed significant effects for all variables and their interactions on change in Chi-square values. Large effect sizes were observed for sample size, magnitude of noninvariance, and their interactions, as indicated by partial eta squared. Specifically, larger sample sizes and higher levels of noninvariance were associated with larger changes in Chi-square values, suggesting decreased model fit. The effect size of sample size on ΔRMSEA was medium, with larger sample sizes leading to increased ΔRMSEA values, while the effect size of magnitude levels of noninvariance was large on the change in RMSEA, with higher levels of noninvariance also leading to increased ΔRMSEA values. Additionally, the effect size of magnitude levels of noninvariance was large on ΔCFI, with higher levels of noninvariance leading to decreased ΔCFI values. ΔRMSEA displayed increased sensitivity to these differences with larger sample sizes, while there was a decline in the sensitivity of ΔCFI.Show less

Leiden University Student Repository

Documents

In Collections

Fit Check: A Simulation Study on the Sensitivity of Fit Indices in Scalar Invariance Assessment