Cross-Loadings in Scale Development: Monte Carlo Study of Structural Item-Total Correlation Analysis with Small Samples

Abstract

This study investigated the possibility of using item correlations with subscales as a tool to diagnose cross-loadings in the scale development process using smaller sample sizes than required for factor analysis. A Monte Carlo simulation using R examined sample sizes from 30-120 under several conditions of correlations, numbers of items (8-36), and numbers of factors (2-3). Within each condition, most items were generated to load on their expected dimensions, but some items were generated that (a) cross-loaded on multiple dimensions or (b) loaded on no dimensions. Based on its consistently larger average loading differences between loading power (loading correctly on the right dimension) and loading error (wrongly loading on an incorrect dimension), combined especially with its consistently lower loading errors, Structural Item-Total Correlation Analysis (SITCA) diagnosed cross-loading and non-loading items most effectively across most of the conditions when sample sizes were approximately 50-60.