The development of cut off points for multi dimensional constructs: Comparison of a model based clustering approach to percentile norms, standard scores and normative comparisons

Cut-off scores can be useful guides for researchers, clinicians or respondents in interpretating responses on psychometric instruments. Statistical methods involved in the development of cut-off scores are predominantly based on univariate analysis and in the presence of an observable criterion variable or normative sample data. To date, little is known about statistical procedures in defining multiple cutting scores for several latent variables within a multidimensional construct (i.e., subscales of a questionnaire). This study aims to evaluate the discriminatory power of various univariate and multivariate methods in cutting score development, and to explore a model-based clustering approach in setting cut-off scores for (sub)scales of multidimensional questionnaires. The investigated approach aims to cluster the responses on multidimensional constructs by using model-based clustering to classify respondents into ordered groups. Subsequently, cut-off points were identified as the...Show moreCut-off scores can be useful guides for researchers, clinicians or respondents in interpretating responses on psychometric instruments. Statistical methods involved in the development of cut-off scores are predominantly based on univariate analysis and in the presence of an observable criterion variable or normative sample data. To date, little is known about statistical procedures in defining multiple cutting scores for several latent variables within a multidimensional construct (i.e., subscales of a questionnaire). This study aims to evaluate the discriminatory power of various univariate and multivariate methods in cutting score development, and to explore a model-based clustering approach in setting cut-off scores for (sub)scales of multidimensional questionnaires. The investigated approach aims to cluster the responses on multidimensional constructs by using model-based clustering to classify respondents into ordered groups. Subsequently, cut-off points were identified as the intersection points of kernel density curves between the clusters or as (sub)scale split-rules in predicting cluster membership by the use of classification trees. A simulation study was conducted to compare the methods under medium and large sample sizes, various psychometric circumstances (the magnitude of factor correlations and item loadings), and across simulated homogenous and heterogeneous populations. Results of a simulation study show this model-based cluster approach is often preferred over percentile norms and linear standard scores in terms of discriminatory power, especially in defining two cut-off points. Recommendations for model selection and strategies to obtain cut-off points are given. Future research should focus to integrate the presented methodology into a case study using existing questionnaires to examine substantive implications of the calculated cutting scores. This study gives an overview the commonly known statistical techniques involved in cut-off score attainment, provides a multivariate framework to evaluate the discriminatory power in setting cutting scores, and lays out a promising novel application of model-based clustering in developing cut-off points for multidimensional questionnaires.Show less

Leiden University Student Repository

Documents

In Collections

The development of cut off points for multi dimensional constructs: Comparison of a model based clustering approach to percentile norms, standard scores and normative comparisons