Growth mixture models (GMMs) can be used to uncover the heterogeneity in individual growth trajectories, using the discrete latent class variable to predict between class differences of growth...Show moreGrowth mixture models (GMMs) can be used to uncover the heterogeneity in individual growth trajectories, using the discrete latent class variable to predict between class differences of growth patterns in the continuous growth factors. GMMs can incorporate the distal outcomes, defined as the consequence of individuals in latent classes, allowing researchers to explore the consequence of the growth trajectories. Prior work on continuous latent growth variables as predictors of the distal outcome has shown that this prediction is sensitive to the time coding choices in the latent curve models (LCMs). However, the GMM allows categorical latent class variables to directly predict the distal outcome rather than the continuous latent growth factors in the LCMs, and may be less susceptible to the issues in the continuous case. Here, we examine the effect of the time coding approaches on the predictive relationship between categorical latent variables and the distal outcomes, and explore the role of estimation approaches, measurement quality, latent class proportions and sample sizes in determining the reliable recovery of the distal outcome and the latent classes. The results suggest that, the one-step approach, is robust across simulation conditions despite yielding downward biased estimates in standard error, but that the two-step approach results convergence issues, biased parameter recovery, and biased class proportions when the measurement model is weak. We demonstrate that similar issue occurs in both simulated and empirical data. In sum, there is no evidence for an influence of time coding choices on predicting the distal outcomes in the GMMs.Show less
Most neuroimaging studies are affected by small sample sizes and poor reproducibility of research findings. Therefore, aggregating data from multiple research centres is crucial to the development...Show moreMost neuroimaging studies are affected by small sample sizes and poor reproducibility of research findings. Therefore, aggregating data from multiple research centres is crucial to the development of the neuroimaging field. For this reason, MRI site harmonization is essential, as it allows for comparison and joint analysis of MRI data from multiple studies. MRI site harmonization aims to remove inter-site variability, while maintaining variance of interest. However, neuroimaging studies generally have low numbers of subjects to estimate the harmonization model. This paper examines the effect of dataset size on the quality of MRI site harmonization, and whether this effect is dependent on age differences between sites and the size of site differences. In order to evaluate the quality of MRI site harmonization we calculated the extent to which the correlation between GMD and age was recovered. To answer our research questions, we studied the performance of MRI site harmonization using a variety of training dataset sizes in an empirical study. Our empirical study shows no clear effect of the size of the training dataset. In addition, we studied the performance of MRI site harmonization in a simulation study, where we varied the number of subjects in the training dataset, the age differences between the centres, and the size of the centre effects. Our simulation study shows that the effect of training dataset size is minimal. The effect is only present when sites differ largely in mean age and when site effects are small. Thus, in all other conditions, inter-site variability is successfully removed, while variance of interest is preserved. This leads us to the conclusion that the limited effect of training dataset size suggests that prospects for the quality of harmonization in multi-centre studies with small datasets are promising.Show less
Psychological science has been plagued by a history of studies with low statistical power. The common solution for this issue has been to increase the sample sizes of studies but this also...Show morePsychological science has been plagued by a history of studies with low statistical power. The common solution for this issue has been to increase the sample sizes of studies but this also increases time and money spent. Another solution that has been mostly ignored by the scientific community is the use of sequential procedures for statistical analysis. Sequential procedures allow the researcher to analyse the data multiple times as new participants are recruited. So, several analyses are done until the null hypothesis is either accepted or rejected. In practice, this might lead to efficiency gains in the form of smaller sample sizes. The statistical properties of sequential procedures have been found to be adequate based on simulation studies. However, there is a lack of information about the efficiency gains and consistency of these procedures with actual psychological data. The current study aims to evaluate the efficiency and consistency of the group sequential (GS) procedure and the sequential probability ratio test (SPRT) with real data. Efficiency is the reduction in sample sizes while consistency refers to (in)congruence between the decision based on the sequential procedure and the conventional procedure. We re-analysed an existing multi-lab registered replication report (RRR) namely the study by Verschuere et al. (2018) using the GS procedure and SPRT procedure. Using these procedures, efficiency gains ranging from 27% to 62% were found. The GS procedure outperformed the SPRT in terms of efficiency. In the labs that showed efficiency gains, the decision regarding the null hypothesis was the same compared to the decision based on the conventional procedure. Taken together, the sequential procedures yielded the same overall conclusion as Verschuere et al., but at a considerably smaller sample size. This shows the potential of these procedures for psychological science as a whole.Show less
Clustering algorithms are important for data mining, and K-means is one of the most well-known clustering algorithms currently available. In cases in which data are high-dimensional, however, mere...Show moreClustering algorithms are important for data mining, and K-means is one of the most well-known clustering algorithms currently available. In cases in which data are high-dimensional, however, mere application of K-means to a data set may fail to uncover clusters due to presence of masking variables, the curse of dimensionality, and difficulties in interpretation of the obtained solution. A commonly used work-around is to apply dimension reduction to the data prior to performing cluster analysis, a practice called Tandem Analysis (TA). A vulnerability of TA is that the applied dimension reduction is not guaranteed to preserve cluster structure present in the original data, jeopardising the usefulness of subsequent cluster analysis. Multiple authors have provided algorithms that reduce dimensionality of a data set and perform cluster analysis on the reduced data, either in a sequential fashion or a simultaneous fashion, all aiming to find suitable low-dimensional representations of data while also keeping cluster structures intact. In this thesis, a novel approach to reducing dimensionality and performing cluster analysis on the low dimensional representation of the data - called SICA - is described and thoroughly tested in two systematically manipulated simulation studies and applied to three empirical data applications. Results show that SICA is a computationally efficient algorithm well able to extract components from the original data that preserve cluster structures, but that performance depends on characteristics of the data and the model of data generation. In addition, the correctness and validity of the clusterings obtained through SICA is high, although it does not always outperform currently available methods in this regard and is dependent on the same characteristics of the data and model generation as the other algorithms. Limitations and implications for future research are discussed.Show less
Despite the growing popularity, no clear general definition of data science and artificial intelligence has been established. People are often left into the unknown when it comes to the specific...Show moreDespite the growing popularity, no clear general definition of data science and artificial intelligence has been established. People are often left into the unknown when it comes to the specific definition of these fields. In this study, the first step towards defining these fields is made. Three text analyses models were used to extract the general topics from various data science or artificial intelligence related program or course descriptions. These topics were used to be able to get a grasp on what skill sets are taught to data science and artificial intelligence students. Afterwards, an analysis of posterior classification of the topics per university was performed to explore the differences and similarities between the universities on their orientation of data science and artificial intelligence programs. General and specific skill sets are uncovered and differences between the universities are described in this paper. The results of this paper might be insightful for institutes that have no clear view whether their vacancies might be fit for data science or artificial intelligence graduates.Show less
Longitudinal data are often collected in different research areas such as medicine, biology, education, and psychology. We can build a transitional model using longitudinal binary data, which aims...Show moreLongitudinal data are often collected in different research areas such as medicine, biology, education, and psychology. We can build a transitional model using longitudinal binary data, which aims to model the probability of transition between the response categories. In this type of data is common to find missing values due to dropouts, costs, and organizational problems. The missing-indicator model is often used as a method to handle missing values in this type of data. This method consists in creating a new category for the missing values. Therefore, the binary logistic model changes to a baseline-category logit model. This study aims to evaluate the bias of the estimated coefficients when the missing-indicator method is used in the response of a binary transitional model. Based on an empirical example, a Monte Carlo simulation with three factors is carried out: (1) type of missingness, (2) sample size, and (3) proportion of missing data. The coefficients bias from the baseline-category logit model is evaluated using boxplots and a three-way MANOVA analysis. The results suggest that sample size, the proportion of missing data, type of missingness and the interaction between sample size and proportion affect the bias of the estimated coefficients; nonetheless, the effect size is small. When each dependent variable is analysed separately using ANOVA, the effects of the proportion of missing and the interaction between sample size and proportion were statistically significant for only one coefficient. However, the effect size is still small. Therefore, the conclusion is that the estimated coefficients' bias for all the missingness types is low.Show less
Multivariate binary data with multiple binary response variables arise in many areas of research, including biology, psychology, medicine, dentistry, and other empirical sciences. In such data, the...Show moreMultivariate binary data with multiple binary response variables arise in many areas of research, including biology, psychology, medicine, dentistry, and other empirical sciences. In such data, the effect of a predictor on the response variable and the effect of a predictor on the association structure between the response variables is of interest. Multinomial Restricted Unfolding (MRU) is a probabilistic multidimensional unfolding model that can be used to analyse multicategory response variables in the presence of predictors. In this thesis, we investigated an extension of the MRU model to analyse multivariate binary data focusing on how diagnoses of depressive and anxiety disorders are influenced by personality traits and how the association between two disorders is affected by these personality traits. We compared the results using usual and squared Euclidean distances for the main effects and associations MRU models. We have demonstrated that MRU models using squared and usual Euclidean distances can be used to analyse multivariate binary data, representing well the changes in log odds and the changes in log odds ratio. Our results indicated that the MRU models using squared Euclidean distances are more straightforward and easier to be interpreted than those using usual Euclidean distance. However, despite the more complicated interpretation, the model using the usual Euclidean distance is more flexible, which might lead to a better model fit. Regarding the change in log odds of having GAD, the main effects model results indicated a constant change in slope between different pairs of categories that represent GAD. For the associations model, the parallel lines indicate that the change in the slope was constant within a pair of categories that represents GAD but not the same for different pairs of categories. When usual Euclidean distances were used, the change in slope was not constant for both main effects and associations models. Regarding the interpretation rules to express the change in the log odds ratio, the main effects model showed that the association structure does not dependent on the value of the predictor variable. However, for the associations model, the log odds ratio is dependent on the value of the predictor variable, in which a constant change in slope is shown. When using usual Euclidean distances to explore the association structure between two response variables, the change in slope was not constant for both models.Show less
As the availability of data becomes more widespread and computational technology develops, the need to model several outcome variables at once increases. To do this for multiple dichotomous outcome...Show moreAs the availability of data becomes more widespread and computational technology develops, the need to model several outcome variables at once increases. To do this for multiple dichotomous outcome variables, De Rooij and Groenen (2021) proposed the MELODIC family for simultaneous binary logistic regression in a reduces space. As an added feature, 4 different forms of regularization on the singular values were proposed to let the algorithm itself select the true dimensionality of the data set. In this paper a simulation study was performed to provide empirical evidence for the functionality of the regularization features on data sets with differing numbers of subjects, predictor variables, and outcome variables. The results show that the hard-thresholding regularization consistently estimates the correct dimensionality with regularization on the logarithm of the singular values slightly outperforming the regularization on the unaltered singular values.Show less
Cut-off scores can be useful guides for researchers, clinicians or respondents in interpretating responses on psychometric instruments. Statistical methods involved in the development of cut-off...Show moreCut-off scores can be useful guides for researchers, clinicians or respondents in interpretating responses on psychometric instruments. Statistical methods involved in the development of cut-off scores are predominantly based on univariate analysis and in the presence of an observable criterion variable or normative sample data. To date, little is known about statistical procedures in defining multiple cutting scores for several latent variables within a multidimensional construct (i.e., subscales of a questionnaire). This study aims to evaluate the discriminatory power of various univariate and multivariate methods in cutting score development, and to explore a model-based clustering approach in setting cut-off scores for (sub)scales of multidimensional questionnaires. The investigated approach aims to cluster the responses on multidimensional constructs by using model-based clustering to classify respondents into ordered groups. Subsequently, cut-off points were identified as the intersection points of kernel density curves between the clusters or as (sub)scale split-rules in predicting cluster membership by the use of classification trees. A simulation study was conducted to compare the methods under medium and large sample sizes, various psychometric circumstances (the magnitude of factor correlations and item loadings), and across simulated homogenous and heterogeneous populations. Results of a simulation study show this model-based cluster approach is often preferred over percentile norms and linear standard scores in terms of discriminatory power, especially in defining two cut-off points. Recommendations for model selection and strategies to obtain cut-off points are given. Future research should focus to integrate the presented methodology into a case study using existing questionnaires to examine substantive implications of the calculated cutting scores. This study gives an overview the commonly known statistical techniques involved in cut-off score attainment, provides a multivariate framework to evaluate the discriminatory power in setting cutting scores, and lays out a promising novel application of model-based clustering in developing cut-off points for multidimensional questionnaires.Show less
The identification of the dependence between variables is a common task in Psychology. The most common approaches to this task are Pearson’s, Spearman’s, and Kendall correlation coefficients. A...Show moreThe identification of the dependence between variables is a common task in Psychology. The most common approaches to this task are Pearson’s, Spearman’s, and Kendall correlation coefficients. A clear limitation of those coefficients is that they can only identify linear and monotonic relationships. In recent years, several methods to identify nonmonotonic associations have been developed. Nevertheless, there is not a clear answer to which method should be used when facing unknown conditions in research as is often the case in Psychology. In this study, we aimed to identify which dependence test performs the best under conditions that can be found in psychological research. Method: A simulation was performed to compare nine dependence tests through hypothesis testing. The conditions assessed were sample size, type of relationship, and noise. Three approaches were employed to summarize the statistical power and analyse the results: complete class, average power, and Maximin. Results: There was not a uniformly most powerful test across all conditions. However, several nonmonotonic tests presented a good performance in terms of power for most conditions. Moreover, Mutual Information (MI) estimated through Kernel Density Estimation with the Epanechnikov kernel and the Sheater-Jones plugin bandwidth outperformed all other methods in terms of the analysis approaches of this study. Conclusion: For the evaluated conditions we recommend the use of MI estimated through the defined settings. Nevertheless, other modern tests should not be immediately discarded as their difference in performance with MI is small and could be due to the design of this specific simulation.Show less
Cut-off scores can be useful guides for researchers, clinicians or respondents in interpretating responses on psychometric instruments. Statistical methods involved in the development of cut-off...Show moreCut-off scores can be useful guides for researchers, clinicians or respondents in interpretating responses on psychometric instruments. Statistical methods involved in the development of cut-off scores are predominantly based on univariate analysis and in the presence of an observable criterion variable or normative sample data. To date, little is known about statistical procedures in defining multiple cutting scores for several latent variables within a multidimensional construct (i.e., subscales of a questionnaire). This study aims to evaluate the discriminatory power of various univariate and multivariate methods in cutting score development, and to explore a model-based clustering approach in setting cut-off scores for (sub)scales of multidimensional questionnaires. The investigated approach aims to cluster the responses on multidimensional constructs by using model-based clustering to classify respondents into ordered groups. Subsequently, cut-off points were identified as the intersection points of kernel density curves between the clusters or as (sub)scale split-rules in predicting cluster membership by the use of classification trees. A simulation study was conducted to compare the methods under medium and large sample sizes, various psychometric circumstances (the magnitude of factor correlations and item loadings), and across simulated homogenous and heterogeneous populations. Results of a simulation study show this model-based cluster approach is often preferred over percentile norms and linear standard scores in terms of discriminatory power, especially in defining two cut-off points. Recommendations for model selection and strategies to obtain cut-off points are given. Future research should focus to integrate the presented methodology into a case study using existing questionnaires to examine substantive implications of the calculated cutting scores. This study gives an overview the commonly known statistical techniques involved in cut-off score attainment, provides a multivariate framework to evaluate the discriminatory power in setting cutting scores, and lays out a promising novel application of model-based clustering in developing cut-off points for multidimensional questionnaires.Show less
The ability to correctly classify an individual’s clinical status could help pave the way for early treatment programs for disorders and illnesses of mental- and physical nature alike. Neuroimaging...Show moreThe ability to correctly classify an individual’s clinical status could help pave the way for early treatment programs for disorders and illnesses of mental- and physical nature alike. Neuroimaging data could serve as a basis in reaching this goal of accurate classification. The use of said data, however, does come with challenges. A prominent one of which is the fact that neuroimaging data is highly dimensional, meaning that the amount of features largely exceeds the number of subjects within the data set. Furthermore, research has indicated that the use of heterogeneous, but complimentary, data derived from multiple modalities can be an asset to a model used in a classification setting (Zhang et al., 2010; Schouten et al., 2016). The challenge arises in how to combine the different modalities within a single model. A solution could be the use of machine learning algorithms which search for patterns in the data to draw conclusions. Current literature is, however, lacking meaningful comparisons between different machine learning techniques. Within this project, three different algorithms (support vector machines, Gaussian process classification and multiple kernel learning) have been selected to get insight into (1) whether or not machine learning is able to cope with challenges in the use of neuroimaging data, (2) the difference in performance between these methods and (3) whether or not the use of multiple modalities leads to better results in classification. To this end, 16 alcohol-dependent respondents have been selected along with 32 age-matched healthy controls and have been subjected to both MRI and PET. Models have been trained on data from both separate modalities and on data combining the two modalities. The performances of the models have been assessed by leave-one-subject-out cross-validation and expressed in balanced accuracy and area under the curve. Results indicate that the chosen methods are effective in overcoming challenges arising in the use of neuroimaging data as a means of classification. High balanced accuracies have been found ranging from 76.56% (GPC using PET data) to 100% (GPC using MRI data). Different situations are cause for different solutions and the right choice of algorithm seems to be dependent on, for instance, the fact if either unimodal- or multimodal data is used. Also, settings/optimization of parameters within the model can make a large impact on accuracy. It is therefore advised that researchers try different algorithms and settings before selecting a technique. Different options need to be weighed in order to receive the best possible outcome.Show less
Objective: The Generalized linear mixed-model (GLMM) tree is a decision-tree method which allows for subgroup detection in a wide range of multilevel datasets. This thesis provides a first...Show moreObjective: The Generalized linear mixed-model (GLMM) tree is a decision-tree method which allows for subgroup detection in a wide range of multilevel datasets. This thesis provides a first evaluation of how missing data can be handled in GLMM trees, by assessing the performance of listwise deletion (LD), mean or mode single imputation (SI), multiple imputation (MI) and missingness incorporated in attributes (MIA), in terms of predictive accuracy and tree size accuracy. Method: Different missingness mechanisms, proportions of missing cases and missing data were artificially introduced into data retrieved from the Early Childhood Longitudinal Study Kindergarten class of 1998-1999. Results: As expected, MI yielded the highest performance overall, closely followed by MIA, which exhibited an approximately similar performance. SI performed somewhat worse than MIA and MI, whereas LD showed a substantially inferior performance. Individually, MI and MIA performed very similar for lower amounts of missing data, MI slightly outperformed MIA for higher amounts of missing data, missing completely at random (MCAR) and missing at random (MAR) data and MIA slightly outperformed MI for MNAR (missing not at random) data. When comparing the size of fitted GLMM trees with those fitted on the complete data, MI tended to overfit and yield ensembles of more complex trees, whereas LD, SI and MIA tended to underfit and yield simpler decision trees. Furthermore, the performance of LD was lowest across all conditions and deteriorated even further as the number of cases with missing increased. Conclusion: For handling missing data in GLMM trees, MI is recommended predominantly for prediction purposes, but lacks interpretability. Alternatively, MIA is recommended for interpretability and when a smaller tree size is preferred. Conversely, using either LD or SI is discouraged, even though SI is preferred over LD.Show less
The article screening process for meta-analyses is time-intensive and laborious. Statistical learning and natural language processing techniques can be used to partially automate this process. In...Show moreThe article screening process for meta-analyses is time-intensive and laborious. Statistical learning and natural language processing techniques can be used to partially automate this process. In this study, the performance of four models were compared using a range of evaluation metrics. The first model was built using Latent Dirichlet Allocation (LDA) to extract the topics from the articles to be used as input for a random forest. The second model was built using LDA topics as input for an Extreme Gradient Boosted (XGBoost) tree. The third and fourth models added to the first two by also incorporating a bibliometric feature as input to the respective classifiers. To compare these models, the article catalogues from two meta-analyses pertaining to the field of psychology were gathered and processed. Thus, two real life data-sets were used for the analysis. All four models were built using the full body of text from the article as input for the LDA. These four models were pitted against a benchmark model which represented the more conventional approach to automated article screening. In both datasets, all four proposed models outperformed the benchmark model across all the performance metrics. In the first dataset, the model using LDA topics and bibliometric features as input to the XGBoost was the highest performing model. In the second dataset, the model using LDA topics and bibliometric features as input to the random forest was the highest performing model. The results of this study support the growing evidence that the partial automation of article screening for meta-analyses is indeed possible with a high level of efficiency.Show less
