This thesis presents a small-scale multiverse analysis approach to explore the influence of stress on (dis)honest decision-making and aims to reproduce the findings and expand on the study by Speer...Show moreThis thesis presents a small-scale multiverse analysis approach to explore the influence of stress on (dis)honest decision-making and aims to reproduce the findings and expand on the study by Speer et al. (2023). The hypothesis implies that the effect of stress on decision-making is moderated by an individual’s moral default. The analyses used logistic regression and logistic mixed effects models, focusing on the baseline tendency to cheat (“moral default”) and how this tendency possibly changes in response to stress. The findings suggest that outcomes are sensitive to the choice of statistical model. Logistic regression models indicated significant interaction effects in six pathways, while logistic mixed effect models showed significance in only two out of 20. The analyses also examined the influence of dummy and effect coding, finding that while effect coding resulted in smaller standard errors, the different coding of the stress conditions did not significantly alter the overall conclusions and p-values. Different outlier exclusion methods emphasise the role of the researcher’s degrees of freedom, revealing how different decisions on the outlier rule can significantly influence the results. This study contributes to the understanding of the complex relationship between stress and moral decision-making.Show less
In the paper of Wessel et al. (2020), a multiverse analysis is performed, in four samples. Wessel et al. (2020), had a main hypothesis regarding the suppression effect. This suppression effect was...Show moreIn the paper of Wessel et al. (2020), a multiverse analysis is performed, in four samples. Wessel et al. (2020), had a main hypothesis regarding the suppression effect. This suppression effect was tested by a think/no-think task. The researchers of Wessel et al. (2020), expected that compared to never-suppressed targets (baseline), participants would recall a lower percentage of targets that were to be suppressed. This suppression effect was expected to occur for cues that were part of the original study context (SP test), and for cues that were semantically related to the words in the original study context (IP test). The researchers of Wessel et al. (2020), tested this suppression effect in a multiverse analysis. They performed a dependent samples t-test on all outlier criteria and a Wilcoxon signed-rank test without the outlier criteria. A reproduction of the results found in the dependent samples t-test and Wilcoxon signed-rank test of Wessel et al. (2020), had been done. Then another multiverse analysis was performed, with different analytic choices than in the paper of Wessel et al. (2020), to see if by choosing other analytical pathways findings would become statistically more significant. As alternative analytical pathways, logarithmic transformation was added on the dependent variable of the dependent samples t-test and the Wilcoxon signed rank test on all outlier criteria was performed. The results showed no real evidence for the suppression effect. All of the p-values found in the multiverse analysis were not statistically significant.Show less
Op teveel punten binnen de wetenschappelijke methodologie valt verbetering te behalen. Eén van deze punten is hoe men de beschikbare data gebruikt in een onderzoek. In empirisch onderzoek wordt bij...Show moreOp teveel punten binnen de wetenschappelijke methodologie valt verbetering te behalen. Eén van deze punten is hoe men de beschikbare data gebruikt in een onderzoek. In empirisch onderzoek wordt bij het samenstellen van een dataset een aantal arbitraire keuzes gemaakt, wat ertoe leidt dat een dataset in zekere zin actief wordt geconstrueerd. In dit onderzoek wordt toegelicht hoe deze arbitraire keuzes in de praktijk tot verschillende resultaten kunnen leiden. Dit wordt gedaan door een multiversele analyse uit te voeren op ruwe data verzameld door Vogel, Rose en Crane (2018). Bij een multiversele analyse worden meerdere analyses uitgevoerd over meerdere verwerkte datasets. Dit in tegenstelling tot klassiek onderzoek, waarbij er slechts één analyse over een dataset wordt uitgevoerd. In de betreffende multiversele analyse zijn er 124 analyses uitgevoerd. Van de hieruit voortvloeiende resulaten kwam 74,2%, 83,9% respectievelijk 93,5% overeen met het oorspronkelijke resultaat. Gevonden afwijkingen van het oorspronkelijke resultaat waren dusdanig insignificant dat de gevonden conclusies van het oorspronkelijke onderzoek niet zonder meer kunnen worden aangenomen.Show less
This paper describes a multiverse analysis and examines its use as a possible solution for the problem of researcher degrees of freedom in psychological research. Analyzing the role of arbitrary...Show moreThis paper describes a multiverse analysis and examines its use as a possible solution for the problem of researcher degrees of freedom in psychological research. Analyzing the role of arbitrary choices researchers make in their data-analysis and how it relates to the replication crisis is a central theme of this paper. The target of the current multiverse is a replication study on the sound-symbolism effect. The soundsymbolism effect refers to people associating the sound of certain (non)- words with certain shapes. This multiverse study aimed to examine what effect alternative paths of data-processing and modelling had on the findings of the replication study. The results suggest the outcome of the replication study was mostly robust to the alternative data-processing and modelling options that were proposed in this multiverse. The paper also includes a discussion on the methodology and limitations of the multiverse analysis as well as a discussion on the limitations of the target study. In addition, suggestions on future multiverse analyses, for example, combining it with a sequential testing procedure, is included.Show less
This paper’s aim was to apply the multiverse analysis method to the research paper “Children Prioritize Humans Over Animals Less Than Adults Do” by Wilks et al., (2021). The research hypothesis of...Show moreThis paper’s aim was to apply the multiverse analysis method to the research paper “Children Prioritize Humans Over Animals Less Than Adults Do” by Wilks et al., (2021). The research hypothesis of the latter paper was that children exhibit the tendency of preferring humans over animals less than adults. This preference refers to the bias that adults usually show when they are faced with an ethical dilemma which requires them to save either a person or an animal. After administering a series of ethical dilemmas to children and adults, a regression analysis indicated that children exhibited less speciesist behavior than adults. The multiverse analysis is a method that allows to check for the robustness of a found effect by comparing it with other results, which are products of the same dataset. As part of the multiverse analysis in question, different alternatives concerning data-processing and -analysis were explored and a matrix of different paths was created. This procedure resulted in 18 new regression analyses, which also indicated that children show less humans-over-animals bias. In addition to the main hypothesis, some exploratory analyses recorded in the original paper and including other variables aside from age were further explored. Finally, the topic of multiverse analysis was discussed extensively and further extensions of the current multiverse were proposed.Show less
In recent times, many researchers have introduced research methods to help increase transparency in quantitative social science. One of the introduced research methods is performing a multiverse...Show moreIn recent times, many researchers have introduced research methods to help increase transparency in quantitative social science. One of the introduced research methods is performing a multiverse analysis, which consists of exploring different pathways in data processing with the same statistical method. By doing this, the robustness of statistical results against data processing choices can be checked. In this essay, a multiverse analysis will be performed to test the robustness of statistical results for two hypotheses from a study measuring feelings of disgust towards male homosexuality. The goal of the performed multiverse analysis is to find out how it can provide more insight into an existing dataset. In the original study, the statistical results for both hypotheses were found to be significant, performing a reproduction of statistical results only shows one out of two to be significant. Furthermore, the majority of statistical results found while following alternative pathways turn out to be non-significant. Conclusively, the presented statistical results could be considered not robust against alternative data processing choices. The new insights provided by the performed multiverse analysis could help make a case for multiverse analyses becoming the norm for researchers.Show less
The science of psychology is facing a so-called replication and reproduction crisis. This crisis calls for increased transparency in the way that statistics are reported, in order to make findings...Show moreThe science of psychology is facing a so-called replication and reproduction crisis. This crisis calls for increased transparency in the way that statistics are reported, in order to make findings in psychology more reliable, more likely to be interpreted correctly, and easier to verify and replicate. Undisclosed flexibility in data collection and analysis can severely inflate the false positive rate in psychology (and other disciplines). This issue can be tackled by performing a multiverse analysis. Recent literature proposes that this method guarantees an unprecedent level of transparency for research papers. Building on this stream of research, the present thesis examines the value of using a multiverse analysis to examine the robustness of a non-significant effect. Multiverse analysis refers to a method which involves performing the analysis of interest across the complete set of data sets that arise from several defensible data processing and analysis choices. The demonstration of the multiverse focuses on data collected by Dieleman et al. (2020). The reported results were first reproduced using the R software environment, and then, alternative analysis pathways were examined. The results proved to be reproducible, yet alternative pathways revealed some fluctuation. This thesis contributes to a better understanding of how a multiverse analysis can improve the transparency of psychological science.Show less
With this study we hope to shed light onto the fragility or robustness of significant psychological findings. When researchers have so many degrees of freedom, ways to transform and manipulate...Show moreWith this study we hope to shed light onto the fragility or robustness of significant psychological findings. When researchers have so many degrees of freedom, ways to transform and manipulate their raw data, this not always leads to the sincerest results, even when done unintentionally. With a growing effort to deal with the flaws of psychological research methods, performing multiverse analyses is a way to deal with the numerous analytical and sometimes arbitrary decisions researchers must take when processing their raw data. A multiverse data analysis entails listing all the possible data processing pathways a researcher can take, and running the same analysis for each of these pathways. This results in a wide range of p-values that can be compared to determine the robustness of the claimed results. In this study, we use the data collected by Davoli, O’Rear, McAulay, McNeil, and Brockmole (2020), for their study on how hand position affects performance on multiplication tasks. In their article, Davoli and his colleagues (2020) claim that placing one’s hands close to the study and test materials hinders performance on multiplication tasks (in terms of reaction times), but only when using the same format as in the study phase. When the same format is used, however, there is no effect. The data processing alternatives include using different outlier criteria for reaction times, logarithmically transforming the data and using accuracy of answers as a covariate, among others. Using a multiverse analysis, we were able to uncover the most influential pathways, determine which combination of choices yielded the best and worse results, and question the authenticity of the data processing pathway implemented by the researchers.Show less
Fraude kan op allerlei manieren voorkomen en het is een breed begrip. Het fenomeen dat erachter schuilt is oneerlijkheid. Een populaire manier om oneerlijkheid te meten is de matrix taak. De...Show moreFraude kan op allerlei manieren voorkomen en het is een breed begrip. Het fenomeen dat erachter schuilt is oneerlijkheid. Een populaire manier om oneerlijkheid te meten is de matrix taak. De opdracht van de matrix taak bestaat uit het omcirkelen van twee getallen in een matrix die samen tien maken. Met het aantal goede antwoorden kunnen ze geld verdienen. De participanten krijgen de kans om te liegen over hoeveel matrices ze correct hebben opgelost. Door te liegen en een hoger aantal goede antwoorden door te geven, kan er meer geld worden verkregen. Het aantal correct opgeloste matrices kan worden vergeleken met het aantal matrices dat door de deelnemers wordt opgegeven om de mate van valsspelen te schatten. De matrix taak houdt alleen geen rekening met eerlijke fouten en gaat ervan uit dat mensen goed kunnen optellen. Hierdoor is de matrix taak onbetrouwbaar. In het onderzoek van Heyman et al. (2020) werd deze conclusie getrokken. In deze studie werden echter verschillende participanten uit de analyse gehaald omdat ze hadden geantwoord dat ze al eerder van het onderzoek hadden gehoord of gelezen, evenals participanten die deze vraag hadden opengelaten. Daarnaast werd er in het onderzoek geen rekening gehouden met sekse. Deze variabelen kunnen ervoor zorgen dat het onderzoek niet robuust is. Om dit te onderzoeken werd in deze scriptie een multiverse analyse uitgevoerd. Hierin werd er gekeken naar 16 verschillende analyses. De resultaten daarvan tonen aan dat de bevindingen van Heyman et al. (2020) robuust zijn. Desondanks geeft dit geen zekerheid. De beperkingen van het onderzoek worden besproken in de discussie.Show less
Achtergrond. Structurele MRI-scans worden gebruikt om de diagnose ‘ziekte van Alzheimer’ te ondersteunen. Op dit moment zijn de predictiemodellen op basis van de structurele MRI-scan onvoldoende...Show moreAchtergrond. Structurele MRI-scans worden gebruikt om de diagnose ‘ziekte van Alzheimer’ te ondersteunen. Op dit moment zijn de predictiemodellen op basis van de structurele MRI-scan onvoldoende robuust voor de klinische praktijk. Doel. In dit onderzoek wordt er een robuust predictiemodel ontwikkeld. Robuust wil zeggen een model met zo min mogelijk predictoren, met een zo hoog mogelijke accuratesse wat de toepasbaarheid in de klinische praktijk vergroot. Methode. Voor dit onderzoek zijn de MRI-scans van 76 Alzheimerpatiënten en 173 gezonde ouderen controles gebruikt. In deze data zijn de corticale dikten, subcorticale volumes en grijze stofdichtheid meegenomen. Op deze data heeft een logistische regressieanalyse met LASSOpenalty plaatsgevonden. Vervolgens is er met kruisvalidatie een vergelijking gemaakt tussen een hoge en lage penalty. Van beide modellen zijn de accuratesse, specificiteit en sensitiviteit berekend en hoe dit zich verhoudt met behulp van de ROC-analyse. Er zijn twee lambda-waarden gekozen. Er is gekozen voor de optimale lambda-waarde en voor de lambda-waarde die nog binnen de errorbar is van de optimale waarde, welteverstaan +1 standaarderror. Resultaten. Deze modellen zijn met elkaar vergeleken. Uit het robuuste model met het minimale aantal predictoren komen acht predictoren naar voren. Dit model heeft een AUC-waarde van 0,92 en is accurater dan het model met een lagere penalty. Conclusie. Er is een model ontworpen met een minimaal aantal predictoren dat beter toepasbaar is in de klinische praktijk en een hogere accuratesse heeft. Met dit model kan de ziekte van Alzheimer geclassificeerd worden aan de hand van structurele MRI in de klinische praktijk.Show less
