Data blinding in psychology: Introducing a tool and practical guidelines for common analyses

Making unplanned decisions during data analysis can increase the likelihood of finding evidence that unjustly supports a hypothesis. To reduce this bias, Data blinding hides certain aspects of the data in order to allow researchers to make data-dependent decisions (e.g., checking assumptions to ensure the right analysis is chosen) without knowing the influence of these decisions on results of a hypothesis test. In this study, we examined the suitability of different blinding techniques for three common analyses in psychology: independent samples Student’s t-test, analysis of variance (Type III sum of squares and F-test), and linear least squares regression. For each analysis type, we evaluate whether data blinding techniques interfere with assumption checks and whether they constrain p-hacking. Based on these criteria, we recommend data blinders to center group means for the independent samples Student’s t-test and ANOVA, and to scramble predictor names for regression. We provide...Show moreMaking unplanned decisions during data analysis can increase the likelihood of finding evidence that unjustly supports a hypothesis. To reduce this bias, Data blinding hides certain aspects of the data in order to allow researchers to make data-dependent decisions (e.g., checking assumptions to ensure the right analysis is chosen) without knowing the influence of these decisions on results of a hypothesis test. In this study, we examined the suitability of different blinding techniques for three common analyses in psychology: independent samples Student’s t-test, analysis of variance (Type III sum of squares and F-test), and linear least squares regression. For each analysis type, we evaluate whether data blinding techniques interfere with assumption checks and whether they constrain p-hacking. Based on these criteria, we recommend data blinders to center group means for the independent samples Student’s t-test and ANOVA, and to scramble predictor names for regression. We provide guidelines for researchers and data blinders for each step in the data blinding process and introduce a tool that allows data blinders to easily apply the data blinding techniques to a dataset (i.e., the “blindData” R-package). It is discussed how data blinding, combined with preregistration, benefits the integrity of research output.Show less

Leiden University Student Repository

Documents

In Collections

Data blinding in psychology: Introducing a tool and practical guidelines for common analyses