Less is more! Sequential procedures for psychological data

Psychological science has been plagued by a history of studies with low statistical power. The common solution for this issue has been to increase the sample sizes of studies but this also increases time and money spent. Another solution that has been mostly ignored by the scientific community is the use of sequential procedures for statistical analysis. Sequential procedures allow the researcher to analyse the data multiple times as new participants are recruited. So, several analyses are done until the null hypothesis is either accepted or rejected. In practice, this might lead to efficiency gains in the form of smaller sample sizes. The statistical properties of sequential procedures have been found to be adequate based on simulation studies. However, there is a lack of information about the efficiency gains and consistency of these procedures with actual psychological data. The current study aims to evaluate the efficiency and consistency of the group sequential (GS) procedure...Show morePsychological science has been plagued by a history of studies with low statistical power. The common solution for this issue has been to increase the sample sizes of studies but this also increases time and money spent. Another solution that has been mostly ignored by the scientific community is the use of sequential procedures for statistical analysis. Sequential procedures allow the researcher to analyse the data multiple times as new participants are recruited. So, several analyses are done until the null hypothesis is either accepted or rejected. In practice, this might lead to efficiency gains in the form of smaller sample sizes. The statistical properties of sequential procedures have been found to be adequate based on simulation studies. However, there is a lack of information about the efficiency gains and consistency of these procedures with actual psychological data. The current study aims to evaluate the efficiency and consistency of the group sequential (GS) procedure and the sequential probability ratio test (SPRT) with real data. Efficiency is the reduction in sample sizes while consistency refers to (in)congruence between the decision based on the sequential procedure and the conventional procedure. We re-analysed an existing multi-lab registered replication report (RRR) namely the study by Verschuere et al. (2018) using the GS procedure and SPRT procedure. Using these procedures, efficiency gains ranging from 27% to 62% were found. The GS procedure outperformed the SPRT in terms of efficiency. In the labs that showed efficiency gains, the decision regarding the null hypothesis was the same compared to the decision based on the conventional procedure. Taken together, the sequential procedures yielded the same overall conclusion as Verschuere et al., but at a considerably smaller sample size. This shows the potential of these procedures for psychological science as a whole.Show less

Leiden University Student Repository

Documents

In Collections

Less is more! Sequential procedures for psychological data