Leiden University Student Repository

Kisoentewari, Anish (2022)

Evaluating the robustness of permutation-based multiple testing methods

Master thesis | Statistics and Data Science (MSc)

Statistical hypothesis testing is central to many scientific fields. Testing many hypotheses simultaneously is called multiple testing. The main concern in multiple testing, is to ensure that most of the rejected null hypotheses are indeed false, i.e., that the number of incorrect rejections remains low. A major challenge in multiple testing is to account for the complex dependencies in the data. A powerful approach in this regard, are permutation-based multiple testing methods. These methods make few distributional assumptions. In fact, they often make only one assumption, called joint exchangeability. In this thesis we investigate the robustness of the methods to violations of this assumption. We do this by means of simulations, where we focus on case-control data. We find that, while the theoretical literature always makes the mentioned assumption, it is often not necessary in practice. Thus, this thesis provides further evidence for the validity of these powerful methods in...Show moreStatistical hypothesis testing is central to many scientific fields. Testing many hypotheses simultaneously is called multiple testing. The main concern in multiple testing, is to ensure that most of the rejected null hypotheses are indeed false, i.e., that the number of incorrect rejections remains low. A major challenge in multiple testing is to account for the complex dependencies in the data. A powerful approach in this regard, are permutation-based multiple testing methods. These methods make few distributional assumptions. In fact, they often make only one assumption, called joint exchangeability. In this thesis we investigate the robustness of the methods to violations of this assumption. We do this by means of simulations, where we focus on case-control data. We find that, while the theoretical literature always makes the mentioned assumption, it is often not necessary in practice. Thus, this thesis provides further evidence for the validity of these powerful methods in practice.Show less

Multiple testing

Permutation

High-dimensional

maxT

SAM

Robustness

Heteroscedasticity

Author

Kisoentewari, Anish

Faculty

Faculty of Science

Specialisation

Data Science

Supervisors

Hemerik, J.

ECTS Credits

24

Language

en