Replicating the Uncertain: Using Degrees of Freedom Space of Original Articles to Choose  Between Studies With a High Replication Value

Flexibility in the decisions researchers make during their research can lead to false positive findings. Due to low transparency in published papers in the field of psychology, the amount of flexibility authors had is often unclear. In the current thesis, in a first step a quantitative measure of Replication Value is applied to a random set of studies (n = 1257) from Social Psychology, using citation count as a proxy for impact and sample size as a proxy for uncertainty. This Replication Value has been suggested as an indicator of how worthwhile it is to replicate a study (see Isager et al., in press), and can be applied to a large number of studies due to its quantitative approach. However, Replication Value is based on solely on quantitative proxies. Therefore, it is necessary to also manually examine papers. In a second step of the current thesis, it is manually explored whether the uncertainty that researchers have when making choices during their research can become clearer by...Show moreFlexibility in the decisions researchers make during their research can lead to false positive findings. Due to low transparency in published papers in the field of psychology, the amount of flexibility authors had is often unclear. In the current thesis, in a first step a quantitative measure of Replication Value is applied to a random set of studies (n = 1257) from Social Psychology, using citation count as a proxy for impact and sample size as a proxy for uncertainty. This Replication Value has been suggested as an indicator of how worthwhile it is to replicate a study (see Isager et al., in press), and can be applied to a large number of studies due to its quantitative approach. However, Replication Value is based on solely on quantitative proxies. Therefore, it is necessary to also manually examine papers. In a second step of the current thesis, it is manually explored whether the uncertainty that researchers have when making choices during their research can become clearer by mapping them. Therefore, the studies with the highest Replication Values (n = 10), with the median Replication Values (n = 10), and with the lowest Replication Values (n = 10) were examined on their reporting transparency and potential Researcher Degrees of Freedom. A detailed analysis of the first results indicated that the qualitative analysis of the Researcher Degrees of Freedom of original researchers is helpful to in selecting which study to replicate after making a larger selection based on RV. The findings from this exploratory research are discussed in the context of the field of Social Psychology, with an emphasis on how researchers looking to select a target for replication can use our DFS to map the uncertainty of original work.Show less

Leiden University Student Repository

Documents

In Collections

Replicating the Uncertain: Using Degrees of Freedom Space of Original Articles to Choose Between Studies With a High Replication Value