Open Science, Psychology, and the Art of Not Quite Claiming Causality with Julia Rohrer

In a rare departure from our usual diet of online weirdos, this episode features an academic who is very much not a guru. We’re joined by Julia Rohrer, a psychologist at Leipzig University whose work straddles the disciplinary boundaries of open science, research transparency, and causal inference. Julia is also an editor at Psychological Science and has spent much of the last decade politely pointing out that psychologists often don’t quite know what they’re estimating, why, or under which assumptions.

We talk about the state of psychology after the replication crisis, whether open science reforms have genuinely improved research practice (or just added new boxes to tick), and why causal thinking is unavoidable even when researchers insist they are “only describing associations.” Julia explains why the standard dance of imply causality → deny causality → add boilerplate disclaimer is unhelpful, and argues instead for being explicit about the causal questions researchers actually care about and the assumptions required to answer them.

Along the way we discuss images of scientists in the public and amongst the gurus, how post-treatment bias sneaks into even well-intentioned experimental designs, why specifying the estimand matters more than running ever-fancier models, and how psychology’s current norms can potentially punish honesty about uncertainty. We also touch on her work on birth-order effects and offer some possible reasons for optimism.

With all the guru talk, people sometimes ask us to recommend things that we like, and Julia's work is one such example!

Links

• Julia Rohrer’s website
• The 100% CI blog
• Rohrer, J. M. (2024). Causal inference for psychologists who think that causal inference is not for them. Social and Personality Psychology Compass, 18(3), e12948.
• Rohrer, J. M., Tierney, W., Uhlmann, E. L., DeBruine, L. M., Heyman, T., Jones, B., ... & Yarkoni, T. (2021). Putting the self in self-correction: Findings from the loss-of-confidence project. Perspectives on Psychological Science, 16(6), 1255-1269.
• Rohrer, J. M., Egloff, B., & Schmukle, S. C. (2015). Examining the effects of birth order on personality. Proceedings of the National Academy of Sciences, 112(46), 14224-14229.
• BEMC MAY 2024 - Julia Rohrer - "Causal confusions correlate with casual conclusions"
• Dr. Tobias Dienlin - Less casual causal inference for experiments and longitudinal data: Research talk by Julia Rohrer