Knock Knock Eye: How Step One Pass/Fail Fueled the Research Arms Race

So last episode I made the case that medical student research should be banned, and I clearly touched a nerve. I got flooded with angry messages arguing against me, and honestly, I expected some backlash since it was a genuinely spicy take. But I want to be clear, I was being facetious, obviously nobody's going to actually ban med student research. What I actually want to unpack this episode is why this topic clearly struck such a nerve, and what's really driving the explosion in med student publications. I walk through some new data, including a study that found roughly a quarter of recent ophthalmology journal articles may contain AI-generated text, and another from Yale showing a real gap between the publications applicants claim on their residency applications and what's actually verifiable in PubMed. I also get into Goodhart's Law, the idea that when a measure becomes a target, it stops being a good measure, and how that applies perfectly to what's happened since Step One went pass-fail. I share some thoughts on what might actually help, including capping how many publications students can list, and I make it clear I'm not trying to lead some crusade here, I just think this conversation needed to happen. Takeaways: Since Step One became pass-fail, medical students have shifted their focus to research output as a differentiator, and reported publication numbers have risen sharply as a result. A recent study found that roughly 26% of ophthalmology journal abstracts and 22% of commentaries screened in 2025 scored as likely containing AI-generated text. A Yale-affiliated study found that applicants to ophthalmology residency claimed an average of 4.9 publications, while only 2.4 were actually verifiable in PubMed. Goodhart's Law suggests that once a measure like publication count becomes a target used for residency selection, it stops accurately reflecting research quality or applicant ability. Starting with the 2027 application cycle, the AAMC is renaming the ERAS publications field to "scholarly works" in an attempt to shift emphasis from quantity to quality. To Get Tickets to Wife & Death: You can visit Glaucomflecken.com/live We want to hear YOUR stories (and medical puns)! Shoot us an email and say hi! knockknockhi@human-content.com Can’t get enough of us? Shucks. You can support the show on Patreon for early episode access, exclusive bonus shows, livestream hangouts, and much more! –⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ http://www.patreon.com/glaucomflecken⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Also, be sure to check out the newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://glaucomflecken.com/glauc-to-me/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ If you are interested in buying a book from one of our guests, check them all out here: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.amazon.com/shop/dr.glaucomflecken⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ If you want more information on models I use: Anatomy Warehouse provides for the best, crafting custom anatomical products, medical simulation kits and presentation models that create a lasting educational impact. For more information go to Anatomy Warehouse DOT com. Link: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://anatomywarehouse.com/?aff=14⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Plus for 15% off use code: Glaucomflecken15 As a proud FIGS Ambassador, visit https://WearFIGS.com and use code DrGFIRSTFIGS to shop, stay updated on new launches, and learn more about FIGS advocacy projects. Check out FIGS on social platforms: @wearfigs -- A friendly reminder from the G’s and Tarsus: If you want to learn more about Demodex Blepharitis, making an appointment with your eye doctor for an eyelid exam can help you know for sure. Visit ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠http://www.EyelidCheck.com⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ for more information. Head to http://www.cozyearth.com and use my code KNOCKKNOCK for an exclusive 20% off. Produced by⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Human Content⁠ Learn more about your ad choices. Visit megaphone.fm/adchoices