Diagnostic Testing: Do the stats tell you what you need to know?

Diagnostic testing: what do those statistics actually tell you? Sensitivity, specificity, positive predictive value . . . you’ve probably seen these terms before. Maybe you memorized them for a test. But do you actually know what they mean? In this episode, we take a closer look at how diagnostic tests are evaluated—and how they’re often misinterpreted. From a genetic test for cellulite to a blood test for autism, we explore how “statistically significant” findings can turn into tests that don’t actually help anyone. Along the way we meet the freckle gene, the wanderlust gene, and infidelity gene.

Statistical topics

• Base Rate
• Bayes Rule
• Case-Control Study
• Matching
• Conditional Probability
• Sensitivity
• Specificity
• Positive Predictive Value
• Prevalence
• Negative Predictive Value
• False Positives and Negatives
• True Positives and Negatives

Methodological morals

• “A biomarker paper is not the same thing as a biomarker test.”
• “If your sample doesn't match the real world, then for all of your positive predictive value needs, call on Bayes' theorem.”

Detailed Show Notes with calculations

References

• Emanuele E, Bertona M, Geroldi D. A multilocus candidate approach identifies ACE and HIF1A as susceptibility genes for cellulite. Journal of the European Academy of Dermatology and Venereology; 2010. 24: 930-35.
• https://genomelink.io/traits/cellulite
• https://www.genexdiagnostics.com/
• Ebstein RP, Novick O, Umansky R, et al. Dopamine D4 receptor (D4DR) exon III polymorphism associated with the human personality trait of Novelty Seeking. Nat Genet. 1996;12:78-80.
• Kluger AN, Siegfried Z, Ebstein RP. A meta-analysis of the association between DRD4 polymorphism and novelty seeking. Mol Psychiatry. 2002;7:712-7.
• He Y, Martin N, Zhu G, Liu Y. Candidate genes for novelty-seeking: a meta-analysis of association studies of DRD4 exon III and COMT Val158Met. Psychiatr Genet. 2018 Dec;28(6):97-109.
• Smith AM, King JJ, West PR, et al. Amino Acid Dysregulation Metabotypes: Potential Biomarkers for Diagnosis and Individualized Treatment for Subtypes of Autism Spectrum Disorder. Biol Psychiatry. 2019;85:345-54.
• Sainani K, Goodman S. Lack of Diagnostic Utility of “Amino Acid Dysregulation Metabotypes.”
  Biol Psychiatry. 2018; 85: e41-e42.

Kristin and Regina’s online courses

• Demystifying Data: A Modern Approach to Statistical Understanding
• Clinical Trials: Design, Strategy, and Analysis
• Medical Statistics Certificate Program
• Writing in the Sciences
• Epidemiology and Clinical Research Graduate Certificate Program
• Programs that we teach in:
• Epidemiology and Clinical Research Graduate Certificate Program

Find us on:

Kristin - LinkedIn & Twitter/X

Regina - LinkedIn & ReginaNuzzo.com

• (00:00) - Introduction
• (02:24) - The Cellulite Test
• (06:41) - Understanding Sensitivity and Specificity
• (12:50) - Enter Positive Predictive Value
• (18:40) - Why Base Rates Matter
• (24:06) - More Ridiculous Tests
• (32:33) - The Wanderlust Gene Deep Dive
• (40:30) - The NeuroPoint Autism Test
• (51:37) - Trying to Set the Record Straight
• (01:00:42) - Personal Stories
• (01:03:57) - Wrap-up