Episode 361: 260. METHODS MONDAY-- EVENT RATE

Methods Monday --- Accuracy of Event Rate and Effect Size Estimation in Major Cardiovascular Trials: A Systematic Review | Cardiology | JAMA Network Open | JAMA Network

During the design of a randomized clinical trial (RCT), estimation of the expected event rate and effect size is a key component to calculating the sample size. Overly optimistic estimation of event rates and effect sizes may lead to underpowered trials.

If you expect 1 event per 100 people and you are looking for 5 events then you only need to enroll….. 500 people but if the actual event rate is 1 per 200 people then in order to get 5 events you need to enroll 1000 people!! You can see enrolling 500 people instead of 1000 would underpower your trial

This article, published in JAMA Network Open in April 2024, presents a systematic review of 344 contemporary cardiovascular randomized clinical trials (RCTs) to evaluate the accuracy of estimated event rates and effect sizes

1

. The key findings are:

1. Event rates were frequently overestimated:

• Median observed event rate: 9.0% (IQR, 4.3%-21.4%)
• Median estimated event rate: 11.0% (IQR, 6.0%-25.0%)
• 61.1% of trials overestimated the event rate

1

1. Effect sizes were often overestimated:

• Median observed effect size: 0.91 (IQR, 0.74-0.99)
• Median estimated effect size: 0.72 (IQR, 0.60-0.80)
• 82.1% of trials overestimated the effect size

The drug companies think their drug is way better than it is or observed to be in trials

1

1. Device trials were independently associated with decreased accuracy of event rate estimation compared to drug trials

1

.

The study concludes that the frequent overestimation of event rates and effect sizes in cardiovascular RCTs may contribute to underpowered trials and the inability to adequately test trial hypotheses

1

. This finding has implications for trial design—if we are not accurate or realistic about the interventions we are likely to underpower the study which means you have to do the whole thing all over again or likely all over again and risk FDA rejecting you drug.