How Strong is the Evidence for Psychotropic Drugs?

What you need to know

• The strength of evidence supporting different psychiatric medications varies widely, even among drugs approved by the FDA
• Medications for ADHD had the strongest evidence, while antidepressants often had weaker evidence
• Some approved drugs had only moderate or ambiguous evidence of efficacy in clinical trials
• Considering both effect size and strength of evidence could help guide treatment choices

Understanding the approval process for psychiatric medications

When a new medication is developed to treat a mental health condition, it must go through a rigorous approval process before doctors can prescribe it. In the United States, this process is overseen by the Food and Drug Administration (FDA). The FDA requires pharmaceutical companies to conduct clinical trials to demonstrate that a new drug is safe and effective before it can be approved.

However, the standards for what constitutes “substantial evidence” of a drug’s efficacy have been questioned by some researchers. Traditionally, the FDA has required at least two clinical trials showing a statistically significant positive result. But this approach has some limitations - it doesn’t consider how many failed trials occurred before getting two positive ones, for example.

A team of researchers decided to take a closer look at the evidence supporting various psychiatric medications at the time they were initially approved by the FDA. They wanted to see how the strength of evidence compared across different types of drugs and mental health conditions.

A new way to measure evidence strength

The researchers used a statistical approach called Bayesian analysis to evaluate the clinical trial data for 60 psychiatric medications approved to treat four conditions:

• Schizophrenia (15 antipsychotic medications)
• Depression (16 antidepressants)
• Anxiety disorders (9 antidepressants)
• Attention deficit hyperactivity disorder (ADHD) (20 medications)

Rather than just looking at whether trials were statistically significant or not, the Bayesian approach calculates something called a Bayes Factor. This provides a measure of how strongly the data support the conclusion that the drug is effective, compared to the possibility that it’s not effective.

The researchers categorized the strength of evidence for each drug based on its Bayes Factor:

• Ambiguous: Little to no evidence either way
• Modest: Weak evidence of efficacy
• Moderate: Some evidence of efficacy
• Strong: Substantial evidence of efficacy
• Very strong: Overwhelming evidence of efficacy

Wide variation in evidence strength

The analysis revealed substantial differences in the typical strength of evidence across the four types of medications:

• ADHD medications had the strongest evidence overall, with a median Bayes Factor of 1820 (considered “extreme” evidence)
• Antipsychotics for schizophrenia had the next strongest evidence, with a median Bayes Factor of 365 (considered “very strong” evidence)
• Antidepressants for depression had weaker evidence, with a median Bayes Factor of 94 (on the lower end of “strong” evidence)
• Antidepressants for anxiety disorders had the weakest evidence, with a median Bayes Factor of 50 (just barely reaching “strong” evidence)

However, there was significant variation even within each category. Some antidepressants had very strong evidence, while a few ADHD medications had only modest evidence.

Factors influencing evidence strength

The researchers identified several factors that likely contributed to these differences in evidence strength:

Effect size: ADHD medications tended to have larger effects in clinical trials compared to antidepressants. Larger effects are easier to detect statistically, leading to stronger evidence.

Sample size: Clinical trials for schizophrenia medications typically enrolled more participants than trials for other conditions. Larger sample sizes generally produce more reliable results.

Number of trials: More clinical trials were usually conducted for antidepressants compared to other drug types. While this increased the total amount of data, it also meant more chances for inconsistent or negative results.

Implications for doctors and patients

This analysis highlights that not all FDA-approved medications have equally strong evidence supporting their efficacy. In some cases, drugs were approved despite having only moderate or even ambiguous evidence from clinical trials.

The researchers argue that considering both the effect size (how large of an impact the drug has) and the strength of evidence (how certain we are that the effect is real) could help guide treatment decisions. For example, if two antidepressants have similar effect sizes, but one has much stronger evidence supporting it, that might be the preferable choice.

They also suggest that regulatory agencies like the FDA should consider using methods like Bayesian analysis to more systematically evaluate the overall strength of evidence when making approval decisions.

Limitations to consider

It’s important to note some limitations of this study:

• It only looked at data from clinical trials submitted for initial FDA approval. Post-approval studies might strengthen or weaken the evidence for some drugs.
• The analysis focused solely on efficacy, not on side effects or other safety concerns that are also crucial in evaluating medications.
• Differences in how mental health conditions are diagnosed and measured could impact the ability to demonstrate drug efficacy in clinical trials.

Conclusions

• The strength of evidence supporting psychiatric medications varies widely, even among FDA-approved drugs
• ADHD medications and antipsychotics tended to have the strongest evidence, while antidepressants often had weaker evidence
• Considering both effect size and strength of evidence could help guide treatment choices and regulatory decisions
• More systematic ways of evaluating overall evidence strength may be beneficial in the drug approval process