Postapproval studies of drugs initially approved by the FDA on the basis of limited evidence: systematic review.
Review
Free full text in Europe PMC
Abstract
Free full text
Postapproval studies of drugs initially approved by the FDA on the basis of limited evidence: systematic review
Associated Data
Abstract
Objective To characterize the prospective controlled clinical studies for all novel drugs that were initially approved by the Food and Drug Administration on the basis of limited evidence.
Design Systematic review.
Data sources Drugs@FDA database and PubMed.
Study inclusion All prospective controlled clinical studies published after approval for all novel drugs initially approved by the FDA between 2005 and 2012 on the basis of a single pivotal trial, pivotal trials that used surrogate markers of disease as primary endpoints, or both.
Results Between 2005 and 2012 the FDA approved 117 novel drugs for 123 indications on the basis of a single pivotal trial, pivotal trials that used surrogate markers of disease, or both (single surrogate trials). We identified 758 published controlled studies over a median of 5.5 years (interquartile range 3.4-8.2) after approval, most of which (554 of 758; 73.1%) were studies for indications approved on the basis of surrogate markers of disease. Most postapproval studies used active comparators—67 of 77 (87.0%) indications approved on the basis of single pivotal trials, 365 of 554 (65.9%) approvals based on surrogate marker trials, and 100 of 127 (78.7%) approvals based on single surrogate trials—and examined surrogate markers of efficacy as primary endpoints—51 of 77 (66.2%), 512 of 554 (92.4%), and 110 of 127 (86.6%), respectively. Overall, no postapproval studies were identified for 43 of the 123 (35.0%) approved indications. The median total number of postapproval studies identified was 1 (interquartile range 0-2) for indications approved on the basis of a single pivotal trial, 3 (1-8) for indications approved on the basis of pivotal trials that used surrogate markers of disease as primary endpoints, and 1 (0-2) for single surrogate trial approvals, and the median aggregate number of patients enrolled in postapproval studies was 90 (0-509), 533 (122-3633), and 38 (0-666), respectively. The proportion of approved indications with one or more randomized, controlled, double blind study using a clinical outcome for the primary endpoint that was published after approval and showed superior efficacy was 18.2% (6 of 33), 2.0% (1 of 49), and 4.9% (2 of 41), respectively.
Conclusions The quantity and quality of postapproval clinical evidence varied substantially for novel drugs first approved by the FDA on the basis of limited evidence, with few controlled studies published after approval that confirmed efficacy using clinical outcomes for the original FDA approved indication.
Introduction
In the US, the Food and Drug Administration determines whether a new drug is sufficiently safe and effective to be made available to doctors for use by patients.1 To do this, it must find a balance between requiring sufficient high quality clinical evidence from premarket evaluation and allowing promising new drugs to enter the marketplace quickly with continued evaluation after approval. The FDA maintains a “usual requirement” of “more than one” well controlled clinical trial that independently proves a drug’s efficacy.2 However, it also describes several situations in which fewer trials or studies with non-clinical outcomes, such as surrogate markers of disease, might suffice for premarket evaluation.2 Thus, FDA approval is binary, but the clinical trial evidence that forms the basis of the FDA’s decision varies widely.
From 2005 to 2012 the FDA approved 188 new drugs for 206 indications, more than a third of which were approved on the basis of a single pivotal trial and 44% of which were approved on the basis of pivotal trials that used surrogate markers of disease instead of clinical outcomes for primary endpoints (41 approvals met both criteria).3
Regulation has historically occurred primarily in the period before a drug is approved and becomes widely available. Over the past decade, however, the FDA has adopted “lifecycle evaluation” (box 1).4 5 This approach was originally proposed by the Institute of Medicine in its 2007 report, The Future of Drug Safety, which was commissioned by the FDA after a series of well publicized postmarket drug safety concerns raised questions about drug safety in the US.4 6 In addition to monitoring drug safety, the lifecycle evaluation approach enables regulators to approve drugs based on clinical evidence that is less robust, with the understanding that drugs will continue to be evaluated after approval.
As the FDA continues to use lifecycle evaluation,6 7 8 9 we need a better understanding of the accumulation of clinical evidence after approval. Potential sources of postapproval clinical information could include prospective clinical studies, such as randomized controlled trials; observational studies of existing data, such as administrative claims; and adverse event reporting through programs such as the Sentinel System.10 The FDA also has formal postmarketing requirements (statutory requirements used when the FDA determines that an investigation is required to assess a known or potential serious safety risk) and postmarketing commitments (any other research agreed upon by manufacturers and the FDA that is not an investigation of a potential or known serious safety risk—such as any investigation that primarily evaluates efficacy—and mandates an annual report to the FDA), which frequently include postapproval clinical research studies.11
Despite the FDA’s increased emphasis on generating postapproval evidence, whether prospective controlled trials for the same indication for which the drug was first approved are performed after approval, and whether those that are done confirm the benefits expected on the basis of the premarket pivotal trials, is unknown. For example, one study found that 31 of 36 cancer drugs approved on the basis of surrogate markers from 2008 to 2012 had either unknown effects on overall survival or failed to show a positive effect on survival after a median follow-up of over four years.12 Moreover, even the postapproval studies required by the FDA are not always carried out.13 14 15
We sought to better understand how efficacy evidence from prospective controlled clinical trials accumulates in the years after approval for drugs initially approved on the basis of less robust evidence.
Methods
Study design and sample
We used the Drugs@FDA database to identify all novel drugs approved by the FDA between 1 January 2005 and 31 December 2012 on the basis of either a single pivotal trial or pivotal trials focused on surrogate markers of disease, as described in previous work.3 Our sample included all new drug and biologic licensing applications for small molecule and biologic drugs, excluding generic drugs, reformulations, combination treatments of non-novel agents, and non-treatment agents such as diagnostic and contrast agents. Pivotal trials were those labeled as such in the FDA medical review. If the FDA review did not specify, we identified trials that were described as essential to approval, were otherwise prioritized in the review, or were new efficacy based trials provided as part of a resubmitted application for approval.
We also used the Drugs@FDA database to obtain the year of approval for each novel drug and whether it was considered a pharmacologic (small molecule) or biologic product. We used FDA approval letters, which are hyperlinked in the Drugs@FDA database, to identify orphan drugs, drugs approved through the accelerated approval pathway, and the indication for which all novel drugs were initially approved. Orphan drugs were those classified as such by the FDA, meaning drugs for rare diseases that were given extended market exclusivity. The Drugs@FDA database only indicates orphan status for biologics approved after 2010.
Indications were categorized by expected length of treatment using a previously developed framework:3 expected length of use of less than one month was acute, between one month and two years was intermediate, and more than two years was chronic. We used the World Health Organization’s anatomic therapeutic classification system, contextualized for clinical relevance, to categorize each indication into therapeutic areas,16 which we collapsed into four categories: cancer, cardiovascular disease and diabetes mellitus, infectious disease, and other.
Systematic review of postapproval trials
Two investigators (AMP, JAA) systematically searched all fields of Medline for the international non-proprietary name of each drug to identify all postapproval prospective studies in humans that used an active or placebo control and examined efficacy for the same indication for which the drug was first approved by the FDA. We searched for studies published between 1 January of the year before the drug was approved and 31 December 2014. This timeframe was chosen to find studies that started enrolling before FDA approval but did not produce data soon enough to be included in the FDA submission.
Although Medline is not a complete repository of all publications in all biomedical journals, it is the largest database of biomedical journal articles that can be searched freely using the PubMed system. Nearly all doctors and policy makers depend on it to learn about and obtain access to the findings of clinical trials. As such, it is the most important dissemination source of clinical trial publications to inform and influence real world clinical practice.
We included all English language publications and put no exclusions on types of funder or investigator, including developers of the drug or its competitors, government agencies, and independent investigators.
Two investigators reviewed the titles and abstracts (AMP, JAA) and one (AMP) reviewed the full text to exclude the following: conference abstracts; case reports or series; studies performed on healthy people; pharmacokinetic or drug interaction studies; studies evaluating an indication substantially different from that for which the drug was initially approved; studies without an arm for placebo or active control (multi-arm studies of different dosages of the same drug were considered non-controlled); studies without efficacy endpoints described in the abstract (unless novel or unfamiliar surrogate markers in the abstract could be verified as examining efficacy in the full text); studies that were submitted to the FDA as part of the initial new drug application or were completed before FDA submission; and extension studies or new analyses of data from previously published studies.
We defined a substantially different indication narrowly, meaning that to exclude a publication it had to evaluate an entirely different disease. Investigations of merely a different population (children versus adults) or disease subtype (clear cell versus non-clear cell renal cell carcinoma) were included. Publications for which inclusion was questionable for any reason were reviewed by JSR, and conflicts were resolved by consensus.
Trial data abstraction
One investigator (AMP) abstracted the following information from each publication: electronic publication date (if not available, print publication date); national clinical trial number, if provided; whether funding was from industry or non-industry sources; whether authors were employees of the drug manufacturer; use of randomization; whether subject allocation was double blind, single blind, or open label; study design (superiority or non-inferiority); whether the comparator was placebo, active, or both, with results for studies in the “both” category recorded for the comparison against the active agent; number of patients in the overall and drug arm intention to treat populations; study duration; whether the primary endpoint was a clinical outcome, a clinical scale, or a surrogate marker of disease; the primary endpoint result—that is, whether the drug was found to be statistically significantly better than (positive), no different from (neutral), or worse than (negative) the comparator in studies using a superiority design or whether non-inferiority was established (positive) or not established (negative) in studies using a non-inferiority design; whether secondary endpoints described in the abstract were clinical outcomes, clinical scales, surrogate markers of disease, or mixed; and the results of the secondary endpoints (using the same approach as for primary endpoints). One investigator (JAA) subsequently validated the characterization of studies in a 5% random sample, resolving conflicts by consensus. Any uncertainty was reviewed with JSR.
Primary trial endpoints were classified as clinical outcomes, clinical scales, or surrogate markers based on an established framework and a report from the Institute of Medicine (box 2).3 17 18 Clinical outcomes, such as death or admission to hospital, measure patient survival or function. Clinical scales, such as the Crohn’s disease activity index or the visual analog scale for pain, represent rubrics for the quantification of subjective patient reported symptoms. Surrogate markers, such as levels of glycated hemoglobin or hepatitis C ribonucleic acid, represent biomarkers expected to predict clinical benefit.
For studies with multiple primary endpoints, we categorized study results on the basis of clinical endpoints over surrogate markers, if both were reported. For studies with multiple primary endpoints that were all clinical or all surrogate marker and for studies with multiple secondary endpoints of any type, we categorized studies as statistically significantly better or worse based on whether they featured some non-significantly different endpoints and some significantly better or worse endpoints, respectively. The statistical significance of the result for each endpoint was abstracted directly from the publication. If statistical significance or lack thereof was not reported in the text and a significance threshold was not defined in the paper, we considered P values of less than 0.05 or completely non-overlapping confidence intervals to show a statistically significant difference. Two investigators (AMP, JSR) reviewed studies with mixed results—some significantly better and some significantly worse (or non-inferiority established and non-inferiority not established)—and decided the most appropriate categorization based on the relative clinical importance of the endpoints or, if equally important, the result supported by the preponderance of endpoints. We identified only two studies with multiple primary endpoints that were all surrogate or all clinical with mixed results.
Statistical analysis
All analyses were stratified based on whether the novel drug was first approved by the FDA on the basis of a single pivotal trial, multiple pivotal trials that focused on surrogate markers of disease for the primary efficacy endpoint, or both (a single pivotal trial that focused on surrogate markers). We used descriptive statistics to summarize the characteristics, design features, and findings of each study, as well as sample characteristics, aggregate number of trials, number of patients enrolled, and patient years of observation for each indication. We used χ2, Wilcoxon’s rank sum, and Kruskal-Wallis tests to examine differences across the three categories of approval. We used χ2 and Fisher’s test of exact probability to examine differences in study findings within categories, comparing studies using surrogate markers of disease and clinical outcomes as primary endpoints. Analyses were repeated, stratifying by year of indication approval, therapeutic area, and expected length of treatment. All analyses were performed using Microsoft Excel 2013 and JMP 7.0.1 (SAS Institute; Cary, NC). All statistical tests were two tailed and used a type I error rate of 0.005 to account for the 10 comparisons made across categories of indication approval.
Patient involvement
No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for design or implementation of the study. No patients were asked to advise on interpretation or writing up of results. There are no plans to disseminate the results to study participants or the relevant patient community.
Results
Sample characteristics
Of the 188 novel drugs approved by the FDA for 206 indications between 2005 and 2012, 30 novel drugs for 33 indications were approved on the basis of a single pivotal trial, 48 novel drugs for 49 indications on the basis of trials focused on surrogate markers of disease, and 40 novel drugs for 41 indications on the basis of a single pivotal trial that focused on surrogate markers. Our study sample comprised 117 unique novel drugs approved for 123 indications (one drug had two approvals in different categories). Most approvals, across all three categories, were for pharmacologic drugs for the treatment of chronic disease; a few were granted orphan status or approved through the accelerated pathway (table 11).). The most common therapeutic area was cancer, which represented a third (41 of 123) of the overall sample. The proportion of pharmacologic drugs, accelerated approvals, therapeutic areas, and expected lengths of treatment (P values <0.02) showed statistically significant differences between categories, but orphan status and approval year did not. Median follow-up duration from the date of FDA approval to 31 December 2014 was 5.5 years (interquartile range 3.4-8.2).
Table 1
Characteristics of novel drugs approved by the FDA between 2005 and 2012. Values are numbers (percentages) unless stated otherwise
| Characteristics | Single pivotal trial | Surrogate markers | Both | P value |
|---|---|---|---|---|
| Total indications | 33 (100.0) | 49 (100.0) | 41 (100.0) | |
| Agent type: | ||||
 Pharmacologic | 23 (69.7) | 45 (91.8) | 31 (75.6) | 0.02 |
| Biologic | 10 (30.3) | 4 (8.2) | 10 (24.4) | |
| Orphan status: | ||||
| Yes | 6 (18.2) | 10 (20.4) | 10 (24.4) | 0.85 |
| No | 27 (81.8) | 39 (79.6) | 31 (75.6) | |
| Approval pathway: | ||||
| Accelerated | 0 (0.0) | 12 (24.5) | 9 (22.0) | 0.002 |
| Regular | 33 (100.0) | 37 (75.5) | 32 (78.0) | |
| Therapeutic area: | ||||
| Cancer | 9 (27.3) | 8 (16.3) | 24 (58.5) | 0.004 |
| Cardiovascular disease, diabetes mellitus | 5 (15.2) | 13 (26.5) | 2 (4.9) | |
| Infectious disease | 3 (9.1) | 11 (22.4) | 3 (7.3) | |
| Other | 16 (48.5) | 17 (34.7) | 12 (29.3) | |
| Expected length of treatment: | ||||
| Acute | 4 (12.1) | 6 (12.2) | 4 (9.8) | 0.002 |
| Intermediate | 12 (36.4) | 9 (18.4) | 24 (58.5) | |
| Chronic | 17 (51.5) | 34 (69.4) | 13 (31.7) | |
| Year of approval: | ||||
| 2005-07 | 11 (33.3) | 19 (38.8) | 14 (34.1) | 0.93 |
| 2008-10 | 10 (30.3) | 15 (30.6) | 11 (26.8) | |
| 2011-12 | 12 (36.4) | 15 (30.6) | 16 (39.0) |
Indications approved on the basis of a single pivotal trial
We performed systematic reviews of the literature for all 33 indications for the 30 novel drugs that were approved by the FDA on the basis of a single pivotal trial. We identified a total of 77 published postapproval controlled studies using an active or placebo control and examining efficacy for the same indication for which the drug was first approved (table 22).). Of these, 81.8% (n=63) were randomized, 45.5% (n=35) used open label allocation, 66.2% (n=51) used primary efficacy endpoints that were surrogate markers of disease, 87.0% (n=67) used an active drug as control, 94.8% (n=73) used a superiority design, and 33.8% (n=26) were funded by industry. The median overall intention to treat population was 96 (interquartile range 56-260), and the median study duration was four weeks (interquartile range 0.5-26). Among the 60 superiority studies using active comparators, one of 13 (7.7% ) studies examining clinical primary endpoints reported positive results, whereas 30 of 47 (63.8%) studies examining surrogate markers reported positive results (table 33).). Among the 10 superiority studies using placebo comparators, 62.5% (five of eight) of clinical endpoint studies and 50.0% (one of two) of surrogate marker studies reported positive results.
Table 2
Characteristics of prospective controlled postapproval studies of novel drugs of the same indication for which the drug was first approved by the FDA between 2005 and 2012 on the basis of a single pivotal trial, pivotal trials that used surrogate markers of disease as primary endpoints, or both. Values are numbers (percentages) unless stated otherwise
| Characteristics | Single pivotal trial | Surrogate markers | Both | P value |
|---|---|---|---|---|
| Total studies | 77 (100.0) | 554 (100.0) | 127 (100.0) | |
| Randomized | 63 (81.8) | 493 (89.0) | 112 (88.2) | 0.18 |
| Allocation: | ||||
| Double blind | 18 (23.4) | 223 (40.3) | 22 (17.3) | <0.001 |
| Single blind | 24 (31.2) | 44 (7.9) | 13 (10.2) | |
| Open label | 35 (45.5) | 287 (51.8) | 92 (72.4) | |
| Primary endpoint: | ||||
| Clinical outcome or scale | 23 (29.9) | 14 (2.5) | 9 (7.1) | <0.001 |
| Surrogate marker | 51 (66.2) | 512 (92.4) | 110 (86.6) | |
| No primary efficacy endpoint | 3 (3.9) | 28 (5.1) | 8 (6.3) | |
| Comparator: | ||||
| Active | 67 (87.0) | 365 (65.9) | 100 (78.7) | <0.001 |
| Placebo | 10 (13.0) | 148 (26.7) | 24 (18.9) | |
| Active and placebo | 0 (0.0) | 41 (7.4) | 3 (2.4) | |
| Trial design: | ||||
| Superiority | 73 (94.8) | 492 (88.8) | 99 (78.0) | <0.001 |
| Non-inferiority | 4 (5.2) | 62 (11.2) | 28 (22.0) | |
| Intention to treat population*: | ||||
| Entire trial | 96 (56-260) | 127 (49-429) | 207 (63-453) | 0.01 |
| Intervention group | 50 (30-119) | 68 (26-201) | 111 (36-230) | 0.01 |
| Study duration: | ||||
| Duration in weeks* | 4 (0.5-26) | 24 (8-48) | 26 (12-52) | <0.001 |
| Duration ≥24 weeks | 21 (27.3) | 288 (52.0) | 83 (65.4) | <0.001 |
| Funder type: | ||||
| Industry or mixed funding that includes industry | 26 (33.8) | 337 (60.8) | 98 (77.2) | <0.001 |
| All others (non-profit, government, mixed non-industry, none, not specified) | 51 (66.2) | 217 (39.2) | 29 (22.8) |
*Values are medians (interquartile ranges).
Table 3
Findings from prospective controlled postapproval studies with primary efficacy endpoints of novel drugs of the same indication for which the drug was first approved by the FDA between 2005 and 2012 on the basis of a single pivotal trial, pivotal trials that used surrogate markers of disease as primary endpoints, or both. Values are numbers (percentages)
| Study findings | Single pivotal trial | Surrogate markers | Both |
|---|---|---|---|
| Superiority studies | 70 | 464 | 91 |
| Active comparators*: | |||
| Clinical primary endpoint (n=25): | 13 (100) | 8 (100) | 4 (100) |
| Positive | 1 (7.7) | 2 (25.0) | 1 (25.0) |
| Neutral | 9 (69.2) | 6 (75.0) | 2 (50.0) |
| Negative | 3 (23.1) | 0 (0.0) | 1 (25.0) |
| Surrogate marker of disease (n=429): | 47 (100) | 317(100) | 65 (100) |
| Positive | 30 (63.8) | 143 (45.1) | 27 (41.5) |
| Neutral | 8 (17.0) | 129 (40.7) | 29 (44.6) |
| Negative | 9 (19.1) | 45 (14.2) | 9 (13.8) |
| Placebo comparator: | |||
| Clinical primary endpoint: | 8 (100) | 4 (100) | 3 (100) |
| Positive | 5 (62.5) | 2 (50.0) | 2 (66.7) |
| Neutral | 2 (25.0) | 2 (50.0) | 1 (33.3) |
| Negative | 1 (12.5) | 0 (0.0) | 0 (0.0) |
| Surrogate marker of disease: | 2 (100) | 135 (100) | 19 (100) |
| Positive | 1 (50.0) | 108 (80.0) | 13 (68.4) |
| Neutral | 1 (50.0) | 27 (20.0) | 5 (26.3) |
| Negative | 0 (0.0) | 0 (0.0) | 1 (5.3) |
| Non-inferiority studies | 4 | 62 | 28† |
| Clinical primary endpoint: | 2 (100) | 2 (100) | 2 (100) |
| Positive | 2 (100.0) | 2 (100.0) | 1 (50.0) |
| Surrogate marker of disease: | 2 (100) | 60 (100) | 25 (100) |
| Positive | 2 (100.0) | 59 (98.3) | 23 (92.0) |
*Includes studies with both active and placebo comparators, recorded with respect to active comparator.
†One trial was unable to demonstrate non-inferiority or inferiority and is not included.
We aggregated studies by indication and found that the median number of postapproval studies for each indication was 1 (interquartile range 0-2) (see supplementary appendix figure 1a) and the median number of studies that were randomized and double blind was 0 (0-1) (table 44).). The median number of patients enrolled was 90 (0-509), and the median number of patients in the intervention arm was 52 (0-250) (see supplementary appendix figure 2a). The median number of patient years was 30.0 (0-352.4), and the median number of intervention patient years was 15.0 (0-212.9) (see supplementary appendix figure 3a). No postapproval studies were identified for 15 of 33 (45.5%) approved indications. Seven indications (21.2%) had one or more postapproval, randomized controlled, double blind positive study showing superior efficacy, six of which (18.2%) used a clinical outcome for the primary endpoint.
Table 4
Aggregated number and patients enrolled in prospective controlled postapproval studies of novel drugs of the same indication for which the drug was first approved by the FDA between 2005 and 2012 on the basis of a single pivotal trial, pivotal trials that used surrogate markers of disease as primary endpoints, or both. Values are medians (interquartile ranges) unless stated otherwise
| Characteristics | Single pivotal trial (n=33) | Surrogate markers (n=49) | Both (n=41) | P value |
|---|---|---|---|---|
| Studies per indication | 1 (0-2) | 3 (1-8) | 1 (0-2) | <0.001 |
| Randomized and double blind studies | 0 (0-1) | 1 (0-3.5) | 0 (0-0) | <0.001 |
| Total patients enrolled | 90 (0-509) | 533 (122-3633) | 38 (0-666) | <0.001 |
| Intervention patients enrolled | 52 (0-250) | 352 (104-2080) | 19 (0-378) | <0.001 |
| Total patient years of exposure | 30.0 (0-352.4) | 448.5 (23.1-2952.0) | 4.7 (0-998.7) | 0.003 |
| Intervention patient years of exposure | 15.0 (0-212.9) | 350.0 (12.2-1412.8) | 3.8 (0-693.9) | 0.003 |
| Indications with (No (%)): | ||||
| ≥1 study | 18 (54.5) | 41 (83.7) | 21 (51.2) | 0.001 |
| ≥1 randomized and double blind study | 10 (30.3) | 31 (63.3) | 9 (22.0) | <0.001 |
| ≥1 randomized and double blind study with clinical primary endpoint | 10 (30.3) | 3 (6.1) | 3 (7.3) | 0.005 |
| ≥1 study with positive results | 9 (27.3) | 37 (75.5) | 15 (36.6) | <0.001 |
| ≥1 randomized and double blind study with positive results | 7 (21.2) | 28 (57.1) | 6 (14.6) | <0.001 |
| ≥1 randomized and double blind study with clinical primary endpoint and positive results | 6 (18.2) | 1 (2.0) | 2 (4.9) | 0.02 |
Indications approved on the basis of surrogate markers
We performed systematic reviews of the literature for all 49 indications for the 48 drugs that were approved by the FDA on the basis of multiple pivotal trials evaluating surrogate markers of disease. We identified a total of 554 published postapproval controlled studies using an active or placebo control and examining efficacy for the same therapeutic indication for which the drug was first approved (table 22).). Of these, 89.0% (n=493) were randomized, 51.8% (n=287) used open label allocation, 92.4% (n=512) used primary efficacy endpoints that were surrogate markers of disease, 65.9% (n=365) used an active drug as control, 88.8% (n=492) used a superiority design, and 60.8% (n=337) were funded by industry. The median overall intention to treat population was 127 (interquartile range 49-429), and median study duration was 24 weeks (8-48). Among the 325 superiority studies using active comparators, two of eight (25.0%) studies examining clinical efficacy endpoints reported positive results, whereas 143 of 317 (45.1%) of those examining surrogate markers reported positive results (table 33).). Among the 139 superiority studies using placebo comparators, 50.0% (two of four) of clinical endpoint studies and 80.0% (108 of 135) of surrogate marker studies reported positive results.
We aggregated studies by indication and found that the median number of postapproval studies for each indication was 3 (interquartile range 1-8) (see supplementary appendix figure 1b), and the median number of randomized and double blind studies was 1 (0-3.5) (table 44).). The median number of patients enrolled was 533 (122-3633), and the median number of patients in the intervention arm was 352 (104-2080) (see supplementary appendix figure 2b). The median number of patient years was 448.5 (23.1-2952.0), and the median number of intervention patient years was 350.0 (12.2-1412.8) (see supplementary appendix figure 3b). No postapproval studies were identified for eight of 49 (16.3%) approved indications. Twenty eight indications (57.1%) had one or more postapproval, randomized controlled, double blind positive study showing superior efficacy, one of which (2.0%) used a clinical outcome for the primary endpoint.
Indications approved on the basis of a single pivotal trial evaluating surrogate markers
We performed systematic reviews of the literature for all 41 indications for the 40 drugs that were approved by the FDA on the basis of a single pivotal trial evaluating surrogate markers of disease. We identified a total of 127 published postapproval controlled studies using an active or placebo control and examining efficacy for the same therapeutic indication for which the drug was first approved (table 22).). Of these, 88.2% (n=112) were randomized, 72.4% (n=92) used open label allocation, 86.6% (n=110) used primary efficacy endpoints that were surrogate markers of disease, 78.7% (n=100) used an active drug as control, 78.0% (n=99) used a superiority design, and 77.2% (n=98) were funded by industry. The median overall intention to treat population was 207 (interquartile range 63-453), and the median study duration was 26 weeks (12-52). Among the 69 superiority studies using active comparators, one of four (25.0%) studies examining clinical efficacy endpoints reported positive results, and 27 of 65 (41.5%) of those examining surrogate markers reported positive results (table 33).). Among the 22 superiority studies using placebo comparators, 66.7% (two of three) of clinical endpoint studies and 68.4% (13 of 19) of surrogate marker studies reported positive results.
We aggregated studies by indication and found that the median number of postapproval studies for each indication was 1 (interquartile range 0-2) (see supplementary appendix figure 1c), and the median number of randomized and double blind studies was 0 (0-0) (table 44).). The median number of patients enrolled was 38 (0-666), and the median number of patients enrolled in the intervention arm was 19 (0-378) (see supplementary appendix figure 2c). The median number of patient years was 4.7 (0-998.7), and the median number of intervention patient years was 3.8 (0-693.9) (see supplementary appendix figure 3c). No postapproval studies were identified for 20 of 41 (48.8%) approved indications. Six indications (14.6%) had one or more postapproval, randomized controlled, double blind positive study showing superior efficacy, two of which (4.9%) used a clinical outcome for the primary endpoint.
Subgroup analyses
Medians and rates were largely consistent when examined by year of indication approval, by therapeutic area, or by expected length of treatment, with the exception of medians by year of indication of approval among approvals based on a single pivotal trial using surrogate markers (see supplementary appendix table 1) and medians by therapeutic area among approvals based on multiple trials using surrogate markers (see supplementary appendix table 2); no consistent differences were found between expected lengths of treatment (see supplementary appendix table 3).
Discussion
We found substantial variation in the quantity and quality of studies of novel drugs published after they were approved by the FDA on the basis of a single pivotal trial, pivotal trials that used surrogate markers of disease, or both. We found few published randomized controlled, double blind studies showing superior efficacy based on clinical outcomes that examined the same indication for which the drug was first approved by the FDA after a median follow-up of 5.5 years. These findings have important implications for clinical care. Both doctors and patients have high expectations for the safety and efficacy of a drug approved by the FDA.19 20 21 22 23 But less than one third of new drug indications approved by the FDA on the basis of a single pivotal trial had at least one postapproval trial showing superior efficacy; even fewer used clinical outcomes. Similarly, less than one 10th of new drug indications approved by the FDA on the basis of surrogate markers of disease had at least one postapproval trial validating the use of the surrogate marker by showing superior efficacy using clinical outcomes. Our work corresponds with a previous study demonstrating that highly cited studies are infrequently followed by subsequent clinical studies that confirm original effects24 and is consistent with a study showing that cancer drugs approved on the basis of surrogate markers are infrequently followed by clinical studies demonstrating improved overall survival.12
As well as finding few robustly designed, confirmatory studies, we also found noticeable variability in the degree to which novel drugs were studied in the postapproval period, both within and between the categories of approval. Although a small number of approved indications had dozens of qualifying postapproval studies, more than one third had no published postapproval controlled studies investigating efficacy. Furthermore, across approval categories, the studies showed statistically significant differences in all features of trial design, except randomization, as well as in aggregated median numbers of studies and in patient enrollment. For many novel drugs, the problem is not that postapproval studies are poorly designed or have negative efficacy results, but rather that they are not being published or performed at all. Our findings are consistent with previous studies showing that even postapproval studies required by the FDA to be conducted by manufacturers are not always performed in a timely manner13 14 15 and with previous work demonstrating the variability in the number and completion of postmarketing studies of high risk medical devices.25
The majority of the postapproval studies that we identified focused on surrogate markers of disease. Using surrogate markers instead of clinical outcomes for trial endpoints has become increasingly controversial,26 with several approved drugs ultimately failing to confirm any clinical benefit27 28 29 and an analysis of oncology surrogate markers finding variable correlation with overall survival.30 Surrogate markers have been used for pivotal clinical studies to facilitate more rapid regulatory evaluation, with the expectation that subsequent clinical studies will focus on clinical outcomes. We found that this occurs infrequently, as approximately 90% of postapproval studies of drugs for indications approved on the basis of surrogate markers also used surrogate markers of disease for trial endpoints.
Recent proposals for FDA reform include creating a comprehensive benefit and risk assessment and management plan to be updated at regular intervals and with any shift in a drug’s benefit to risk profile7; increasing reliance on surrogate markers, smaller and shorter trials, and evidence derived from registries and observational studies31; and switching to a “consumer reports” approach of grades for efficacy, safety, and degree of evidence.32 Our findings show that caution would be needed for these approaches—high quality postapproval evidence does not necessarily accumulate and may require additional regulatory requirements. To strengthen lifecycle evaluation of recently approved drugs, requirements for postapproval studies might need to be heightened or might need to specify study design characteristics or trial endpoints in detail to ensure that these studies provide high quality evidence to further inform clinical practice. Of course, it must be remembered that publishing one short term, small postapproval study might be inadequate for a hypertension drug but entirely reasonable for one treating a rare disease. A customized, specific plan for postapproval studies will be key to generating an adequate amount of postapproval evidence on comparative efficacy, which should take into consideration disease prevalence and severity, the drug’s anticipated length of use, and the availability of other treatment options.
Limitations of this study
Our study has several limitations. Firstly, we restricted our analysis to studies that had been published in the biomedical literature and indexed in Medline and did not search other databases, such as Embase, Scopus, research conference proceedings (abstracts), or ClinicalTrials.gov. Because our study comprised 123 unique searches to determine whether new drugs approved on the basis of limited evidence continued to be studied in the postmarketing period, we considered this to be a common sense strategy to identify all clinically relevant trial papers. Moreover, our Medline search for publications was extensive, using a systematic query method to capture all clinical trial results that were most likely to inform and influence clinical practice. We might have missed studies that had been conducted but not indexed in Medline, but without widespread dissemination any such studies are unlikely to have informed clinical practice. During project planning, we performed a non-systematic pilot search of ClinicalTrials.gov and found that nearly every entry either did not report study results or was already published in a journal indexed on Medline and had been identified through our systematic search, leading us to conclude that searching this database for additional trials would not be worth the considerable effort. Moreover, we are likely to have identified all controlled studies that confirmed or validated the efficacy of novel drugs given the ample literature suggesting that positive, completed studies are most likely to be published.33 34 To ensure that all relevant studies were identified, we performed a robustness check of our search strategy, searching both Scopus and ClinicalTrials.gov for all indications for which we had found no postapproval studies (47 of 123). We found no additional studies that had been published in a journal not indexed by Medline, four unpublished studies reporting results with statistical analyses affecting three indications on ClinicalTrials.gov, and five additional studies affecting four indications that had been published in a journal indexed on Medline that were not identified on our original search. We included these published studies in our results; only three offered evidence on drug efficacy, two of which showed superiority of the drug and one found inferiority. Thus, we do not expect our results or interpretation to be affected by adopting a wider search strategy that includes Scopus and ClinicalTrials.gov.
Secondly, we limited our analysis to prospective studies that examined efficacy for the first approved indication and used a comparator as a control, as we think that this generates the most informative, high quality data to understand drug efficacy. However, non-comparative or retrospective analyses might also have been published and provide important insights, as might studies for other indications, although our classification of indication was broad to include as many publications as could possibly be relevant. Thirdly, we included only English language publications. Fourthly, we completed our systematic reviews on 31 December 2014, and relevant clinical studies might have been published since then. However, the median follow-up period after approval exceeded five years, and the value of evidence to inform and direct practice diminishes as time passes. Finally, we only examined novel drugs approved on the basis of limited evidence. Although these approvals accounted for more than half of the FDA’s approvals from 2005 to 2012, our findings may not be applicable to drugs approved on the basis of multiple pivotal trials or trials focused on clinical outcomes.
Conclusion
The quantity and quality of postapproval clinical evidence varied substantially for novel drugs approved by the FDA on the basis of a single pivotal trial, pivotal trials that used surrogate markers of disease, or both. Fewer than 10% of approved indications had one or more published randomized controlled, double blind study showing superior efficacy based on clinical outcomes that examined the same indication for which the drug was first approved by the FDA after a median of 5.5 years after approval. These findings should inform both clinical decision making and regulatory policy regarding requirements before and after approval of novel drugs.
Web extra.
Extra material supplied by authors
Appendix: supplementary material
Notes
Contributors: AMP and JSR conceived and designed this study, conducted the statistical analysis, and drafted the manuscript. AMP and JAA acquired the data. JSR provided supervision. All authors participated in the analysis and interpretation of the data and critically revised the manuscript for important intellectual content. AMP and JSR had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. AMP and JSR are guarantors.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: HMK and JSR receive support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the Centers of Medicare and Medicaid Services to develop and maintain performance measures that are used for public reporting, from Medtronic and the FDA to develop methods for postmarket surveillance of medical devices, from the Blue Cross Blue Shield Association to better understand medical technology evaluation, and from the Laura and John Arnold Foundation to support the Collaboration on Research Integrity and Transparency at Yale. HMK reports that he chairs a scientific advisory board for UnitedHealthcare. NDS and JSR receive support from the FDA to establish the Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation. AMP, NSD, and JAA declareno support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.”
Funding: This project was supported in part by the Robert E Leet and Clara Guthrie Patterson Trust Awards Program in Clinical Research, Bank of America, NA, Trustee. The sponsor had no role in the design of the study, analysis or interpretation of findings, or drafting the manuscript and did not review or approve the manuscript before submission. The authors assume full responsibility for the accuracy and completeness of the ideas presented. AMP was supported in part by the SUNY Downstate College of Medicine Alumni Fund Summer Research Fellowship. Ethical approval: The Yale University Institutional Review Board approved this study and considered it non-human subjects research.
Transparency: AMP and JSR, on behalf of the authors, affirm that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Data sharing: Requests for collected data and statistical code can be made to the corresponding author at [email protected].
References
Articles from The BMJ are provided here courtesy of BMJ Publishing Group
Full text links
Read article at publisher's site: https://doi.org/10.1136/bmj.j1680
Read article for free, from open access legal sources, via Unpaywall:
https://www.bmj.com/content/bmj/357/bmj.j1680.full.pdf
Citations & impact
Impact metrics
Citations of article over time
Alternative metrics
Smart citations by scite.ai
Explore citation contexts and check if this article has been
supported or disputed.
https://scite.ai/reports/10.1136/bmj.j1680
Article citations
Design and Features of Pricing and Payment Schemes for Health Technologies: A Scoping Review and a Proposal for a Flexible Need-Driven Classification.
Pharmacoeconomics, 15 Oct 2024
Cited by: 0 articles | PMID: 39405025
Review
Analysis of post-marketing requirements for oncology drug conditional approvals in the United States and China.
Heliyon, 10(15):e35454, 30 Jul 2024
Cited by: 0 articles | PMID: 39170111 | PMCID: PMC11336774
Mapping Payment and Pricing Schemes for Health Innovation: Protocol of a Scoping Literature Review.
Pharmacoecon Open, 8(5):765-772, 21 May 2024
Cited by: 1 article | PMID: 38773050 | PMCID: PMC11362434
Review
Free full text in Europe PMCTowards a coordinated approach for managing accelerated patient access to potentially beneficial medicines: reporting the perspectives of a multi-stakeholder, international workshop.
Health Aff Sch, 2(6):qxae069, 22 May 2024
Cited by: 1 article | PMID: 38855055
The Critical Intersect of Regulations, Health Technology Assessment, and Drug Safety Assessments.
Drug Saf, 47(4):289-299, 19 Dec 2023
Cited by: 0 articles | PMID: 38113017
Go to all (52) article citations
Data
Data behind the article
This data has been text mined from the article, or deposited into data resources.
BioStudies: supplemental material and supporting data
Similar Articles
To arrive at the top five similar articles we use a word-weighted algorithm to compare words from the Title and Abstract of each citation.
Comparison of treatment effect sizes from pivotal and postapproval trials of novel therapeutics approved by the FDA based on surrogate markers of disease: a meta-epidemiological study.
BMC Med, 16(1):45, 21 Mar 2018
Cited by: 5 articles | PMID: 29562926 | PMCID: PMC5863466
Premarket Pivotal Trial End Points and Postmarketing Requirements for FDA Breakthrough Therapies.
JAMA Netw Open, 7(8):e2430486, 01 Aug 2024
Cited by: 0 articles | PMID: 39190303 | PMCID: PMC11350476
Analysis of Postapproval Clinical Trials of Therapeutics Approved by the US Food and Drug Administration Without Clinical Postmarketing Requirements or Commitments.
JAMA Netw Open, 2(5):e193410, 03 May 2019
Cited by: 10 articles | PMID: 31074812 | PMCID: PMC6512282
Characteristics of efficacy evidence supporting approval of supplemental indications for prescription drugs in United States, 2005-14: systematic review.
BMJ, 351:h4679, 23 Sep 2015
Cited by: 18 articles | PMID: 26400844 | PMCID: PMC4580725
Review
Free full text in Europe PMC
Funding
Funders who supported this work.
NCATS NIH HHS (1)
Grant ID: UL1 TR001863
