To the Editor:
The growth of systematic reviews and metaanalyses (SRMA) has outpaced the growth of randomized clinical trials (RCT) in many medicine subspecialties1. This may reflect technological advances in SRMA production, fewer barriers to publish, or academic pressure to produce citations2. The value of disproportionate SRMA growth has been brought into question3, but the nature of RCT growth has undergone less scrutiny. In rheumatology, nearly 4 in 5 RCT receive pharmaceutical industry funding4, which could influence the relative proportion of early-stage efficacy studies as opposed to postmarketing safety studies. In this letter we describe the relative growth of rheumatology RCT and SRMA, as well as the phase of clinical trials over time, neither of which have been previously assessed in the field of rheumatology.
We conducted a cross-sectional study using the R package RISmed (R Foundation for Statistical Computing), which extracted bibliographic content from the database PubMed. The inclusion period began on January 1, 1995, to account for systematic errors in PubMed’s categorization of SRMA3, and ended on June 31, 2017, to account for delays in medical subject headings (MeSH) indexing. SRMA were searched as a single category using “Systematic Review[ptyp]” OR “Meta Analysis[ptyp].” RCT were searched using “Randomized Controlled Trial[ptyp].” The phase of clinical trial was searched using a query for each phase (phase I, phase II, phase III, phase IV).
The following MeSH headings were used to identify rheumatology manuscripts: “Rheumatic Diseases[Majr],” “Arthritis[Majr],” “Polychondritis, Relapsing[Majr],” “Dermatomyositis[Majr],” “Lupus Erythematosus, Cutaneous[Majr],” “Lupus Erythematosus, Systemic[Majr],” “Mixed Connective Tissue Disease[Majr],” “Scleroderma, Systemic[Majr],” “Undifferentiated Connective Tissue Diseases[Majr],” “Vasculitis[Majr],” “Crystal Arthropathies[Majr],” and “Immunoglobulin G4-Related Disease[Majr].” The ratio of SRMA to RCT was calculated by dividing SRMA by RCT (Figure 1B). Analyses were performed on RStudio v1.2.5033 (R Foundation for Statistical Computing).
(A) Yearly count of systemic reviews and metaanalyses (SRMA) to randomized clinical trials (RCT) from 1995 to 1997. (B) Ratio of SRMA to RCT, calculated by dividing yearly SRMA by yearly RCT. (C) Yearly count of clinical trials, stratified by phase of clinical trial.
From 1995 to 2017, we identified 3529 SRMA and 7469 RCT. The yearly production of SRMAS (11 in 1995, to 420 in 2017, 3718% growth) outpaced the yearly production of RCT (145 in 1995, to 452 in 2017, 212% growth; Figure 1A). The ratio of SRMA to RCT increased from 0.08 in 1995 to a nearly equivalent production of 0.93 in 2017 (Figure 1B). Over the same time period, 1169 trials received a MeSH heading denoting their clinical phase. These included 125 (10.7%) phase I trials, 506 (43.3%) phase II trials, 479 (41.0%) phase III trials, and 59 (5.1%) phase IV trials (Figure 1C).
Rheumatology SRMA have grown 15 times faster than rheumatology RCT and approached equivalent yearly production by 2017. Well-conducted SRMA on unaddressed clinical questions provide substantial value to rheumatology and should be applauded. Modern reporting guidelines5, preregistration of SRMA protocols6, and systematic review software have encouraged their production and improved their quality. However, it has been suggested that many SRMA are redundant or methodologically flawed3, primarily burnishing academic curriculum vitae or providing marketing tools for the pharmaceutical industry7. An ideal ratio of SRMA to RCT production is not known, but disproportionate SRMA growth of this magnitude may not provide commensurate value. Future work should investigate both the quality of SRMA in rheumatology and the degree to which they are influenced by industry sponsors.
RCT growth resulted almost exclusively from increases in phase II/III trials, which generate dose and efficacy data. Such trials may be underpowered to identify relevant safety signals and often do not reflect “real-world” settings8. Phase IV “postmarketing” trials may address these limitations, providing clinical effectiveness data from larger, heterogenous groups over longer periods of time. Even after the US Food and Drug Administration (FDA) received authorization to require postmarketing trials in 20079, publication of phase IV trials did not increase. This finding could be related to safer drugs that require less monitoring, but it seems more likely that the pharmaceutical industry is responding to public policy incentives. These incentives encourage phase III trials, which may result in FDA approval.
This study was limited to MeSH terms and published papers, which may not reflect the entire medical literature. These limitations notwithstanding, our data suggest that the incentives driving the current rheumatology research agenda have brought about marked growth of SRMA as compared to RCT and phase II/III trials as compared to phase IV trials. Rheumatologists should consider whether this represents an ideal allocation of research activity and public policymakers may be encouraged to require a higher proportion of phase IV trials.
Footnotes
Michael Putman is supported in part by Grant Number T32 AR007611-13 from the National Institute of Arthritis and Musculoskeletal and Skin Diseases.
- Copyright © 2021 by the Journal of Rheumatology
