Is Real-world Evidence Really Real?

Many important questions in medical research are investigated based on observational data, including “real-world evidence” studies.¹ In fact, the vast majority of published medical research relies on data obtained from observational studies.² This has been acknowledged by the increasing interest, focus, and acceptance of these types of studies. Although observational studies do have obvious weaknesses in the study design (in particular, inference of causality), it seems that the medical research paradigm has changed during the past few years. Focus has shifted from the downsides of observational studies to recognizing that they are important complements to randomized controlled clinical trials (RCTs).

While biologic disease-modifying antirheumatic drugs (bDMARDs) have clinically proven efficacy in RCTs, results from RCTs may not be directly applicable to patients seen in a real-life setting, for example, because patients eligible for RCTs may have fewer comorbidities than those seen in daily practice. Multiple parallel cohort studies (one for each drug) may provide complementary information on bDMARDs in clinical practice. The emphasis of registers is primarily on monitoring long-term safety, and in particular, safety signals that RCTs are not powered to detect. However, other aspects can also be studied, such as the possibility of medications losing their effectiveness after long-term use.

In this issue of The Journal of Rheumatology, treatment response and real-world persistence of bDMARD therapy in an Australian cohort of patients with ankylosing spondylitis is described.³ Here, the authors showed the same pattern of response as has been previously described in many other cohorts—the first bDMARD used showed greater efficacy and persistence than the second or third agent used. However, they also found that treatment persistence in the overall first-line population was longer among patients receiving golimumab (GOL) than for patients receiving other bDMARDs. This was not reported in another retrospective cohort analysis of prescriptions recorded by the Australian Commonwealth Department of Human Services (Pharmaceutical Benefits Scheme), where no significant differences were found between first bDMARD persistence of adalimumab, etanercept, or GOL in patients with AS.⁴ The disparity between this and the current study regarding first bDMARD persistence clearly illustrates the reality of real-world evidence—apparent inconsistency.

A very important aspect when interpreting registry data is the lack of randomization between treatment groups. If no randomization is performed, the clinician will choose a treatment based on many different aspects such as efficacy, practicability, disease severity, or comorbidity. For example, a patient with very severe disease activity is unlikely to be prescribed a bDMARD with only moderate efficacy. Moreover, unspecified cut-offs for determining when to switch therapy may influence study results, and treatment discontinuation can be due to factors unrelated to efficacy (e.g., pregnancy or the need for major surgery). Further, dose escalation or shortening of the dosing intervals (a way to extend treatment persistence) may occur more frequently with some drugs than others due to administration form or cost.

Indeed, there is a need for in-depth understanding of data sources outside normal epidemiologic understanding and craftmanship. The critical reader should always ask themselves the following questions:

• Are the data collected indirectly from other registers developed for purposes other than addressing the research question?

• Is there a payer’s bias in the tender system?

• What is the completeness of data?

• Who funded the study?

• What are the societal healthcare settings in which the data were gathered?

As pharmacovigilance systems, biologic registries can provide credible, reliable, and epidemiologically sound data, but such registries are not designed to provide conclusive evidence of treatment efficacy or effectiveness. Importantly, they cannot serve as replacements for clinical trials.

Finally, it should be stressed that real-world evidence or observational data should be reported transparently so that readers can follow what was planned, done, reported, and concluded; acknowledged guidelines on how to report and conduct observational studies should also be followed. The reliability of research depends on a critical assessment by others of the strengths and weaknesses in study design, conduct, and analysis. This also applies to a priori planning of analysis of data already on file in large registers. It is good to keep an open mind and allow for serendipity; however, we should work prospectively with observational data and not be driven by mass testing and deductions from results of this.

• © 2022 by the Journal of Rheumatology