Elsevier

Vaccine

Volume 31, Supplement 10, 30 December 2013, Pages K62-K73
Vaccine

Review
A systematic review of validated methods for identifying systemic lupus erythematosus (SLE) using administrative or claims data

https://doi.org/10.1016/j.vaccine.2013.06.104Get rights and content

Highlights

  • Few studies provide validated algorithms for identification of SLE in a broad based population.

  • The majority of studies assessed algorithms including ICD-9 code 710.0 in selected populations.

  • The selected populations are primarily characterized as those seen by a rheumatologist.

  • The PPV of ICD-9 code 710.0 in selected populations is in the range of 70–90%.

  • Of the limited data in general populations the PPV of ICD-9 code 710.0 is in the range of 50–60%.

Abstract

Purpose

To examine the validity of billing, procedural, or diagnosis code, or pharmacy claim-based algorithms used to identify patients with systemic lupus erythematosus (SLE) in administrative and claims databases.

Methods

We searched the MEDLINE database from 1991 to September 2012 using controlled vocabulary and key terms related to SLE. We also searched the reference lists of included studies. Two investigators independently assessed the full text of studies against pre-determined inclusion criteria. The two reviewers independently extracted data regarding participant and algorithm characteristics and assessed a study's methodologic rigor using a pre-defined approach.

Results

Twelve studies included validation statistics for the identification of SLE in administrative and claims databases. Seven of these studies used the ICD-9 code of 710.0 in selected populations of patients seen by a rheumatologist or patients who had experienced the complication of SLE-associated nephritis, other kidney disease, or pregnancy. The other studies looked at limited data in general populations. The algorithm in the selected populations had a positive predictive value (PPV) in the range of 70–90% and of the limited data in general populations it was in the range of 50–60%.

Conclusions

Few studies use rigorous methods to validate an algorithm for the identification of SLE in general populations. Algorithms including ICD-9 code of 710.0 in physician billing and hospitalization records have a PPV of approximately 60%. A requirement that the code is obtained from a record based on treatment by a rheumatologist increases the PPV of the algorithm but limits the generalizability in the general population.

Introduction

Mini-Sentinel, a pilot project sponsored by the United States Food and Drug Administration (FDA), aims to inform and facilitate the development of an active surveillance system, the Sentinel System, for monitoring the safety of FDA-regulated medical products [1]. Mini-Sentinel is one facet of the Sentinel Initiative, an FDA effort to develop a national electronic system that will complement existing methods of safety surveillance.

To support this goal, Mini-Sentinel uses administrative and claims data to examine relationships between medical product exposures and health outcomes [1], [2]. A first step in developing the Sentinel system is to understand the validity of algorithms (i.e., combinations of billing, procedural, or diagnosis codes, or pharmacy claims) for identifying health outcomes of interest in administrative data. Mini-Sentinel program collaborators selected health outcomes of interest using an expert elicitation process through which investigators developed a list of candidate outcomes based on input from global vaccine safety experts. A panel of 5 vaccine experts then prioritized the list via an iterative process using criteria including clinical severity, public health importance, incidence, and relevance. Two musculoskeletal conditions, systemic lupus erythematosus (SLE) and rheumatoid arthritis, were included on the list of conditions [3].

Understanding algorithms used to identify health outcomes helps to determine the validity of any safety signals observed in these data. Thus, the goal of this project was to identify algorithms used to detect SLE and describe the performance characteristics of these algorithms as reported by the studies in which they were used.

SLE is an autoimmune disease with diverse clinical manifestations in association with autoantibodies to components of the cell nucleus. The expression of tissue injury and clinical manifestations of SLE are believed to be determined by genetic, epigenetic, environmental, hormonal and immunoregulatory factors [4]. It occurs most commonly in young women with a peak incidence between the ages of 15 and 40 years and a female:male ratio of 6–10:1. In the United States, people of African, Hispanic, or Asian ancestry have a higher prevalence of SLE and greater involvement of vital organs compared to other racial or ethnic groups. The estimates of the prevalence of SLE in the United States vary widely with a reported range of as high as 1,500,000 to as low as 161,000 [5], [6]. The annual number of deaths with SLE as the underlying cause was reported as 879–1406 from 1979 to 1998, with the highest number reported among black women 45–64 years of age [7]. Patients with SLE have 80–90% survival at 10 years. The presentation of SLE is highly variable and can include various signs and symptoms involving many organ systems including dermatologic, musculoskeletal, renal, nervous, cardiovascular, and pulmonary systems. Considering the clinical heterogeneity of SLE the American College of Rheumatology established 11 criteria to improve the consistency of the diagnosis and to provide some standardization for entry into clinical trials or outcome studies. A definite diagnosis is considered to be made with 4 or more criteria occurring either simultaneously or in succession [8], [9]. The American College of Rheumatology Criteria (ACR) for systemic lupus erythematosus are provided in the appendices. Most patients with SLE have general constitutional symptoms including fatigue, malaise, fever, anorexia, and weight loss. The presence of anti-nuclear antibodies is the hallmark of the disease and is present in over 90% of patients.

Section snippets

Materials and methods

A detailed description of the methods for the project can be found in the accompanying paper by McPheeters et al. [10]. Briefly, we searched the MEDLINE database via the PubMed interface using the strategies outlined in Appendix A. We developed the search strategy by building on prior Mini-Sentinel approaches to searching [2]. We assessed the need to assess gray literature, including that located via Google Scholar, by testing prior approaches. We also tested EMBASE and other databases to

Results

Our searches identified 658 citations of which 50 met our inclusion criteria (Fig. 1). Among the 50 studies meeting our criteria, 12 described validation of the algorithm used to identify SLE cases. We focus on those studies in this review, and Table 1 summarizes these study characteristics. Characteristics of studies not describing validation of the algorithm are summarized in Table 2. The appendices include our search strategies and a list of studies not meeting our review criteria.

Five

Discussion

The main finding of this review is that few studies use rigorous methods to validate an algorithm for the identification of SLE in general populations. When administrative or claims data are screened for the presence of ICD-9 code of 710.0 in physician billing and hospitalization records in selected populations, for example those seen in a rheumatology clinic, the PPV is in the range of 70–90%. Of the limited data in general populations, the PPV was in the range of 50–60%. Given that vaccine

Conflict of interest

The authors have no conflicts to declare.

Funding source

Mini-Sentinel is funded by the Food and Drug Administration (FDA) through Department of Health and Human Services (HHS) Contract Number HHSF223200910006I. The views expressed in this document do not necessarily reflect the official policies of the Department of Health and Human Services, nor does mention of trade names, commercial practices, or organizations imply endorsement by the U.S. government.

Role of the funding source: FDA staff reviewed articles prior to publication but had no role in

Authorship statement

All authors declare that they have participated in: (1) the conception and design of the study, or acquisition of data, or analysis and interpretation of data, (2) drafting the article or revising it critically for important intellectual content, and (3) final approval of the version submitted.

References (59)

  • Trends in Deaths from Systemic Lupus Erythematosus – United States, 1979–1998

    MMWR Morb Mortal Wkly Rep

    (2002)
  • E.M. Tan et al.

    The 1982 revised criteria for the classification of systemic lupus erythematosus

    Arthritis Rheum

    (1982)
  • M.C. Hochberg

    Updating the American College of Rheumatology revised criteria for the classification of systemic lupus erythematosus

    Arthritis Rheum

    (1997)
  • McPheeters, M, Sathe, N., Jerome, R., Carnahan, R. Methods for Systematic Reviews of Administrative Database Studies...
  • S. Bernatsky et al.

    The accuracy of administrative data diagnoses of systemic autoimmune rheumatic diseases

    J Rheumatol

    (2011)
  • S. Bernatsky et al.

    A population-based assessment of systemic lupus erythematosus incidence and prevalence – results and implications of using administrative data for epidemiological studies

    Rheumatology (Oxford)

    (2007)
  • A.L. Naleway et al.

    Epidemiology of systemic lupus erythematosus in rural Wisconsin

    Lupus

    (2005)
  • M.R. Arbuckle et al.

    Rapid clinical progression to diagnosis among African-American men with systemic lupus erythematosus

    Lupus

    (2003)
  • S.E. Andrade et al.

    Validity of health plan and birth certificate data for pregnancy research

    Pharmacoepidemiol Drug Saf.

    (2013)
  • A. Broder et al.

    Undertreatment of disease activity in systemic lupus erythematosus patients with endstage renal failure is associated with increased all-cause mortality

    J Rheumatol

    (2011)
  • L.B. Chibnik et al.

    Identification and validation of lupus nephritis cases using administrative data

    Lupus

    (2010)
  • S.S. Lim et al.

    The Georgia Lupus Registry: accuracy of hospital discharge data in identifying systemic lupus erythematosus

    Arthritis Rheum

    (2007)
  • M.L. Sanchez et al.

    Can the weighted criteria improve our ability to capture a larger number of lupus patients into observational and interventional studies? A comparison with the American College of Rheumatology criteria

    Lupus

    (2003)
  • K.H. Costenbader et al.

    Defining lupus cases for clinical studies: the Boston weighted criteria for the classification of systemic lupus erythematosus

    J Rheumatol

    (2002)
  • J.N. Katz et al.

    Sensitivity and positive predictive value of Medicare Part B physician claims for rheumatologic diagnoses and procedures

    Arthritis Rheum

    (1997)
  • L.T. Hiraki et al.

    Prevalence, incidence, and demographics of systemic lupus erythematosus and lupus nephritis from 2000 to 2004 among children in the US medicaid beneficiary population

    Arthritis Rheum

    (2012)
  • M. Tanzer et al.

    Inpatient healthcare utilization by children and adolescents with systemic lupus erythematosus and kidney involvement

    Arthritis Care Res (Hoboken)

    (2012)
  • C.M. Vassileva et al.

    Influence of systemic lupus erythematosus on procedure selection and outcomes of patients undergoing isolated mitral valve surgery

    J Card Surg

    (2012)
  • C. Barnabe et al.

    Prevalence of systemic lupus erythematosus and systemic sclerosis in the First Nations population of Alberta, Canada

    Arthritis Care Res (Hoboken)

    (2012)
  • Cited by (75)

    • Autoimmune disease and risk of postpartum venous thromboembolism

      2023, Research and Practice in Thrombosis and Haemostasis
    • Infection detection in patients with systemic lupus erythematosus using a hospital administrative database

      2022, Revista Colombiana de Reumatologia
      Citation Excerpt :

      The second source of information we used was the review of medical records (contains all the clinical characteristics of the patient registered by the treating physician, including diagnostic test results), to confirm the diagnoses, and to extract the laboratory results as well as the data related to lupus activity. To improve the accuracy of SLE diagnosis in our data source we used an algorithm described by Moore et al.11 that classifies patients as having SLE if they satisfy the following condition: the case when a patient has records with those codes at least in two outpatient claims, such that the difference length of time between them could not be neither less than 30 days nor greater than 2 years. The associated codes to SLE were M320, M321, M328 and M329 and if he receives at least one immunosuppressive medicine, for the process of validation of this algorithm the author used Bayesian estimation techniques resulting in an estimated specificity of 99.9% (95% CI 99.8% to 100%) and an estimated sensitivity of 45% for physician billing data and sensitivity of 42–6% for hospital data.12

    View all citing articles on Scopus
    View full text