Abstract
Objectives. We assessed the accuracy of case definition algorithms for psoriasis and psoriatic arthritis (PsA) in health administrative data and used primary care electronic medical records (EMR) to describe disease and treatment characteristics of these patients.
Methods. We randomly sampled 30,424 adult Ontario residents from the Electronic Medical Record Primary Care database and identified 2215 patients with any possible psoriatic disease–related terms in their EMR. The relevant patient records were chart abstracted to confirm diagnoses of psoriasis or PsA. This validation set was then linked to health administrative data to assess the performance of different algorithms for physician billing diagnosis codes, hospitalization diagnosis codes, and medications for psoriatic disease. We report the performance of selected case definition algorithms and describe the disease characteristics of the validation set.
Results. Our reference standard identified 1028 patients with psoriasis and 77 patients with PsA, for an overall prevalence of 3.4% for psoriasis and 0.3% for PsA. Most patients with PsA (66%) had a rheumatology-confirmed diagnosis, while only 30% of the patients with psoriasis had dermatology-confirmed diagnosis. The use of systemic medications was much more common with PsA than with psoriasis. All algorithms had excellent specificity (97–100%). The sensitivity and positive predictive value were moderate and varied between different algorithms (34–72%).
Conclusion. The accuracy of case definition algorithms for psoriasis and PsA varies widely. However, selected algorithms produced population prevalence estimates that were within the expected ranges, suggesting that they may be useful for future research purposes.
Psoriasis is an immune-mediated skin disease that tends to be chronic with an unpredictable course1. Psoriasis is associated with a significant effect on quality of life as well as comorbidities such as depression and cardiovascular diseases2,3,4. In addition, up to a third of patients with psoriasis develop an inflammatory arthritis termed psoriatic arthritis (PsA), which runs the course of a chronic, progressive disease, and can lead to severe joint damage and disability5.
Currently, there is limited information on the population-level epidemiology, quality of care, and disease outcomes of psoriasis and PsA in Canada. Canada’s public health system provides universal coverage for hospital and physician services, which allows secondary use of administrative health databases for research. Canadian administrative data have been used extensively in the past to investigate the disease outcomes, comorbidities, and quality of care for several rheumatic conditions. However, it has not been used to specifically investigate psoriasis and PsA. While administrative data provide an efficient source of population-based data, the databases were designed for administrative purposes and not for research6. The accuracy of diagnostic coding can be an issue, particularly since there are few incentives for physicians to code well when only information regarding the services provided is audited7. Therefore, it is important to establish methods to accurately identify patients with a particular medical condition in administrative data before these resources can be used for research.
We previously published a study on the prevalence and incidence of psoriatic disease in Ontario, Canada8. As part of that study, we developed methods to identify patients with psoriasis and PsA in administrative data. In this manuscript, we report the accuracy of various diagnostic coding algorithms for psoriasis and PsA. Additionally, we describe the characteristics of patients with psoriasis and PsA from the primary care setting.
METHODS
Study design and databases
Ontario is the most populated province in Canada, with over 13.4 million residents in 2018. All Ontarians are insured by the Ontario Health Insurance Plan (OHIP), a single-payer, universal health insurance that covers all hospital and physician services and procedures; however, outpatient prescription medications are funded only for patients aged 65 years or older and residents who have very high drug costs relative to their income. All healthcare encounters are recorded in administrative healthcare databases, which are linked using an encoded health insurance number that is unique to each Ontarian eligible to receive insured health services in the province.
The validation process was based on the initial identification of reference standard cases of psoriasis, PsA, and nonpsoriatic cases. Similar to previous validation studies of other case definitions for Ontario administrative data, we used the Electronic Medical Record Primary Care (EMRPC, also known as EMRALD) database, a primary care electronic medical record (EMR) database, for our reference standard9,10,11,12. At the time of our study, EMRPC included electronic clinical data from over 350 primary care physicians across Ontario and over 400,000 patients who were anonymously linked with provincial administrative databases12. Clinically relevant information in EMRPC includes all electronic data about primary care consultations, including current and past medical history, laboratory and imaging test results, prescriptions and consultation letters from specialists, and discharge summaries. We randomly sampled 30,424 adult Ontario residents from EMRPC and identified 2215 patients with any possible psoriatic disease–related terms in their EMR. The relevant patient records were chart abstracted to confirm diagnosis of psoriasis or PsA. The entire sample was then linked with the provincial administrative health databases using the unique health insurance number to identify the optimal combination of diagnostic billing codes and hospitalization data (“algorithms”) that would identify true cases.
All analyses were performed at ICES (www.ices.on.ca) using linked, coded datasets. Our study was approved by the Research Ethics Board at Sunnybrook Health Sciences Centre, Toronto, Canada.
Use of data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a research ethics board.
Creation of validation set
We derived a validation dataset that included “true positive” (patients with psoriasis and/or PsA) and “true negative” (patients without psoriasis or PsA) reference standards from EMRPC. First, we obtained a random sample of 30,424 patients from EMRPC (15% of the sample population) that included individuals aged 20 years or older who had a valid health insurance number and whose EMR start date was at least 2 years prior to the study date. Within this random sample of 30,424 individuals, we performed a targeted search for any possible psoriatic disease-related terms (Supplementary Table 1, available with the online version of this article) in the EMR. The search included structured fields (e.g., medical conditions list, family history, and prescriptions) and free text entries (e.g., family physician visit notes and consultation summaries from specialists). A total of 2215 patients with suspected psoriasis and/or PsA were identified.
We then performed a chart abstraction of these patients to determine whether they had psoriasis and/or PsA. We assumed that patients with no psoriasis-related terms in their EMR did not have psoriasis and/or PsA, and these were classified as true negative cases without reviewing their charts. Using a standardized data abstraction tool, 2215 patients had their entire medical records reviewed by one of 3 trained physicians who were blinded to the patients’ diagnoses codes in the administrative data. Each patient was classified as “definite,” “possible,” or “not” psoriasis and/or PsA based on the level of evidence in the medical chart using predefined criteria. The diagnosis of psoriasis relies on a typical clinical appearance, since widely validated classification criteria are unavailable13. Since the majority of patients with psoriasis in Ontario are managed by their primary care physician (PCP), we accepted a clinical diagnosis by any physician as the gold standard for psoriasis. For PsA, we accepted a clinical diagnosis by a rheumatologist or an internal medicine specialist as the gold standard for PsA. The highest levels of evidence to support a PsA were (1) diagnosis by a rheumatologist, or internal medicine specialist; or (2) a PCP-documented PsA diagnosis with supporting evidence (e.g., joint involvement, or treatment) but without a supporting specialist consultation note. When the diagnosis of PsA or psoriasis was unclear (“possible”) or changed over time, the case was reviewed by a rheumatologist with an expertise in psoriatic disease in order to classify the patient. If the uncertainty regarding the diagnosis of psoriasis was not resolved at this point, the case was discussed with a dermatologist with an expertise in psoriasis. The decision was based on the level of evidence available in the EMR.
In addition, we obtained information from the medical records regarding disease characteristics and use of medications. Interrater reliability was assessed by abstracting 10 charts by each of the 3 abstractors. κ scores for interrater reliability for the diagnosis of psoriasis and PsA showed perfect agreement (100%).
Development of algorithms in administrative data
Once the reference standard of 2215 patients were classified as having or not having psoriasis and/or PsA, we linked all 30,424 cases from EMRPC with the provincial administrative databases. Administrative data were obtained for these cases for the period of April 1, 1991, to March 31, 2015. The aim of this study was to determine the optimal combination of healthcare encounters to form the algorithms that would identify true cases, considering the reference standard diagnosis as the gold standard.
We used the OHIP Claims History Database to identify physician billing diagnosis codes. OHIP is the provincial insurance plan covering all Ontarians14. Physicians are reimbursed by submitting claims to OHIP for medical services provided. A diagnosis code is provided with each claim, which represents the main reason for the visit. These diagnoses are coded using a modification of the International Classification of Diseases Version 9 (ICD-9)15. This system includes a specific diagnosis code for psoriasis (696); however, there is no specific code for PsA. Therefore, we used a combination of codes for other seronegative spondyloarthritis (SpA; 721) and psoriasis (696) to identify patients with PsA.
Hospital diagnoses for psoriasis and PsA were identified using the Canadian Institute for Health Information’s Discharge Abstract Database, which contains detailed information regarding treatments and procedures rendered during all acute hospital admissions. Supplementary Tables 2 and 3 (available with the online version of this article) lists the ICD and OHIP diagnosis codes, and medication codes tested for the algorithm development.
We prespecified a set of over 300 algorithms using combinations of psoriasis and PsA diagnosis codes on physician bills, primary and secondary hospital discharge diagnoses, prescription drug claims, various time intervals between the healthcare services, and whether the service was provided by a relevant specialist (rheumatologist/internal medicine for PsA, dermatologist for psoriasis). The algorithms were applied to all administrative data for the study period (1991–2015).
Statistical analysis
Descriptive statistics were used to characterize the patients who comprised our reference standard in EMRPC. We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each algorithm. For the primary analysis, we considered only patients who were classified as “definite” psoriasis or PsA after the chart abstraction as cases for the corresponding diseases. We performed a sensitivity analysis considering both classifications of “definite” and “possible” as cases. We decided a priori to prefer algorithms yielding the highest combination of PPV and sensitivity. We reviewed the reasons for false positive and false negative cases using the optimal algorithms.
RESULTS
A total of 2215 patients with psoriatic disease–related terms in the EMR were identified out of a random sample of 30,424 patients in EMRPC (15% of the sample population; Figure 1). Based on our reference standard definition, we identified 1028 patients with definite psoriasis (401 with possible psoriasis) and 77 patients with definite PsA (63 with possible PsA), which resulted in overall psoriasis and PsA prevalence of 3.38% and 0.25%, respectively. The sex distribution was almost equal between males and females (47.1% of patients with psoriasis and 46.7% of patients with PsA were males).
Selection of the study population. EMR: electronic medical record; EMRPC: Electronic Medical Record Primary Care database; PsA: psoriatic arthritis.
Characteristics of patients with psoriatic disease in primary care
The disease characteristics of the patients based on EMR chart abstraction are shown in Table 1. Only 29.9% of the patients with psoriasis had a documented diagnosis by a dermatologist in the EMR. Of the patients with psoriasis, 14.7% were defined as having severe psoriasis, based on their use of systemic medications or phototherapy, or on their descriptions of severe or extensive skin psoriasis in the EMR. In the majority of the patients with PsA (66.2%), the diagnosis was documented by a rheumatologist. Most of the patients with available information about the nature of joint involvement had peripheral arthritis (94%) and 26.7% had axial involvement. Dactylitis and enthesitis were documented in 28.6% and 5%, respectively.
Characteristics of the patients with psoriasis and PsA in EMRPC (definite cases).
A total of 85.7% and 71.8% of the patients with PsA and psoriasis, respectively, were prescribed topical or systemic medications for their skin and/or joint disease (Table 2). However, while most of the patients with PsA were treated with systemic medications (58.4% nonbiologic systemic and 28.5% biologic medications), only a small proportion of patients with psoriasis were using systemic treatments or phototherapy (8.6% systemic nonbiologic, 4.6% biologic, and 6.2% phototherapy).
Treatments used by the reference standard cases for psoriasis and PsA (only definite cases included).
Performance of algorithms for psoriasis
The properties of selected psoriasis and PsA case definition algorithms are presented in Table 3 and Table 4. For “definite” psoriasis cases, all algorithms had excellent specificity (96.7–100%); however, sensitivity was low to moderate and varied between the different algorithms (33.8–71.2%), depending on the number of psoriasis diagnosis codes obtained from physician billing claims. The algorithms also had a low to moderate PPV for identifying psoriasis (42.7–73.2%). The PPV improved with algorithms that required multiple physician billing codes for psoriasis; however, algorithms with psoriasis codes by dermatologists did not improve the PPV and led to a reduction in sensitivity. Extending the duration of observation window for diagnosis codes improved the sensitivity. Sensitivity analysis that combined patients classified as “definite” and “possible” psoriasis as cases in the reference standard led to a slight improvement in the PPV (48.9–76.1%) while sensitivity was reduced (23.9–59.6%).
The accuracy of selected psoriasis case definition algorithms.
The accuracy of selected PsA case definition algorithms.
Performance of algorithms for PsA
Regarding PsA, similar to psoriasis, the algorithms had excellent specificity (99.9%); however, the sensitivity was about 50% while the PPV was moderate, ranging from 53.2% to 67.9%. The PPV improved with algorithms that included more PsA diagnosis codes, particularly when assigned by rheumatologists. Algorithms that incorporated information about prescription of psoriasis-specific medications resulted in only minor increase in the sensitivity. A sensitivity analysis that included patients classified as “definite” or “possible” PsA cases in the reference standard led to an improvement in PPV (62.3–72.7%) and a reduction in sensitivity (28.6–31.4%).
Discordance analysis
To further understand the potential causes for the discordance between selected optimal algorithms and the reference standard diagnosis, we reviewed the false positive and false negative cases. The optimal algorithm for psoriasis was “at least 1 hospitalization psoriasis code ever OR 3 physician diagnosis codes (claims) for psoriasis by any physician,” which had a sensitivity of 40.6%, PPV of 71%, and specificity of 99.4%. Using this algorithm, there were 609 patients who were classified as false negative (the administrative data algorithm failed to identify as psoriasis) and 170 patients that were misclassified as false positive cases (the administrative data algorithm mistakenly identified as psoriasis). Among those who were classified as false negative, 80% of the patients had never seen a dermatologist and were solely managed by their family physician, and 65% of them were not prescribed any psoriasis treatment. Therefore, these were largely patients with mild or inactive psoriasis. The majority of the false positive patients with psoriasis (85%) were patients whose administrative data diagnosis code predated the EMR data, indicating patients with a remote history of psoriasis that was inactive in recent years.
The optimal algorithm for PsA was “1 hospitalization PsA code ever OR 3 physician diagnosis codes for seronegative SpA (at least 1 by a rheumatologist) and 1 physician code for psoriasis.” This algorithm had a sensitivity of 48.1%, PPV of 67.3%, and specificity of 99.9%. The use of this administrative data algorithm failed to identify 40 patients (false negative) and wrongly identified 19 patients as having PsA (false positive). Of those who were identified as false negative, 23% of the patients did not have a definite diagnosis of psoriasis (no history or possible psoriasis) and were therefore not identified, as the algorithm required a psoriasis diagnosis code. A total of 45% had no consultation notes from rheumatologists in the EMR and 58% were not using any medications for PsA, so it is likely that these patients had mild PsA. Regarding the patients classified as false positive PsA, 26% were classified as possible PsA based on EMR data, while for 30%, the diagnosis of PsA was ruled out or they had other rheumatologic conditions (miscoding by the rheumatologist). In 14%, there was either no mention of PsA in EMR, or the diagnosis code claims predated EMR data.
DISCUSSION
Population-based information about the epidemiology and healthcare utilization of psoriatic disease in North America is limited. Administrative data can be used to improve these gaps in knowledge; however, the accuracy of tools used to identify patients needs to be validated. In our study, we described the validation process and identified the optimal algorithms for identifying patients with psoriasis and PsA in Ontario administrative health data. Additionally, we described the diseases’ characteristics, patients’ access to specialists, and medication use in a primary care setting.
Our study showed that the optimal algorithms for identifying psoriasis and PsA had excellent specificity, adequate PPV, and modest sensitivity. The tested algorithms for both psoriasis and PsA yielded lower PPV than that reported in other validation studies in administrative data16,17,18,19. This is likely attributed to different study samples (reference standards), which affected the disease prevalence and severity as well as the type of comparator groups. These factors have a significant effect on the performance of the algorithms tested. In general, sampling from specialty clinics results in falsely elevated PPV owing to a high prevalence of case patients. A common alternative approach for validation involves sampling patients by the presence or absence of diagnosis codes in administrative data and confirming the diagnosis disease status by chart review16,17. However, while this approach provides metrics such as PPV and NPV, unbiased estimates of the performance of the algorithm cannot be generated since the prevalence of the disease remains unknown. Our study used a random sample of 2 diseases with low prevalence in the primary care setting. This provided a relatively unbiased assessment of the accuracy of the algorithms to detect psoriasis and PsA and partially explains the lower sensitivity and PPV compared to previous validation studies.
The algorithm properties can also be discussed in the context of common clinical practice for psoriatic disease in Ontario. The chart abstraction revealed that the majority of the patients with psoriasis in Ontario are solely managed by their family physicians. Since physicians are required to enter only a single diagnosis code per visit, it is likely that family physicians enter diagnostic codes for other concomitant comorbidities when visits involve several health issues. This practice explains the relatively low sensitivity of the psoriasis algorithms and the lack of significant improvement in PPV in algorithms requiring administration of psoriasis code by a dermatologist. The addition of psoriasis-specific medications to the algorithms was tested; however, since the administrative database includes prescription drug claims for a subset of the population, this information did not significantly improve the properties of the algorithms. By an extension of the duration of the observation period for assigning > 1 diagnosis code of psoriasis to “ever” rather than shorter time periods, we allow patients with lower rate of healthcare encounters related to psoriasis to meet the criteria for inclusion in the cohort. This is expected to include more patients with milder psoriasis who tend to follow up less regularly.
Identification of PsA was further complicated by the lack of a specific OHIP diagnosis code for PsA. Consequently, we used a combination of diagnosis codes for psoriasis and seronegative SpA. Most patients with PsA were managed by rheumatologists, and diagnostic codes for SpA were rarely assigned by nonrheumatologists, likely increasing the accuracy of the diagnosis. However, the requirement for a psoriasis diagnostic code in addition to an SpA code explains the lower sensitivity, since many of the patients with PsA had mild psoriasis that may not have required medical attention and therefore did not result in a claims-based diagnosis. When we changed our definition of PsA based on the level of evidence in EMR to a more liberal definition of PsA, such as one that allows those with possible PsA diagnosis, the PPV increased, but the sensitivity was lower. Overall, the estimated population prevalence of psoriasis and PsA, as indicated by both the proportion of the reference standard and the number of patients identified with the condition using our optimal algorithms, were well within the expected range for the prevalence of psoriasis and PsA in the general population20,21,22.
This study also provided information about the pattern of psoriatic disease and the use of systemic medications in the primary care setting. The proportion of patients with PsA among psoriasis patients was 7%, which is lower than the reported prevalence typically found in a dermatological setting (20–30%)5,23. This estimate is close to the reported prevalence in a previous population-based study and may be attributed to the inclusion of milder psoriasis, which is associated with lower risk of PsA but also potentially the underdiagnosis of PsA24. Only 7.6% of the patients with psoriasis ever used systemic medications, and many were using these medications for their PsA. It is not possible to determine the precise proportion of patients with active moderate-to-severe psoriasis using EMR data; however, according to a 2014 study, 55% of patients with moderate-to-severe psoriasis are not being treated to the established standards of care25. Patients’ low access to specialty care and low usage of systemic medications highlighted potential gaps in the care of patients with psoriasis in Ontario. In contrast, the majority of the patients with PsA in our reference standard set were seen by a rheumatologist, and a significant proportion of them were using systemic nonbiologic (58.4%) and biologic medications (28.5%).
Our study had several limitations. The case definition was based on chart abstraction and relied on the clinical diagnosis of the assessing physician. The diagnosis of psoriasis primarily relied on clinical assessment. In the absence of a diagnostic test, ambiguous cases may have been misclassified as psoriasis by less experienced clinicians. Misclassification is also a potential risk when documentation is missing in the EMR (e.g., lack of consultation notes from specialists). Strengths of our study include the randomly selected reference standard population and the rigorous case ascertainment. We also reported the accuracy of multiple administrative data algorithms, allowing researchers to select an algorithm based on the characteristics of the intended study population and study aims.
In summary, our study showed that administrative data algorithms can identify patients with psoriasis and PsA who receive regular primary care with adequate accuracy. These algorithms are less effective in identifying patients who do not have access to rheumatology and dermatology specialist care, possibly owing to a milder case of disease. These results will inform future population-based studies using Ontario administrative data and will fill the gaps in knowledge about the epidemiology of psoriatic disease.
ONLINE SUPPLEMENT
Supplementary material accompanies the online version of this article.
Footnotes
This study was supported by grants from the International Psoriasis and Arthritis Team (IPART) and the Canadian Rheumatology Association [Canadian Initiative for Outcome in Rheumatology Care (CIORA)]. L. Eder is supported by a New Investigator Salary Award from the Arthritis Society. K. Tu is supported by a research scholar award by the Department of Family and Community Medicine at the University of Toronto. This study was also supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Parts of this material are based on data and information compiled and provided by the MOHLTC and the Canadian Institute for Health Information. The analyses, conclusions, opinions, and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.
- Accepted for publication October 25, 2019.
REFERENCES
In this issue
