Abstract
Objective. We previously developed and performed an initial validation of a screening questionnaire, the Toronto Psoriatic Arthritis Screen (ToPAS), for psoriatic arthritis (PsA). In our original analysis, we found that the index constructed appeared to discriminate well between those with a confirmed diagnosis of PsA and those without PsA in various clinical settings. However, it was suggested that ToPAS would benefit from additional refinement to the questions and the scoring system, because items pertaining to axial involvement were not included in our original index. Subsequently, a second version of ToPAS was developed, ToPAS 2, which incorporated the suggested refinements. We aimed to validate ToPAS 2 as a screening instrument for PsA.
Methods. ToPAS 2 was administered to 3 “diagnostic” groups of individuals — patients with PsA, patients with psoriasis, and healthy controls, and the data collected were analyzed.
Results. It was found that the new version of ToPAS, ToPAS 2, again performed well, with the axial domain now featuring in the new scoring system. The constructed index, ToPAS2_cap, had an overall area under the receiver-operation curve of 0.910, with overall values of sensitivity and specificity, at a cutpoint of 8 (or 7), of 87.2% (92.0%) and 82.7% (77.2%), respectively.
Conclusion. ToPAS 2 shows much promise as a screening instrument for identifying PsA both in people with psoriasis and in individuals from the general population. Its performance against other proposed screening instruments for PsA should be evaluated in other clinics and for other study designs.
The Classification of Psoriatic Arthritis (CASPAR) criteria have facilitated the recognition of psoriatic arthritis (PsA), but a rheumatologist is required to establish diagnosis1. The Toronto Psoriatic Arthritis Screen (ToPAS) was developed as a tool to screen for the presence of PsA2. It was developed for use in clinical settings and for epidemiological studies where it would be almost impossible to apply the CASPAR criteria because of the requirement that all patients be reviewed by a rheumatologist. ToPAS was developed through a review of items seen in patients with PsA and based on expert opinion from rheumatologists, dermatologists, and epidemiologists and covered 4 domains — skin, joint, nail, and spine. It was administered to 5 groups of patients — those with PsA, those with psoriasis, those from a general dermatology clinic, a general rheumatology clinic, and family medicine clinics. Based on the data collected, the ToPAS simplified scoring index had high overall discriminatory power [area under the curve (AUC) of 0.954] and had, at a cutpoint of 8, overall sensitivity and specificity of 86.8% and 93.1%, respectively2.
However, it was suggested that the instrument would benefit from additional pictures depicting joint inflammation and dactylitis, and that questions regarding axial involvement required refinement in light of the absence of the spine domain in the ToPAS index. Consequently, an adapted version of ToPAS, ToPAS 2, was developed.
The aim of our investigation was to validate ToPAS 2 as a screening tool for PsA.
MATERIALS AND METHODS
Refinement of ToPAS
The following changes were made to ToPAS to produce ToPAS 2: (1) an additional image of cutaneous psoriasis was added to question 1; (2) a succinct question on family history of psoriasis was added (question 5); (3) images of inflamed joints and dactylitis were added to questions 7 and 8; and (4) the spine domain questions were refined to ask about pain and stiffness together and whether there was improvement with activity (questions 9 to 11).
ToPAS 2 comprises 13 questions: 12 relating to the 4 domains of skin, joints, nail, and spine, and an additional question (question 13) on diagnosis of certain rheumatic and rheumatic-related conditions. The questionnaire is available from www.uhnres.utoronto.ca/studies/cpsrd/topasii_forWeb2.htm.
Patients
ToPAS 2 was administered to 3 diagnostic groups: patients with a confirmed diagnosis of PsA, patients with psoriasis (without PsA), and healthy controls. These patients and healthy controls came from the Toronto PsA Clinic of Toronto Western Hospital, from a prospective study of psoriasis at Toronto Western Hospital, and from psoriatic arthritis and psoriasis family members.
Confirmatory diagnosis of PsA
PsA diagnosis was based on a rheumatologist’s evaluation and all patients with PsA fulfilled CASPAR criteria for PsA1.
Statistical methods
To evaluate ToPAS 2, domains consistent with those created in ToPAS (i.e., skin, joint, nail, and spine) were constructed based on the questions in ToPAS 2, where a “yes” response to a question scored 1 and otherwise the question scored zero. The 4 domains of skin, joint, nail, and spine were constructed from the following questions:
-
Skin domain, [skin], is based on summing questions 1, 3, 4, and 5 for a total 0–4. Scores of 4 were recoded to 3 to cap [skin] at a maximum score of 3. If all 4 questions remained unanswered by the subject then [skin] was coded as missing.
-
Joint domain, [joint], is based on questions 6, 7, 8, and 12. These questions were summed and then capped to a maximum of 3. If all 4 questions were unanswered, then [joint] was recorded as missing.
-
Nail domain, [nail], is based on either of questions 2a or 2b being answered “yes,” with a maximum score of 1. If question 2 remained unanswered, then [nail] was coded as missing.
-
Spine domain, [spine], score ranged from 0 to 2 and was based on questions 9 to 11. A score of 0, 1, or 2 was given if none, 1, or more than 1 question was answered “yes,” respectively. The score was missing if these questions were all left unanswered.
The statistical analysis was based on the operating characteristics of the newly adapted scoring index, defined, in the same spirit as the original, as
where weights of 1 are applied to skin, nail, and spine domains and a weight of 2 is applied to the joint domain for a possible range of 0–12. Our intention was not to completely revamp our original ToPAS index, which is already in current use, but to refine it. However, an alternative version of ToPAS 2 index, ToPAS2_uncap, was considered. This version did not cap the various domains but allowed the scores in them to reflect the actual number of items answered positively. Thus the skin and joint domains had scores ranging from 0 to 4, the nail domain had scores ranging from 0 to 2, and the spine domain had scores ranging from 0 to 3. These domains were weighted as for ToPAS2_cap and summed to obtain ToPAS2_uncap. ToPAS2_uncap scores ranged from 0 to 17.
The operating characteristics of the indexes were based on the area under the receiver operating characteristic (ROC) curve (AUC), and the sensitivity and the specificity at a number of cutpoints.
RESULTS
ToPAS 2 was administered to 556 individuals, of whom 131 had PsA, 336 had psoriasis, and 89 were healthy controls, unaffected by psoriasis or PsA. One hundred of the 131 PsA diagnosed individuals were from the Toronto PsA Clinic. Of the 336 psoriasis-only individuals, 307 came from the prospective study of patients with psoriasis. This study had 323 participants, of whom 16 had a confirmed diagnosis of PsA at the time of administration of the ToPAS 2 questionnaire. Table 1 shows the study characteristics of these 556 individuals by diagnostic group at the time of administration of the ToPAS 2 questionnaire. The percentages of males in the 3 diagnostic groups were 57%, 55%, and 36% for the PsA, psoriasis, and unaffected groups, respectively. Over 40% of individuals in the PsA and psoriasis groups showed nail involvement when this information was collected. The mean Psoriasis Area and Severity Index scores (SD) in these 2 groups were, respectively, 4.00 (6.53) and 5.07 (4.93). For patients with PsA, the mean disease durations were 20.9 years and 11.7 years regarding the skin and arthritis aspects of the disease, respectively. The psoriasis group had mean disease duration of 18.0 years.
Characteristics of 556 patients completing version 2 of the Toronto Psoriatic Arthritis Screening questionnaire.
Table 2 cross-tabulates the uncapped scores for the 4 domains (obtained by summing all the relevant questions) by diagnostic group. It is clear from this table that the distributions of scores from the nail, spine, and joint domains for the patients with PsA are very different from those from the Psoriasis and Unaffected groups. As expected, no statistically significant differences in the distribution of scores from the skin domain were found between the PsA and psoriasis groups (p = 0.28, chi-squared test on 4 df).
Cross-tabulation of the uncapped scores from the 4 domains by diagnostic group. Data are n (%).
Table 3 presents the operating characteristics of the newly adapted ToPAS 2 screening index, ToPAS2_cap, for the overall comparison of the 131 patients with a confirmed diagnosis of PsA to the remaining 425, and for the 2 separate comparisons of the patients with PsA to the 336 patients with psoriasis and to the 89 unaffected controls. The overall discriminatory power of ToPAS2_cap was 0.910 based on the AUC. At a cutpoint of 8, as originally chosen for the ToPAS scoring index, the overall sensitivity and specificity were 87.2% and 82.7%, respectively. At a cutpoint of 7, the sensitivity increased to 92.0% and the specificity decreased to 77.2%. AUC values were 0.898 and 0.954 for the other 2 comparisons, with corresponding specificities of 80.2% and 92.1% (at a cutpoint of 8). The cutpoint of 8 was found to maximize the Youden’s index, defined as sensitivity + specificity − 1, over the values of ToPAS2_cap for the PsA versus the rest and PsA versus psoriasis comparisons. A cutpoint of 7 was found to maximize the Youden’s index for the PsA versus unaffected contrast.
Operating characteristics of ToPAS2_cap, ToPAS2_uncap, and ToPAS2_orig for the 3 comparisons: PsA versus the rest, PsA versus psoriasis, and PsA versus unaffected.
The operating characteristics of the uncapped version of the ToPAS 2 index, ToPAS2_uncap, gave AUC that were higher than for the capped version (Table 3). This was unsurprising because of the higher resolution of the ToPAS2_uncap scale. A cutpoint of 8 was found to maximize Youden’s index for the overall comparison of PsA against the rest. For the comparison of PsA against psoriasis, the optimal cutpoint was 9, while for the comparison of PsA with unaffected, the optimal cutpoint was 8.
If the scoring system of ToPAS 2 is redefined by excluding the spine domain, to correspond to the original ToPAS index with 3 domains (and named ToPAS2_orig = [skin] + [nail] + 2 × [joint]), then its AUC values for the overall comparison of PsA against the rest, and the separate comparisons of PsA against psoriasis and PsA against unaffected (Table 3), tend to indicate less or similar discriminatory power compared to ToPAS2_cap. The corresponding sensitivities of ToPAS2_orig at a cutpoint of 8 (7) are all 81.1% (86.6%) and the specificities are, respectively, 87.8% (83.7%), 85.7% (80.5%), and 95.5% (95.5%), reflecting decreased sensitivity to identify subjects with PsA but increased specificity to correctly identify patients without PsA compared to ToPAS2_cap at the same cutpoint. The optimal cutpoint for ToPAS2_orig based on maximizing Youden’s index was 7. The ROC curves for the 3 scoring indices are shown in Figures 1 to 3.
Receiver operating characteristic curve for the Toronto Psoriatic Arthritis Screen 2 (ToPAS 2). PV: predictive value.
Receiver operating characteristic curve for the uncapped Toronto Psoriatic Arthritis Screen 2 index, ToPAS2_uncap. PV: predictive value.
Receiver operating characteristic curve for the original Toronto Psoriatic Arthritis Screen index, ToPAS2_orig. PV: predictive value.
DISCUSSION
PsA may be a severe disease leading to joint damage, reduced quality of life, and increased mortality3. It is clear that early diagnosis and treatment are crucial to avoid untoward outcomes. Studies have demonstrated that patients who are reviewed by rheumatologists earlier in the course of their disease fare better4,5. While the CASPAR criteria facilitate the diagnosis of PsA, they require a rheumatologist to identify inflammatory arthritis. Because it is not possible for all patients with psoriasis to be seen by a rheumatologist, it would be helpful for primary care physicians and dermatologists, who usually see these patients first, to identify patients suspected of having PsA and refer them for rheumatological consultation.
A number of screening tools have been developed to identify PsA in patients with psoriasis6,7. The original ToPAS was developed to identify PsA both among patients with psoriasis and in individuals in the general population. ToPAS was found to be highly sensitive and specific in its development and initial validation2. However, when tested in other clinics, the sensitivity and specificity were not as high8,9,10.
ToPAS 2 was developed to provide better face validity to the instrument, because the original ToPAS index did not include spinal involvement. Additionally, ToPAS 2 incorporates further pictures of cutaneous psoriasis, joint inflammation, and dactylitis to improve its performance, as well as more carefully worded questions regarding spinal involvement. These refinements led us to investigate 2 indices for ToPAS 2, ToPAS2_cap, and ToPAS2_uncap, which incorporated the spine domain in their scoring systems. Although there appears to be very little to choose between ToPAS2_cap and ToPAS2_uncap, we would recommend ToPAS2_cap because (1) it is more consistent with the original ToPAS index, (2) overall it tends to better balance (optimize) sensitivity and specificity, with the sensitivity for ToPAS2_cap being larger at the cutpoint of 8 than the sensitivity for ToPAS2_uncap at the comparable cutpoint of 8, and (3) it is less overly influenced/dictated by the weight of 2 attached to the joint domain when declaring an individual as having PsA. ToPAS2_cap proved to have excellent overall sensitivity (87.2%) at a cutpoint of 8. Its specificity at this chosen cutpoint was also high, and is most impressive in unaffected individuals, suggesting that it may function well in epidemiological studies.
Overall, ToPAS 2 has high sensitivity and specificity. It is important for a screening instrument to be highly sensitive, because one would not want to miss people who have the disease. Thus, in this regard, the scoring system for ToPAS 2 outperforms the original scoring system for ToPAS. The specificity is a bit lower, particularly among patients with psoriasis, presumably because the instrument screens patients positive for psoriasis with other forms of arthritis. For the purposes of identifying patients whom dermatologists or primary care physicians should refer to a rheumatologist, the specificity is not as important, because patients with rheumatological disorders other than PsA would benefit from a rheumatologist’s consultation.
Nevertheless, it is important in other settings (e.g., clinical trials) to have a reasonably high specificity, to avoid having an excessive number of non-PsA patients referred to a rheumatologist to determine diagnosis. Importantly, ToPAS 2 was highly specific (92.1%) when comparing patients with PsA to unaffected individuals.
We recognize the limitation of adopting a retrospective case-control type design to investigate the performance of ToPAS 2, in particular regarding answering questions involving the performance of our instrument in screening for PsA in an unselected psoriasis population. Unfortunately, we could not establish a representative cohort of patients with psoriasis from the Toronto Western Hospital Psoriasis clinic for use in evaluating ToPAS 2. However, we could look at the data from our prospective study of 323 psoriasis individuals, of whom only 16 (5%) had a confirmed diagnosis of PsA at the time of administration of the ToPAS 2 questionnaire. These data showed that, at a cutpoint of 8, ToPAS2_cap and ToPAS2_uncap had estimated sensitivities of 66.7% (95% CI: 40.6%–85.4%) and 80% (95% CI: 53.0%–93.4%), respectively, and estimated specificities of 80.6% (95% CI: 75.7%–84.7%) and 72.2% (95% CI: 66.9%–77.0%), respectively. However, there is great uncertainty attached to the estimates of the sensitivities because these are based on a small number of individuals with a confirmed diagnosis of PsA (i.e., 15 of the 16; the 2 indices could not be calculated for 1 subject) from the prospective psoriasis study.
Based on this investigation, we suggest that the ToPAS 2 screening questionnaire and the ToPAS2_cap scoring index show much promise as screening tools for identifying PsA both in patients with psoriasis and in individuals in the general population. Their performance against other proposed screening instruments for PsA should be evaluated in other clinics and for other study designs.
Footnotes
-
The Psoriatic Arthritis Program is funded in part by The Arthritis Society, the Canadian Institutes of Health Research, and the Krembil Foundation. Prof. V.T. Farewell and B.D. Tom were supported by the UK Medical Research Council (Unit Programme numbers U105261167 and MC_UP_1302/3).
- Accepted for publication January 16, 2015.
In this issue
