Abstract
Objective. To identify cutpoints reflecting Patient Acceptable Symptom State (PASS) and Minimal Clinically Important Improvement (MCII) in patient-reported multi-attribute health status classification systems and health status measurements among patients with rheumatoid arthritis (RA), ankylosing spondylitis (AS), and psoriatic arthritis (PsA).
Methods. We identified patients with RA, AS, and PsA from the Norwegian disease-modifying antirheumatic drug (DMARD) register (NOR-DMARD). The patients (n = 4225) had started with DMARD and responded to the PASS and MCII anchoring questions at the 3-month followup examination. Receiver operating characteristics (ROC) curves with 80% specificity and the 75th percentile approach were used to identify PASS and MCII cutpoints in the EuroQol-5 Dimensions (EQ-5D) and the Short-Form-6 Dimensions (SF-6D) indexes, but also in other patient-reported outcomes (joint pain and patient global visual analog scale and Modified Health Assessment Questionnaire).
Results. The PASS cutpoints estimated with 80% specificity were around 0.70 in EQ-5D in all diseases and around 0.65 in SF-6D. The cutpoints were around 0.65 and 0.60, respectively, when the 75th percentile approach was used. The MCII cutpoints assessed by 80% specificity varied from 0.10 to 0.19 in EQ-5D and from 0.07 to 0.10 in SF-6D.
Conclusion. The cutpoints for PASS in EQ-5D and SF-6D indicate that PASS corresponds to a health-related quality of life that is far from perfect health. Somewhat different cutpoints were identified for both PASS and MCII with 80% specificity versus the 75th percentile method.
- PATIENT OUTCOME ASSESSMENT
- PATIENT ACCEPTABLE SYMPTOM STATE
- MINIMAL CLINICALLY IMPORTANT IMPROVEMENT
- HEALTH-RELATED QUALITY OF LIFE
- DISEASE-MODIFYING ANTIRHEUMATIC DRUGS
- TUMOR NECROSIS FACTOR INHIBITORS
Rheumatoid arthritis (RA), ankylosing spondylitis (AS), and psoriatic arthritis (PsA) are chronic inflammatory rheumatic diseases with major impact on health-related quality of life (HRQOL)1–3. Because of the considerable influence on patients’ daily lives and the chronic character of these diseases, the patients’ perspective has increasingly been the focus over the last decade4. Multi-attribute health status classification systems such as EuroQol-5 Dimensions (EQ-5D) and other patient-reported outcomes regarding pain and function have been established as important elements in the evaluation of antirheumatic therapies and supplement assessments of inflammatory activity and radiographic damage5.
The concepts of Patient Acceptable Symptom State (PASS) and Minimal Clinically Important Improvement (MCII) can be used for reporting the proportion of patients in an acceptable state and the proportion of patients who have experienced an important improvement in the condition6. These concepts were thoroughly discussed at Outcome Measures in Rheumatology (OMERACT 8) and a survey following the meeting confirmed the relevance of using PASS and MCII in rheumatology4. A task force of the European League Against Rheumatism and the American College of Rheumatology (EULAR/ACR) has also highlighted the importance of reporting improvement, but also stated as well the sustainability of an acceptable level of the disease7.
Three methods have been used for identification of levels/changes in clinical measures and patient-reported outcomes that correspond to PASS/MCII. These methods include the 75th percentile approach and 2 approaches using receiver-operating characteristic (ROC) analyses8–10. ROC graphs have been used for assessing both the maximum accuracy8,9 and 80% specificity11.
Multi-attribute health status classification systems are used as composite measures for HRQOL since they summarize information from several domains into 1 single measure. Mapping multi-attribute health status classification systems to the concepts of MCII and PASS will improve our understanding of thresholds that correspond to the patients’ perception of acceptable levels of health and important improvements. The relevance is also related to the use of quality-adjusted life-years as the measure of health benefit in economic evaluation. Thus, our objective was to take advantage of data collected in a large treatment database8 to identify PASS and MCII cutpoints for multi-attribute health status classification systems and other patient-reported outcomes in patients with RA, PsA, and AS.
MATERIALS AND METHODS
Patient population
The Norwegian disease-modifying antirheumatic drug register (NOR-DMARD) was established in December 2000 and encompasses longitudinal information on patients who start with DMARD (including biological agents) for inflammatory arthropathies. The register includes patients from 5 rheumatology departments in Norway: Oslo, Drammen, Lillehammer, Trondheim, and Tromsø. In August 2008, there were 8078 cases registered. Assessments are performed at baseline, after 3, 6, and 12 months, and yearly thereafter. The register includes information about demographic variables, disease activity measures, and patient-reported outcomes8. All patients included in the register have signed an informed consent form. The register is approved by the Data Inspectorate and the ethics committee in Norway.
Study design
We identified patients with RA, AS, and PsA who had started with DMARD, had a 3-month followup examination, and had responded to the PASS and MCII anchoring questions. Two approaches were used to identify cutpoints for patient-reported outcomes corresponding to PASS and MCII.
Patient-reported outcomes
The EQ-5D is a multi-attribute health status classification system developed by the EuroQol Group, a multidisciplinary research group including participants from 5 European countries12. The instrument determines 5 dimensions of HRQOL: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression, classified in 3 levels (in total, 243 different states), complemented by a global health assessment on a visual analog scale (VAS)13. The utility weights give the single index score, where 0 represents death and 1 represents full health14. EQ-5D utility weights have been estimated in several countries with similar results. The commonly used UK weights go from −0.594 (extreme problems on all 5 dimensions) to 1.00 (no problem on any of the 5 dimensions), with 35% of all possible 243 states below 0. The EQ-5D has been examined regarding validity, reliability, and responsiveness in RA14.
The Short-Form-6 Dimensions (SF-6D) is another multi-attribute health status classification system, composed of 6 multilevel dimensions of health derived from the 8 dimensions of the Medical Outcomes Study Short Form-36 (SF-36) quality of life generic questionnaire15. An algorithm computes a utility score from 0 to 1 in SF-6D, with 0 for death and 1 for full health and no health states below 016. The current SF-6D algorithm implies that no health state is below 0.3 except for death15. The SF-6D has been shown to be responsive to clinically relevant changes in RA patients treated with infliximab17.
The Health Assessment Questionnaire (HAQ) is a physical function scale, which includes 20 questions concerning activities of daily living (ADL) classified in 8 categories18. The Modified Health Assessment Questionnaire (MHAQ) is a modified version of the HAQ and includes 8 ADL questions (1 for each category in the HAQ). The score in MHAQ ranges from 0 to 3 (0 = without any difficulty, 3 = unable to do)19.
Patients’ subjective assessments of pain and global health status were measured by a 100 mm VAS, where 0 = not present or excellent, and 100 = very severe or poor health20.
PASS and MCII anchoring questions
The question on perceived satisfactory health state was, “Is your current condition satisfactory, when you take your general functioning and your current pain into consideration?”10. The response options were “yes” or “no.” For the MCII the wording of the first question was, “Has the treatment in this follow-up study improved your health condition considerably?” and the response options were “yes” or “no.” The second question was, “Since you started treatment in this followup study, is your health condition improved, unchanged or worse?” The answers were measured on a 5-point Likert scale, from “much better” to “much worse.” The options “much better” and “better” were merged and regarded as considerable improvement in the analyses.
MCII is a rating of the extent to which patients’ health conditions have improved, and this type of rating may be called a transition rating. The MCII was checked for validity according to a method for transition ratings suggested by Guyatt, et al21.
Statistical analyses
The 75th percentile approach implies that among patients who consider their condition as satisfactory or have experienced an important improvement, the limit/cutoff is set at the 75th percentile of the scores from the patient-reported outcome22. Using the ROC curve23, the limit was set according to the 80% specificity rule, which implies that 80% of patients who state that they are not in a satisfactory condition or have not experienced an important improvement have a score on the measurement in question that corresponds to the cutoff value or a worse health state/less improvement11.
RESULTS
The analyses were based on 4225 responses from patients with RA, PsA, or AS who had answered to both PASS and MCII questions at the 3-month followup examination. SF-6D, MHAQ, patient global VAS, and VAS Pain were available for over 95% of the included patients, while EQ-5D was available for 28% due to later inclusion in the data collection procedure (March 2006; Table 1). The main treatment types varied somewhat with diagnosis group, and a higher share of AS patients received anti-tumor necrosis factor-α (anti-TNF-α) monotherapy compared to RA and PsA patients (Table 1). Additional characteristics of the RA, AS, and PsA patients available for analyses of PASS and MCII cutpoints for SF-6D and EQ-5D after 3 months are presented in Table 2.
Number of observations for various health measures at 3 months after initiated DMARD treatment.
Demographic and health status variables for patients included in the PASS/MCII cutpoint analyses. Mean (SD) for continuous variables, percentages for counts.
PASS cutpoints
In the multi-attribute health state classification systems the cutpoints for PASS were somewhat lower or the same for RA compared to AS and PsA (Table 3). PASS cutpoints for health status measures were similar across diseases, with the exception of a slightly higher cutpoint for MHAQ in patients with RA.
PASS cutpoints with 2 methodological approaches for health-related quality of life and health status measures after 3 months of DMARD treatment in patients with RA, PsA, and AS.
Higher cutpoints were seen in EQ-5D and SF-6D with the 80% specificity approach in the ROC analyses than with the 75th percentile method (lower for health status measures). The results estimated with the 80% specificity method mean that 80% of the patients who were not in PASS have an index value of ~0.70 or lower in EQ-5D and ~0.65 or lower in SF-6D. The cutpoints were around 0.65 for all diseases in EQ-5D with the 75th percentile approach and the corresponding cutpoints were around 0.60 in SF-6D. This means that among patients who stated that they had a satisfactory health condition, 75% had an EQ-5D score of around 0.65 or higher and 75% had an SF-6D score of around 0.60 or higher.
MCII cutpoints
The results from the analyses of the dichotomous question and the 5-scale question were similar and we therefore present results only from the dichotomous question (Table 4). The cutpoints for the multi-attribute health status classification systems varied from 0 to 0.05 for all analyzed diagnoses with the 75th percentile and from 0.07 to 0.19 with the 80% specificity method. The cutpoints for health status measures indicated small improvements in patient global VAS and joint pain VAS in all diagnoses, while the MCII cutpoints were around 0 for MHAQ with the 75th percentile method. The MCII absolute cutpoints were larger with the ROC analysis approach for EQ-5D, SF-6D, and health status measures (Table 4). The area under the curve (AUC) assessed by the ROC curve analysis for the MCII yielded estimates from 0.68 to 0.83 with the different instruments, indicating an overall inferior agreement compared with the PASS. The average improvement after 3 months of treatment was larger for AS than for RA and PsA patients in all measurements (Table 2). However, the MCII cutpoints are rather similar across diagnosis groups in SF-6D, MHAQ, pain VAS, and patient global VAS.
MCII cutpoints with 2 methodological approaches for changes in health-related quality of life and health status measures after 3 months of DMARD treatment in patients with RA, PsA, and AS.
The results of the testing for validity for MCII indicated that changes in patient global VAS after 3 months of treatment correlated best with MCII. Moderate correlations were observed for the changes in utility values with the MCII (from 0.29 to 0.39 in EQ-5D and from 0.38 to 0.43 in SF-6D, depending on diagnosis). Among the dimensions in EQ-5D, the dimension “pain/discomfort” had highest correlation coefficients with MCII in all diagnoses. In logistic regression analyses with the MCII dichotomous answer as the dependent variable, both the pre-and post-scores of all the instruments were significantly associated (p < 0.05), even though the post-scores explained more of the variance of MCII in all diagnoses. These results indicate that it is not only the post-score that explains the MCII but also the change from pre-to post-scores, and these findings contribute to the validity of the MCII.
DISCUSSION
This report highlights that cutpoints for PASS are rather similar across diseases in multi-attribute health state classification systems, around 0.65–0.70 for EQ-5D and around 0.60–0.65 for SF-6D estimated with 2 methods: the 75th percentile approach and ROC analyses with 80% specificity. The absolute value cutpoints for MCII were larger with the ROC 80% specificity method and around 0 with the 75th percentile method. Aletaha, et al also showed that the ROC approach performed better than the 75th percentile method when they identified MCII cutpoints for disease activity measures in the same patient cohort11.
The AUC assessed by the ROC curve analysis for PASS yielded estimates from 0.74 to 0.85, where 1.00 represents a perfect relationship and 0 no relationship at all. Since random guessing on average yields 0.5, no instrument intended to classify a relationship should have a value below 0.523. Patient global VAS gave the largest AUC for PASS among the estimated measures.
We selected the 80% specificity method as our primary approach in the ROC analyses, based on results from previous studies11. However, we performed sensitivity analyses by using the maximum accuracy method. The cutpoint with this approach corresponds to the largest numeric sum of sensitivity and specificity. However, sensitivity and specificity can be traded against each other and the maximum accuracy can in some cases be achieved with more than 1 combination of sensitivity and specificity, and the choice of combination is then ambiguous. The maximum accuracy approach gives poor comparability across ROC analyses.
This study revealed that the cutpoints for PASS and MCII can vary not so much with disease as with the identification method. The 80% specificity method in the ROC analyses gave overall higher cutpoint values in the multi-attribute health status classification systems and lower values in the health status measurements than the 75th percentile approach. If the concepts of PASS and MCII are to be used to evaluate responses to treatment, the method used for identification of cutpoints should be standardized. The 75th percentile approach has been used for PASS estimates in patients with knee and hip osteoarthritis (OA)24, in patients with knee OA and rotator cuff syndrome22, in patients with AS9,10, and in patients with RA8. ROC graphs have been used for analyses of the cutpoint for PASS in patients with AS9 and in patients with RA with the maximum accuracy approach8. Identified PASS and MCII cutpoints can be used to determine the proportions of patients who achieve a level within an endpoint for acceptable state or important change, respectively.
PASS and MCII cutpoints have previously not been assessed for multi-attribute health status classification systems. A cutpoint for MCII of 0.10 of a utility in EQ-5D corresponds to the lowest level of improvement that is perceived as important for patients. This finding may also help to put small changes in utilities into the perspective of the patient and help to calculate the proportions of patients with important improvement in HRQOL. Importantly, cutpoints were rather similar across the 3 diseases.
The identified cutpoints corresponding to PASS illustrate that an acceptable state for the patients corresponds to a state of quality of life that is far from perfect health. It is well known that patients generally perceive their quality of life to be higher than that of the general population when asked to evaluate their quality of life25,26, because patients adapt to their condition. It has therefore been debated whether patients or the general public should evaluate their health states25. When comparing PASS to EQ-5D or SF-6D it should be noted that the PASS concept is assessed directly by patients while the EQ-5D and the SF-6D have been developed using the general population for assessing values to their health states14,16. In order to illustrate the concept of PASS, we can choose the EQ-5D cutpoint value assessed by the 80% specificity method, which is 0.69–0.73, depending on disease. An EQ-5D value of 0.691 corresponds to a patient who has some problems in walking about, no problems with self-care, some problems with performing his/her usual activities, and moderate pain or discomfort, and who is not anxious or depressed. Patients in health states inferior to this level of health problems are not regarded as being in PASS, which seems reasonable. On the other hand, it is important to be aware that patients can have problems with mobility, daily activities, and pain, and still state that they are in a satisfactory health state. The PASS concept could give physicians indications of patients accepting lower treatment goals than what might be achieved with the treatment options of today, because patients might have low treatment expectations or adapt to their conditions. A considerable proportion of the analyzed patients stated that they were in an acceptable symptom state at baseline (about 30%–37%, depending on diagnosis). These patients had had a referral from their doctors for starting or changing DMARD treatment. It would be useful to further explore prediction variables for PASS and MCII such as age, gender, education level, etc.
For practical relevance it is important to consider the sensitivity of SF-6D and EQ-5D as measures for HRQOL in patients with inflammatory rheumatic diseases. The SF-36 (from which the SF-6D is derived) has been shown to be responsive to improvements in patients with RA27. The EQ-5D has been examined in 233 patients with RA regarding its construct validity, responsiveness to change, and reliability. The patients reported improvement, deterioration, or no change over a 3-month period and the EQ-5D index was found to be responsive to the self-reported change and to be as reliable as VAS pain, swollen joint score, and disease activity assessed by patients or physicians14. Both EQ-5D and SF-6D have been assessed regarding responsiveness in RA patients treated with infliximab and both were found to be responsive during treatment17. In another study of reliability and responsiveness of the 2 utility instruments, the SF-6D but not the EQ-5D was found to be sufficiently reliable28.
EULAR/ACR has published recommendations on how to report disease activity in clinical trials with RA. The first point in the recommendations states that “each trial should report disease activity states and response” where “state” is defined as a level of disease activity and “response” as a change score. Although these recommendations primarily concern disease activity, it is specified that they also should be used for other important domains such as function and damage. Further, the recommendations state that the sustainability of the primary outcome should be reported, but identification of standards for this issue is also part of the research agenda7. PASS is a patient-reported measure of state and MCII is a patient-reported measure of change. PASS cutpoints for composite disease activity have been found to be in the area of moderate disease activity levels and a moderate disease activity level is not sufficiently low to prevent radiographic progression8. Similarly, a cutpoint of 0.65–0.70 in a utility instrument indicates that the HRQOL is far from perfect. Thus, 0.70 may be a goal for a state that is acceptable and achievable by a rather large proportion of patients, but does not reflect current ambitions that focus on remission as a treatment goal29.
Because this study was based on routine care data from 5 secondary level clinics in Norway, the results are likely more representative for patients taking DMARD than are data from clinical trials.
The results from our study reveal that the cutpoint value for assessing a patient in PASS or having experienced an MCII varies with methodology for assessing the cutpoint. Cutpoint values for PASS and MCII are rather similar across RA, PsA, and AS. However, it is not clear what makes a patient state that his/her symptom state is acceptable or that he/she has experienced an important improvement, and further research should be directed at these questions.
Footnotes
-
Supported by a research grant from the South-Eastern Norway Regional Health Authority.
- Accepted for publication July 9, 2009.
