Abstract
Objective We estimated meaningful change thresholds (MCTs) for Patient-Reported Outcomes Measurement Information System (PROMIS) Fatigue and Pain Interference in rheumatoid arthritis (RA).
Methods The responsiveness of several patient-reported outcomes (PROs) was assessed among 521 patients with RA in the Arthritis, Rheumatism, and Aging Medical Information Systems (ARAMIS) cohort. PROMIS Fatigue (7-item) and Pain Interference (6-item) short form instruments were administered at baseline, 6 months, and 12 months. Self-reported retrospective changes over the previous 6 months (a lot better/worse, a little better/worse, stayed the same) were obtained at 6 and 12 months’ follow-up. We estimated MCTs using the mean change in PROMIS scores for patients who rated their change “a little better” or “a little worse.”
Results Baseline fatigue and pain interference scores were near normal (median 54 and 56, respectively). At 6 months, 7.9% of patients reported their fatigue was a little better compared to baseline (mean change [SD]: –2.6 [4.8]) and 22.8% a little worse (1.7 [5.6]). Pain was a little better for 11.5% of patients (–1.9 [6.1]) and a little worse for 24.2% of patients (0.6 [5.7]). At 12 months, results were similar. Thus, the MCT range was 1–2 points for both fatigue and pain interference. Correlations between change scores and retrospective ratings were low (0.13–0.29), indicating possible underestimation of MCT.
Conclusion The group-level MCT for PROMIS Fatigue and Pain Interference is roughly 2–3 points and corresponds to a small effect size, which is consistent with earlier work demonstrating an MCT of 2 points for PROMIS Physical Functioning.
Fatigue and pain are routinely used patient-reported outcomes (PROs), and they are highly prevalent symptoms among patients with rheumatoid arthritis (RA). As meaningful indicators of health and sources of economic burden,1,2,3,4 these PROs are important to inform both patient and provider decisions and to improve health outcomes. To advance fatigue and pain measurement across healthy and chronically ill populations, the Patient-Reported Outcomes Measurement Information System (PROMIS) project created item banks for fatigue and pain interference, along with other domains (www.nihpromis.org).5 The fatigue item bank consists of 95 items assessing the intensity, frequency, and effect of fatigue. The pain interference bank consists of 41 items assessing the extent to which pain interferes with functioning. The item banks can be administered in multiple formats including dynamic computer adaptive tests (CATs) or short fixed-length forms,6 which are parsimonious subsets of items derived from each item bank. PROMIS item banks, CATs, and short forms have demonstrated reliability and validity comparable to legacy instruments.5,7 Scores obtained from short forms and CATs are calibrated on the t-score metric such that the mean in the general United States population is 50 and the SD is 10.8 For both fatigue and pain interference, higher PROMIS scores reflect greater fatigue and pain interference.
Since PRO improvements are usually more relevant to patients than clinical or serological changes alone, they have been considered important aspects of RA management and included in most recommendations for the management of RA.9,10,11 Therefore, interpretation of PROs in research or clinical practice requires definition of an important or meaningful change, or meaningful change threshold (MCT). A previous study has shown that the MCT for PROMIS Physical Function in patients with RA is 2 points.12,13 However, the MCTs for PROMIS Fatigue and Pain Interference still remain unknown. The current study estimated MCTs for the PROMIS Fatigue and Pain Interference scale scores among patients with RA using the Arthritis, Rheumatism and Aging Medical Information System (ARAMIS) data.
METHODS
Data sources. ARAMIS was the first national chronic disease data bank system, funded by the National Center for Health Services Research.8 ARAMIS contains 14,000 RA and osteoarthritis patients, and individuals from normal and healthy aging populations. The characteristics of ARAMIS patients with RA are similar to other RA patient groups reported in the literature relative to age, sex, race, and disease duration.14 Once enrolled, study participants complete questionnaires every 6 months, which address BMI, disability, general health status, fatigue, pain, physical function, and specific disease questions.14,15
Eligibility criteria and study design. Using a retrospective cohort study design, we required all patients with RA in the 2000–2002 ARAMIS cohort to be > 20 years old. After enrollment, patients were asked to self-administer survey questions at baseline and at 6 and 12 months. Surveys were distributed to participants by mail with 3 rounds of follow-up that included postcard and telephone reminders and multiple mailings of the survey.12 The attrition rate over the 1-year follow-up was 13%.
Fatigue. PROMIS defines fatigue as an “overwhelming, debilitating, and sustained sense of exhaustion that decreases one’s ability to carry out daily activities, including the ability to work effectively and to function at one’s usual level in family or social roles.”5 Thus, fatigue is divided conceptually into the experience of fatigue and the effect of fatigue upon physical, mental, and social activities.5 The PROMIS Fatigue 7a-item short form (version 1.0) was administered in this study (Supplementary Data, available from the authors on request).
Pain interference. Pain has been divided conceptually into components of interference with activities, intensity, quality, and behaviors one engages in to avoid, minimize, or reduce pain.5 PROMIS defined pain interference as a measure of the extent to which pain hinders engagement with physical, cognitive, emotional, and recreational activities, as well as sleep and enjoyment in life.16 The PROMIS Pain Interference 6b-item short form (version 1.1) was administered in this study (Supplementary Data, available from the authors on request).
Measurement. Participants reported retrospective changes in fatigue and pain over the previous 6 months at 6 months and 12 months after baseline using the following response categories: a lot better, a little better, stayed the same, a little worse, a lot worse. Responses to these retrospective change items were used to categorize patients into different groups.
Statistical analysis. Anchor-based approaches are widely used to estimate MCTs, and the values of MCTs identified using anchor-based approaches are comparable to other methods.17,18,19 We calculated the mean change in PROMIS scores between baseline and follow-up assessments using the retrospective ratings of change as the anchor variables. We estimated MCTs using the mean change in PROMIS scores for people who retrospectively rated their change as “a little better” or “a little worse.” Effect sizes (ES; mean change/SD of change in that group) were calculated for all groups to provide a standardized measure of the magnitude of change. In addition, we computed 2 times the standard error (SE) of the mean for people who said that they “stayed the same.” We also conducted subgroup analyses to calculate the mean change in fatigue and pain interference stratified by baseline scores at the median. We evaluated the correlation between change scores and retrospective ratings using Spearman rank correlation coefficient. All analyses were conducted using SAS (version 9.3; SAS Institute). The observational study was approved by the Stanford University Institutional Review Board (IRB-17334). Since the data we received were deidentified, individual patient consent was not required for the analysis.
RESULTS
Baseline patient (n = 521) characteristics are listed in Table 1. The study population was 81% female, 87% non-Hispanic White, and 67% were aged 60 years or older. Among them, 69% of patients reported their general health as good, very good, or excellent. Median baseline PROMIS scores were 54 (IQR 48–59) for fatigue and 56 (IQR 51–61) for pain interference. At 6 months, the Spearman correlation coefficient between PROMIS change score and retrospective change rating was 0.24 for pain and 0.29 for fatigue. At 12 months, the correlations were 0.29 (pain) and 0.13 (fatigue).
Patient characteristics of the cohort (n = 521).
At the 6-month visit, 41 patients reported that their fatigue was a little better, with mean change in PROMIS Fatigue scores of –2.6 (SD 4.8, ES –0.55). Patients whose fatigue was a lot better had a similar degree of change in PROMIS Fatigue scores (Table 2). The 119 patients whose fatigue was a little worse had a mean PROMIS Fatigue change of +1.7 (SD 5.6, ES 0.31). There was a larger ES (0.54) in the group of patients whose fatigue was a lot worse, corresponding to a mean PROMIS Fatigue change of +3.8 (SD 7.0). Sixty patients reported that their pain interference was a little better at 6 months, with mean PROMIS pain interference change of –1.9 (SD 6.1, ES –0.31). Patients whose pain interference was a lot better had a similar degree of change in PROMIS scores. The 126 patients whose pain was a little worse had a mean PROMIS pain interference change of +0.6 (SD 5.7, ES 0.11). The ES was much larger for patients whose pain interference was a lot worse, with corresponding mean PROMIS change of +5.5 (SD 6.6, ES 0.84). Results were similar at the 12-month assessment (Table 2). Among patients who said that they stayed the same, twice the SE of the mean was 0.72 (fatigue) and 0.85 (pain interference).
Mean PROMIS change scores in “a little worse/better” subgroups for fatigue and pain interference.
Among patients whose baseline fatigue score was greater (worse) than the median value (54), the mean change for people whose fatigue got a little better was 3.6–4.2 points (ES 0.53–0.99), whereas for those whose fatigue got a little worse, the mean change was ≤ 1 (ES < 0.20). Among patients whose baseline fatigue score was lower (better) than the median value (54), the reverse pattern was observed with the “a little better” group changed by ≤ 1.1 (ES < 0.20), whereas the “a little worse” group worsened by 4.6–5.4 points (ES 0.99; data not shown). Similarly, among patients whose baseline pain interference score was greater (worse) than the median value (56), the mean change for people whose pain got a little better was 4.4 points (ES 0.78–0.82). Among patients whose baseline pain interference score was lower (better) than the median value (56), the “a little worse” group worsened by 3.1–4.6 points (ES 0.56–0.90; data not shown).
DISCUSSION
This study estimated MCTs for the PROMIS Fatigue and Pain Interference scales among patients with RA in the ARAMIS cohort. We found that the MCTs estimated from this cohort are roughly 2–3 points and correspond to a small ES. When stratified by the baseline score, the subgroups with “room to move” reported larger changes of 3–5 points. As a lower limit for measurable change, twice the SE of the mean in the “stayed the same” group was 0.72–0.85 points.
To optimize treatment and to better understand the respective roles of PROMIS measurements in clinical research, one tool for enhancing the interpretability of PROs is the MCT. Substantial evidence demonstrates that results from different PROMIS short forms and CATs derived from the same item bank are consistent. These are effective, reliable, and precise measures of generic symptoms and functional reports across a range of chronic conditions, such as chronic heart failure, chronic obstructive pulmonary disease, RA, cancer, back pain, or major depression.12 Current commonly used strategies for identifying important differences and MCTs include anchor-based and distribution-based methods.17 The anchor-based methods examine the relationship between a measure and anchor to interpret the changes that have clinical relevance,17,18 whereas the distribution-based methods rely on statistical characteristics of distributions.18 Although both approaches have advantages and limitations, several studies have reported that the values identified by these 2 different methods are comparable,17,18,19 with one reporting high levels of agreement across methods19 with κ values of 0.71–1.0.
Our results are consistent with earlier work in this cohort demonstrating an MCT of 2 points for PROMIS Physical Functioning12 and another study using data from the Stepped Care to Optimize Pain care Effectiveness (SCOPE) study that reported that the MCT was 2 points for the depression scale and 2.5 points for the anxiety scale in primary care patients with back pain.20 Another study including pediatric population also demonstrated a 2- to 3-point MCT for depression, pain interference, fatigue, and mobility.21 Moreover, using adults with RA enrolled in a multisite observation cohort, a recent study also found a 1–3 points change in PROMIS scores for several PRO measures when patients felt a little better or a little worse.22 In addition, the mean and IQRs of the PROMIS t-score for fatigue and pain interference scores in our study cohort do not indicate an issue with floor or ceiling effects.
There are several limitations of this study. The ARAMIS cohort did not have information on patients’ medication use and baseline disease severity; therefore, we were not able to examine the effects of these factors in current analysis. Moreover, only approximately 10% of our sample were “a little better” and 28% “a little worse,” and examination of patients initiating a new RA therapy, for example, may have higher yield to estimate MCTs. Based upon this relatively small sample size, we also reported symmetric MCTs (i.e., the same MCTs for improvement and worsening), but in fact, the magnitude of change for improvement may be somewhat different than the MCT for worsening. Additionally, the dataset used for this secondary analysis is over a decade old and may not be representative of the experiences of patients with RA today. Finally, Spearman correlation coefficients between PROMIS change scores and retrospective changes were low (0.13–0.29), indicating potentially underestimated MCTs. A possible driving factor for the low correlation could be the low degree of change observed in the sample, with approximately half of the sample reporting no change. It is a challenge to detect correlation in a dataset with little variability.
In conclusion, the low end of MCTs for PROMIS Fatigue and PROMIS Pain Interference, estimated from this cohort of patients with RA, is roughly 2–3 points and corresponds to a small ES. This is consistent with earlier work in this RA cohort demonstrating an MCT of 2–3 points for PROMIS Physical Functioning. More work is needed to evaluate the performance of PROMIS measurements in different study populations.
ACKNOWLEDGMENT
We thank Dr. James F. Fries (University of Stanford) for providing the dataset and support.
Footnotes
The authors declare no conflicts of interest relevant to this article.
- Accepted for publication February 24, 2021.
- © 2021 The Journal of Rheumatology
