Abstract
Objective. To evaluate the structural validity of the Rheumatology Attitudes Index (RAI), a widely used measure of rheumatic disease–related helplessness in patients with systemic sclerosis (SSc).
Methods. Patients with physician-confirmed SSc from the University of California, Los Angeles (UCLA) Scleroderma Quality of Life Study (n = 208) received clinical examinations and completed self-report questionnaires. The structural validity of the RAI was examined through confirmatory and exploratory factor analysis (CFA/EFA).
Results. A tenable factor structure was not identified through CFA or EFA.
Conclusion. The present structural analysis did not support the use of the RAI with SSc patients.
In the context of chronic illness, patients may appraise their health as unpredictable and uncontrollable, and believe that their efforts to control their illness will be ineffective1. In the rheumatic diseases, this cognitive style, called “helplessness,” has been associated with greater psychological disability, poorer response to medical treatments, and mortality2. Helplessness can also mediate the relationship between depression and functional impairment3. For patients with systemic sclerosis (SSc), a rare rheumatic disease that is characterized by an often unpredictable disease course and sudden symptomatic changes4, helplessness may also be a relevant construct. Previous research examining helplessness in SSc has identified associations between helplessness and depressive symptoms5,6.
The 15-item Arthritis Helplessness Index (AHI) was developed to measure helplessness in rheumatoid arthritis (RA)1. The 4-point response options ranged from 1 (strongly disagree) to 4 (strongly agree). Although the AHI was hypothesized to be unidimensional, in the development study (n = 219 patients with RA) the low internal consistency of the total score (α = 0.69) suggested a multifactorial solution1. In a followup study of patients with RA (n = 368), explanatory factor analysis (EFA) of the AHI found 5 items loaded onto a factor reflecting beliefs that patients cannot control disease outcomes, labeled the Helplessness subscale (α = 0.63), and 6 items loaded onto a factor reflecting beliefs that patients can control disease outcomes, labeled the Internality subscale (α = 0.75)7. The other items were still retained in the measure but they were not included in the scoring. The 2 subscales were significantly correlated at a small magnitude (r = 0.21).
Because the AHI was designed for use in RA, it needed to be adapted to be applied to other rheumatic diseases. The modified version, the Rheumatology Attitudes Index (RAI)8, is identical to the AHI in content, except for the substitution of “condition” for “arthritis” and a modification of the response scale to include a neutral response option 2.5 (do not agree or disagree). Like the AHI, the internal consistency of the RAI total score was marginal (α = 0.68); moreover, only the total score was analyzed8. Additionally, there have been other versions of the RAI response scale, including a 6-point response scale9.
A problem with both the AHI and RAI is their inconsistent application, with some studies using the 6-item Internality subscale only, some studies substituting a 7-item Internality subscale, other studies using the Helplessness subscale as a standalone measure10, and still others using the total score of all 15 items11. Although previous studies have not provided clear rationales for using different variations of the measure, a possible explanation is that the initial RAI development paper8 was published prior to the study, demonstrating that the 2-subscale version of the AHI is psychometrically preferable7. This timeline may contribute to the inconsistent use of the measure. Another reason may be that, in the same study7, 2 different factor solutions were described: a 7-item Internality subscale was found to fit the data in 1 sample of patients with RA (n = 368), whereas a 6-item Internality subscale was found using a cross-validation sample with the patients with RA from the original AHI development study (n = 219). Additionally, in a paper describing the measure in a non-RA rheumatic sample, Engle and colleagues12 stated that the RAI “was initially called the Arthritis Helplessness Index, and was later renamed the Rheumatology Attitudes Index,” implying that the latter subsumed the former. Although both measures have been widely used, many authors using the RAI often refer to the measure as the AHI (e.g., McNearney, et al13). Further details and a summary of the different scoring methods of the measure are available11.
The RAI has been adopted in several studies in SSc, even though its structural validity has not been established in this population6,13,14,15,16,17. In one of these studies, the 5-item Helplessness subscale was used as a standalone measure6, whereas in the other studies the total score has been used13,14,15,16,17. Several of these studies13,14,15,16 reference the same paper12, which described the RAI as a 1-factor measure scored by calculating the sum of all 15 items. A formal psychometric evaluation is needed prior to confidently using the RAI in SSc.
MATERIALS AND METHODS
Patients and procedure
The sample consisted of 208 patients from the University of California, Los Angeles (UCLA) Scleroderma Quality of Life Study, an observational, single-center, cohort study. Participants were at least 18 years old and had a formal diagnosis of SSc confirmed by a study physician. The study was approved by the UCLA Institutional Review Board, study number 7-07-061-01.
Measures
The RAI is a measure of perceived control and helplessness over rheumatic disease-related outcomes. Participants rate a series of 15 statements with responses ranging from 1 (strongly disagree) to 6 (strongly agree). A total score is derived by reverse-scoring 9 of the 15 items, and summing the items. Higher scores reflect greater helplessness. Subscale scores for Helplessness (5 items) and Internality (6 or 7 items) can also be calculated.
Sociodemographic and medical characteristics
Patients self-reported sociodemographic details. Disease severity was physician-evaluated using the modified Rodnan skin score18.
Data analysis
Descriptive statistics were analyzed using SPSS Version 23.0. Confirmatory factor analysis (CFA) was conducted in MPlus version 7.12 to examine the structural validity of AHI/RAI scores. A 1-factor structure using all 15 items, both of the 2-factor structures (i.e., the 5-item Helplessness subscale with either the 6-item or 7-item versions of the Internality subscales), and a 1-factor Helplessness structure were examined.
Overall model fit was determined using the recommendations of Bentler19. Three indicators of model fit were used: (1) the root mean square error of approximation (RMSEA), (2) the standardized root mean residual (SRMR), and (c) the comparative fit index (CFI). For the RMSEA and SRMR, values < 0.08 indicated acceptable fit; for the CFI, values > 0.90 indicated acceptable model fit. A model was determined to fit well if at least 2 of these indicators met criteria for acceptable fit. Maximum likelihood estimation with robust standard errors (i.e., MLR estimation) was used in the present analysis. Modification indices were also requested.
In anticipation of the possibility that the CFA would not identify a solution, an EFA was planned. Items with a primary loading ≥ 0.45 and secondary loading ≤ 0.25 would be retained to maximize practical significance and diminish multivocality20,21. Parallel analysis was also planned to confirm the number of factors that should be retained in the EFA. The aforementioned descriptive fit indices and theoretical interpretations were considered to make final determinations of factor retention by iteratively removing items.
RESULTS
Descriptive statistics
Sample characteristics are summarized in Table 1.
Demographic and medical variables (n = 208). Data are n (%) unless otherwise indicated.
Structural validity
In conducting confirmatory factor analysis, the 1-factor total score solution did not fit well statistically (χ2 [53] = 138.80, p < 0.001) or descriptively (CFI = 0.714, RMSEA = 0.086, SRMR = 0.077). For the 2-factor solution, the Helplessness and Internality latent variables were indicated by 5 and 7 observed items, respectively, as in the Stein, et al7 study. This 2-factor model did not fit well statistically (χ2 [53] = 138.80, p < 0.001), or descriptively (CFI = 0.706, RMSEA = 0.086, SRMR = 0.089). The 5-item Helplessness and 6-item Internality solution offered by Stein, et al7 as an alternative was also tested, but did not fit well statistically (χ2 [43] = 117.63, p < 0.001) or descriptively (CFI = 0.821, RMSEA = 0.091, SRMR = 0.068). The 1-factor Helplessness scale (5 items)10 was also tested and did not fit well statistically (χ2 [5] = 12.367, p < 0.05), but did demonstrate tenable fit descriptively (CFI = 0.944, RMSEA = 0.084, SRMR = 0.041). However, internal consistency reliability was not adequate (α = 0.634). No modifications were undertaken given a lack of theoretical justification for freeing measures.
Exploratory factor analysis
Using the 0.45-factor–loading cutoff criterion, the EFA also failed to identify a tenable solution. The model was also evaluated using a less-conservative factor loading cutoff of 0.40. The parallel analysis indicated that a 2-factor–solution best represented the data when eigenvalues from the present dataset were compared to eigenvalues from randomly simulated data: (1) Factor 1: 2.85 versus 1.33, (2) Factor 2: 1.46 versus 1.22. Using the ≤ 0.40 cutoff, a 2-factor solution fit the data per descriptive indices, such that the 3 descriptive fit indices indicated good fit (χ2 [19] = 22.32, p = 0.269; CFI = 0.985, RMSEA = 0.029, SRMR = 0.032). Seven items either did not load at ≥ 0.40 onto either factor or loaded at > 0.25 onto both factors and were thus removed, with the exception of 1 item, described below. Using this approach, 9 items were retained (Table 2), labeled as Helplessness (5 items) and factor 2 as Internality (4 items). The correlation between the 2 factors (r = 0.22) was similar to that in a previous study7. However, this 2-factor solution was also deemed untenable because it included an item (item 10) that loaded on both factors, but removing it caused the descriptive fit indices to fall below acceptable levels. Further, internal consistency reliability was not adequate for either subscale (Helplessness: α = 0.634; Internality: α = 0.645).
Factor loadings of the Helplessness and Internality factors from the exploratory factor analysis 9-item solution.
DISCUSSION
In the present study of patients with SSc, we did not find support for any of the previously identified factor structures for the RAI through CFA and did not identify an alternative tenable factor structure through EFA. This suggests that the RAI is not an appropriate measure of helplessness in SSc. Brady11 examined measures of helplessness, self-efficacy, mastery, and control that have been used in rheumatology research. A number of measures of constructs conceptually related to helplessness (e.g., self-efficacy) were identified that may be appropriate for use in SSc, pending psychometric validation.
Limitations of the present study should be considered. The literature contains different response options for the RAI’s response scale; the present study used the 6-item response scale. Additionally, the sample was limited to patients in Southern California receiving care at a university-hospital setting.
The present structural analysis did not support the use of the RAI with SSc patients. In the absence of a valid measure of helplessness for this population, measures of related constructs (e.g., self-efficacy) may be considered.
Footnotes
Supported by the grant, Evaluation of Health-Related Quality of Life in Systemic Sclerosis from the Scleroderma Foundation Inc. Dr. Khanna has been funded by the US National Institutes of Health (NIH)/US National Institute of Arthritis and Musculoskeletal and Skin Diseases) grants K24 AR063120 and K23 AR053858. Dr. Khanna has served as a consultant for Bayer and Genentech.
- Accepted for publication February 7, 2017.
