Abstract
Objective. The Brief Cognitive Symptoms Inventory (BCSI) is a short, self-report scale designed to measure cognitive symptomatology in patients with rheumatic disease. To facilitate research and clinical practice, we tested the internal consistency and validity of the BCSI in patients with Sjögren syndrome (SS).
Methods. Patients who met the American-European Consensus Group criteria for SS and healthy controls completed a questionnaire assessing symptoms including cognitive complaints. We calculated Cronbach’s alpha to assess internal consistency and Pearson correlation coefficients to test for association between BCSI, symptoms, and demographic variables. Total score distribution was analyzed to establish cutoff criteria for differentiation of case versus non-case. We compared neuropsychological outcomes of patients with SS above and below the threshold BCSI score to assess the association of cognitive symptoms with objective cognitive deficits.
Results. Complete data were available on 144 patients with SS and 35 controls. Internal consistency of the BCSI was good. Scores were similar in all patient groups and patients reported more cognitive symptoms than controls (p < 0.0001). BCSI scores correlated moderately with pain, depression, anxiety, fatigue, and health quality. High scores for cognitive dysfunction were reported by 20% of the patients with SS and only 3% of controls. Patients with cognitive scores > 50 had more depression, fatigue, pain (effect size all > 1), and worse performance on multiple cognitive domains.
Conclusion. The BCSI should be a useful tool for the study of cognitive symptoms in SS. Both self-report and standardized tests should be considered in screening for cognitive disorders in SS.
Sjögren syndrome (SS) is a relatively common systemic autoimmune disorder characterized by exocrine gland dysfunction, autoantibodies, and extraglandular inflammation. The prevalence is estimated at 0.1–0.6%1. Cardinal symptoms are oral and ocular dryness, but abnormal fatigue and moderate to severe daily pain are reported by the majority of patients and contribute to reduced health quality2,3. Depressive symptoms and poor sleep are also features of SS4,5,6. While problems with memory and concentration are reported more frequently by patients than by age-matched healthy controls, the prevalence of cognitive disorder in SS is uncertain7. Yoshikawa, et al reported that among patients attending a geriatric memory clinic in Japan, a surprising 7.5% were found to meet the American-European Consensus Group (AECG) criteria for SS8.
Despite the large potential effect of SS-associated cognitive dysfunction, especially for the health of elderly persons, there are no guidelines for screening for cognitive disorder nor is there consensus regarding criteria for the diagnosis of cognitive impairment in patients with primary SS (pSS). Prospective data regarding cognitive disorders in SS is limited9,10,11,12,13,14. A validated screening questionnaire could be a useful adjunct to formal neuropsychometric testing; however, the relationship between subjective memory complaints and cognitive impairment is inconsistent. In some studies, deficits in memory and attention are correlated with cognitive complaints15,16. In other studies of patients with immune disorders, including systemic lupus erythematosus (SLE), rheumatoid arthritis (RA), and multiple sclerosis, cognitive symptoms have been poorly predictive of objective performance17,18,19. Interestingly, despite the inconsistent relationship between cognitive complaints and objectively measured cognitive abilities, future cognitive decline has been linked to subjective cognitive complaints20. Both longitudinal data and functional brain imaging have shown a relationship between increased levels of cognitive complaints and age-related changes in memory and brain activity21.
The Cognitive Symptoms Inventory is a self-report measure of cognitive symptoms specifically developed for evaluation of patients with rheumatic disease22. The Brief Cognitive Symptoms Inventory (BCSI) consists of 6 items selected from the 21-item Cognitive Symptoms Inventory. The BCSI appears to have adequate psychometric properties in patients with SLE23. To facilitate research in cognitive function in SS, we tested the internal consistency reliability and validity of the BCSI in SS.
Harboe, et al previously reported a weak association between anti-SSA and cognitive dysfunction11. Because patients with seropositive pSS are more likely to have extraglandular involvement, we were interested in investigating whether serological status had an effect on perceived cognitive function. Consistency of the results across patient groups will be shown as indirect evidence of validity. Correlations with other valid self-report measures and ability of the BCSI to discriminate between patients and healthy controls will be assessed to provide evidence of criterion-based validity. To explore the relationship between BCSI scores and objective tests of cognitive function, a small subset of patients with SS underwent formal psychometric evaluation.
MATERIALS AND METHODS
Study design
The Biomarkers in Primary Sjögren’s syndrome (BioSiPS) registry is a large repository of data collected from patients with sicca and non-disease community controls evaluated for pSS according to AECG criteria24. All participants in BioSiPS undergo a standardized evaluation, including interview, physical examination, tests of gland function, phlebotomy for routine clinical laboratory and serologic tests, and minor salivary gland biopsy25. We collected questionnaire data in 2 groups of patients drawn from the BioSiPS registry 3–5 years after their enrollment in the registry. Objective evaluation of gland function and evaluation of serologic status was performed only at the time of enrollment in the registry. All additional data, including demographic and psychological symptoms, were collected at the time of the patient survey. In the first survey, seronegative patients were oversampled to ensure adequate representation. Data for replication were collected from a second sample consisting of healthy community controls and the first 100 patients with SS enrolled in BioSiPS at the University of Minnesota who had not previously received the survey.
Items of the BCSI
The 6 items of the BCSI are shown in Table 1. Respondents were instructed to focus on how their illness affected their ability to think clearly. They were asked to indicate whether each activity may have been a problem on a scale of 0 (never a problem) to 3 (a problem all of the time — unable to do) over the past 4 weeks. To score the BCSI values, the 6 items were summed. Raw scores were transformed into a new variable by dividing the sum by 18 and multiplying by 100. The transformed score range is 0–100.
Brief cognitive symptoms inventory. Instructions: Now we would like to ask you questions specifically focusing on how your illness affected your ability to think clearly. By circling one number per line, please indicate whether each activity may have been a problem over the past 4 weeks.
A score of 0 is possible if all 6 items on the scale were rated as never. Higher scores indicate worse function (more cognitive symptoms). The BCSI provides a state measure of cognitive symptoms insofar as it indicates present levels of functioning and is limited to the 4-week interval preceding its administration.
Questionnaires and cognitive assessment
In addition to the BCSI, the questionnaire included measures of fatigue [Fatigue Severity Scale (FSS)]26, pain [Brief Pain Inventory (BPI)]27, sleep [Pittsburgh Sleep Quality Index (PSQI)]28, mood [Hospital Anxiety and Depression-Scale29, and Centers for Epidemiologic Studies Depression Scale (CES-D)]30, health quality (12-item Medical Outcomes Study Short Form-12 health survey)31, and perceived cognitive deficits (BCSI)23. All patients with SS who received the questionnaire met the 2002 AECG criteria for pSS24. Our study was approved by the ethics committees at both participating sites. All participants gave informed consent.
To assess the relationship between cognitive symptoms and neuropsychological outcomes, we analyzed data from subjects with pSS drawn from the BioSiPS database who had volunteered to participate in a study of cognitive function and brain imaging32. Adults aged 18–60 years were eligible for our study. Subjects with diabetes, history of alcohol abuse, stroke, and seizure disorder were excluded. These subjects completed a comprehensive psychometric battery and a health questionnaire that included measures of cognitive symptoms (BCSI), depression (CES-D), sleep (PSQI), fatigue (FSS), and pain (BPI). The comprehensive neuropsychological battery included the Hopkins Verbal Learning Test-Revised33, the Stroop Color and Word Test (Stroop C)34, the Digit Span of the Wechsler Adult Intelligence Scale-3 (WAIS-3)35, the Trail Making Test A and B36, the Wisconsin Card Sorting Test37, the Digit Symbol subtest of the WAIS-335, the Controlled Oral Word Association test (COWAT)38, the Boston Naming test39, the Similarities A subtest of the WAIS-335, and the Paced Auditory Serial Addition Test40.
Statistical analysis
Internal consistency reliability of the separate BCSI items was evaluated by calculation of Cronbach’s alpha statistic and corrected component-total correlation coefficients (Cronbach, 1951). Factor structure was investigated with principal components analysis. As the primary analysis of validity, we assessed the degree to which the BCSI detected differences between patients and controls. We calculated Pearson correlation coefficient to test for association between measures of mood, fatigue and pain, sociodemographic variables, and BCSI scores. Total score distribution was analyzed to establish cutoff criteria for differentiation of case versus non-case identity. We then tested the ability of the threshold BCSI score to detect differences in performance in objective tests of cognitive function across multiple cognitive domains by comparing cognitive outcomes in patients above and below the threshold.
RESULTS
Cognitive symptoms in SS and health
In the first survey, 120 adult patients from the BioSiPS database of over 400 patients with SS received the questionnaire. The response rate was 65%, and was similar in seropositive and seronegative patients. The clinical and serologic profile of the non-responders to the survey was similar to the 65% who did respond.
Data from seronegative and seropositive patients were compared. Results were analyzed on 78 patients with SS (92% female) in group I (59% seropositive and 41% seronegative). For the cross validation study, 100 patients received the questionnaire and complete data was available for 66 patients with pSS (96% female, 85% seropositive) and 35 controls (100% female). Over 90% of all the survey respondents were white. There was no difference between patient groups in sex or ethnicity. Seronegative patients in group I were older and had shorter disease duration (Table 2). Controls were younger, and more controls had postgraduate education. Each of the 6 items was rated similarly by seropositive and seronegative patients. Median BCSI scores were 33 for each of the patient groups and 11 for the control group. Cognitive symptom mean scores (SD) were 35.88 (32.2) in the total patient group versus 15.1 (16.6) in controls (p < 0.0001). Thus, the 6-item scale clearly distinguished patients from controls.
Characteristics of the study sample. For variables with superscripts, the group of all patients with pSS together is statistically significantly different from the controls (p < 0.05). Within rows, values with matching superscripts are statistically different from each other (p < 0.05).
Psychometric properties of the BCSI: missing data, floor and ceiling effects, internal consistency reliability, and validity
For the patients in group I, missing data were less than 2%. Item 1 and item 2 each had 1 missing (1.28%), item 3 through item 6 had no items missing. Floor and ceiling effects were minimal based on the finding that 4 patients (5.1%) had thinking score of 0, and 2 patients (2.6%) had thinking score of 100. Internal consistency and factor structure of the BCSI were assessed in 78 patients with pSS in group I. Internal consistency reliability was good, as measured by Cronbach’s alpha of 0.909. This is well above the generally accepted thresholds of 0.7 or 0.8. Corrected item-to-total correlations (Table 3) ranged from 0.64 to 0.91, with the lowest being for question 6 (“Find the correct word during a conversation”). Values of Cronbach’s alpha when individual items are deleted are no better than the overall value of 0.909, indicating that all of the individual items are important. Principal Components Analysis revealed only 1 underlying factor, indicating a single domain structure of the BCSI.
Item-to-total correlations and Cronbach’s alpha with deleted variable.
Correlations between BCSI scores and measures of fatigue, pain, depression, and anxiety are shown for the group I patients in Table 4. With 78 observations, a correlation of 0.291 would be statistically significant at 0.01, and a correlation of 0.367 would be statistically significant at 0.001. The correlations (all r between 0.44–0.65) of the composite BCSI with self-report measures of negative affect, fatigue, and pain, suggest overlap, but not identity with behavioral variables. We calculated Pearson correlation coefficients in group II patients with pSS for the composite BCSI scores and measures of somatic fatigue (FSS, r = 0.62), depression (CES-D, r = 0.58), and sleep disorder (PSQI, r = 0.58). All correlations again suggest overlap, but not identity with the BCSI scores.
Correlation of each item of the BCSI with clinical symptoms in patients with SS.
Neither age nor disease duration were correlated with the BCSI, hence the BCSI score is not confounded by those variables. We did find that subjects with college education had fewer cognitive symptoms than those with no college (p < 0.0001). The difference in cognitive symptoms between the patient group and controls remained significant (p < 0.0001) after adjusting for education (no college, college, postgraduate). Cognitive symptom mean (95% CI) in the patient group was 36.21 (32.60, 39.83) compared to the education-adjusted control mean (95% CI) of 19.90 (12.37, 27.43).
Relationship between cognitive symptoms and objectively measured cognitive performance
A cutoff score of 50 is suggested for the definition of cognitive dysfunction based on the finding that a score of > 50 corresponded to about 80% of the distribution of patient scores and < 5% of the score distribution in controls. A score > 50 represents about 2 SD above the mean value from the control group. For comparison of cognitive test results, 18 patients who underwent formal neuropsychometric evaluation were divided into 2 groups: patients with high BCSI scores (> 50) and patients with scores ≤ 50 (Table 5). Age was no different in the 2 groups. Subjects with high BCSI scores had slightly less education. The “cutoff” score identified subjects with SS who had significantly worse cognitive performance. The greatest difference between groups (effect size > 1) was noted in the Digit Symbol test (ability to sustain attention) and the Wisconsin Card Sorting Test (executive function). The Paced Auditory Serial-Addition Test (working memory) was also significantly worse in patients with high BCSI scores. Moderate effects (0.45 to 0.88) were found on multiple tests: the Hopkins Verbal Learning Test (list learning task, verbal memory), the Digit Span of the WAIS-3, the Trail Making Test A and B (working memory), and the Stroop C (concentration effectiveness). Tests that were not different between groups included Similarities A (language, reasoning), COWAT (verbal fluency), Boston Naming (language), and Trail Making Part A (a simple timed test of visual attention; Part B is considered a measure of executive function because it requires shifting of attention between 2 sets of stimuli)36. Subjects with more severe cognitive symptoms had greater pain severity, depression, fatigue, and worse sleep (effect size > 1).
Symptoms and neuropsychometric outcomes in patients with thinking scores ≤ 50 and greater than 50 years old.
DISCUSSION
Our current study supports the use of the BCSI in patients with pSS as a first step in cognitive assessment. From the standpoint of practical application, the BCSI is very easy to administer and easily scored by hand in only a few minutes. Instructions are clear and easily understood. Administration requires little cost and no special training to interpret.
The psychometric properties of the BCSI appear adequate to support its use in clinical practice and research as a measure of cognitive symptoms. The value of 0.909 for Cronbach’s alpha clearly exceeds the recommended minimum standard for internal consistency and hence the BCSI would appear to be adequate for a wide variety of potential applications. Given the high degree of internal consistency of the individual items, we suggest reporting the total score.
Two self-report instruments, the Cognitive Failures Questionnaire (CFQ) and the Perceived Deficits Questionnaire-Short Form (PDQ), have been used in patients with rheumatic disease but neither questionnaire has been validated in patients with SS41,42,43. The CFQ was meant to identify lapses in memory, attention, and action44. Unlike the PFQ and the CFQ, the BCSI attempts to measure everyday memory and concentration, and appears to have a single factor. The BCSI may be considered the easiest for busy clinicians to score and the most reliable for low-literacy populations.
The BCSI, like the CFQ and other self-report measures of cognition, is correlated with current symptoms of anxiety and depression. The BCSI is associated with measures of fatigue, pain, and sleep quality as well. The moderate magnitude of relationships demonstrated in our study between cognitive symptoms and pain, fatigue, and sleep quality provides evidence of construct validity consistent with the known effects of poor sleep and pain on cognitive function45,46. The effects of pain, poor sleep, fatigue, and depression on cognitive symptoms in our study were equal or greater than objectively measured cognitive deficits. Our findings suggest that evaluation of fatigue, pain, and depressive symptoms will aid in the interpretation of cognitive complaints. We also found that higher education had a positive effect on cognitive symptoms. The effect of education on cognitive symptoms is of interest in light of the known effect of education on cognitive reserve, which tempers cognitive decline and has been linked to executive function.
The usefulness of subjective questionnaires to “screen” for cognitive impairment has been called into question because self-report of cognitive problems is subject to both overreporting and underreporting19. Absence of cognitive complaints does not exclude the presence of cognitive problems. Vogel, et al reported in a study of Danish patients with SLE that the level of subjective cognitive complaints was low even among patients with cognitive impairment41. If the clinical context warrants, formal neuropsychological evaluation should be considered even in the absence of cognitive complaints.
Previous studies have shown that depressive symptoms and cognitive symptoms are highly correlated18. Lack of confidence or low self-regard, rather than cognitive decline, might increase perception of cognitive difficulties, leading patients to overestimate their cognitive deficits. Patients with primary depression have impairments in attention, memory, psychomotor speed, and executive function47. A nonspecific pattern of subcortical impairment is also described in patients with chronic pain46. Future studies aimed at investigating the causes of cognitive dysfunction in SS should include appropriate controls with chronic pain and primary depression.
The mean BCSI scores are not different from those previously reported in a large survey of patients with SS at university centers across the United States, suggesting that our sample is representative of the SS population48. Our data provide strong support for the view that clinicians need to carefully evaluate patients with SS who have concerns about cognitive symptoms. Subjective self-report of cognitive symptoms appears to reflect problems such as the inability to sustain concentration, which can contribute to serious limitations in the ability to complete work-related tasks. Deficits in attention, working memory, and executive function may lead to substantial difficulties with work performance and contribute to work disability.
Multiple factors contribute to cognitive symptoms. Clinicians should consider whether psychological factors such as pain, sleep problems, anxiety, and depression coexist with cognitive symptoms, as has previously been emphasized by Shin, et al in patients with RA17. The BCSI can be used to assess the severity of perceived cognitive difficulties. If the BCSI is used as a clinical tool to screen patients for cognitive symptoms, a cutoff score of > 50 is indicative of a high level of cognitive symptoms, but should be interpreted no further than as indicating that formal neuropsychological assessment may be necessary and should therefore be sought.
When organic cerebral involvement is suspected, detailed psychometric evaluation is necessary to differentiate subcortical cerebral involvement, which is typically nonprogressive, from the pattern of cortical dysfunction associated with early Alzheimer disease. In general, self-rating questionnaires do not reflect age-related changes in cognitive ability49. On the contrary, in 1 study, people in their 50s reported more cognitive lapses than did individuals in older decades49. Whether mild problems with concentration and memory develop into more serious cognitive impairment over time in patients with pSS is unknown.
Our cross-sectional study has several limitations. We did not assess the BCSI sensitivity to change. The effects of sex could not be evaluated given the small number of male participants. Our survey respondents might be representative only of patients with pSS who were motivated to enroll in research of this type; however, our study population reported rates of depression, fatigue, pain, and cognitive symptoms that are similar to those described previously in large pSS cohorts. The design of our study precluded obtaining data from respondents regarding their neurologic status or extraglandular manifestations. Longitudinal data are needed to assess those factors that contribute to cognitive symptoms or modify the advance of cognitive dysfunction. Similarly, it will be important to clarify whether subjective memory complaints predict future cognitive outcomes in SS.
Our suggestion of a cutoff score of > 50 should be regarded as preliminary until there is independent confirmation. Additional research is needed to assess the associations between BCSI scores and performance on objective cognitive tests, particularly in the domains of memory, attention, and executive function. More research is also needed on the relationship between cognitive symptoms and relevant outcomes such as work disability, “presenteeism”, and functional ability.
Cognitive symptoms are a prominent feature of SS in both seropositive and seronegative patients. The BCSI appears to constitute a promising tool for the evaluation of cognitive symptoms. High scores are associated with psychological symptoms, as well as with poor performance in multiple cognitive domains independent of age or disease duration. BCSI scores should be interpreted no further than as indicating that neuropsychological assessment and attention may be necessary, and should therefore be sought.
Acknowledgment
We thank our research assistants Valerie Ferment, Tristan Van de Moortele, and Brian Pogatchnik. We also gratefully acknowledge the kind cooperation of the subjects; without their cooperation, our study would not have been possible.
Footnotes
-
Published online before print September 15, 2014, doi: 10.3899/jrheum.140362
-
Supported by the US National Institutes of Health R01AR50782, “Gene Expression Profiling in Sjögren’s Syndrome”.
- Accepted for publication June 27, 2014.
REFERENCES
In this issue
