Abstract
Objective. There is a need for a cognitive function screening test that can be administered to patients with systemic lupus erythematosus (SLE) in clinic. The objectives of this study were to determine (1) prevalence of cognitive impairment (CI) in SLE by the Montreal Cognitive Assessment (MoCA), Mini Mental State Examination (MMSE), in relation to the Hopkins Verbal Learning Test–Revised (HVLT-R), and Perceived Deficits Questionnaire 5-Item (PDQ-5); and (2) associated factors with CI.
Methods. Consecutive patients followed at a single center were recruited. HVLT-R, MoCA, and MMSE were administered. Sensitivity/specificity, positive (PPV)/negative (NPV) predictive values, and positive likelihood ratio (LR+) of MoCA/MMSE were determined (compared to HVLT-R). A test on intellectual ability and questionnaires on anxiety, depression, and perceived cognitive deficits were completed. Regression analyses determined associations with CI.
Results. Of 98 patients, 48% had CI using MoCA and 31% using HVLT-R. Sensitivity was higher for MoCA (73%) compared to MMSE (27%), though MMSE was more specific (90%) than MoCA (63%). PPV and LR+ were similar in MoCA and MMSE (PPV: 47%, 53%; LR+: 2.0, 2.6, respectively), but NPV was higher in MoCA (84%) than MMSE (74%). PDQ-5 predicted objective CI (HVLT-R: sensitivity 100%, specificity 89%). Although CI was associated with depression in univariate analyses, it did not hold in the multivariate analysis, while longer SLE disease duration and more years of education remained significant.
Conclusion. CI is highly prevalent and MoCA may be a useful tool to screen for CI in SLE. Patients with more years of education were less likely to have CI.
- COGNITIVE IMPAIRMENT
- SYSTEMIC LUPUS ERYTHEMATOSUS
- MONTREAL COGNITIVE ASSESSMENT
- MINI MENTAL STATE EXAMINATION
- HOPKINS VERBAL LEARNING TEST–REVISED
Cognitive impairment (CI) is among the most common of the neuropsychiatric manifestations of systemic lupus erythematosus (SLE), with a prevalence ranging from 20–80%1,2,3,4. Even mild CI can significantly reduce a patient’s quality of life and employment potential5,6,7. As a result, early detection of CI may direct patient care to help patients adapt accordingly to lessen the effects of cognitive decline. Currently, the screening and diagnosis of CI in SLE is delayed and its monitoring is not well developed. This situation is likely due to the lack of appropriate cognitive screening assessment tools for this population8,9,10,11,12,13.
Although there is no pattern of CI that is specific for patients with SLE, Hanly, et al noted that decreased attention, impaired working memory, and executive function (e.g., planning and multitasking) are often affected in patients with SLE in addition to overall cognitive slowing14. The gold standard test recommended by the American College of Rheumatology (ACR) to assess cognitive function is a 1–2 h battery (the ACR-SLE battery) of tests15. This test is time-consuming and is associated with a cost burden, limiting the practicality of its administration as a screening tool on all patients with potential cognitive decline. Kozora, et al confirmed the validity and reliability of the ACR-SLE battery for patients with SLE against the larger 4-h battery15. The ACR-SLE battery encompasses tests that have shown impairment in previous studies of patients with SLE and these tests highlighted a deficit in “complex attention, deficit in learning and recall, verbal and nonverbal fluency, complex psychomotor functions, visuospatial skills and motor dexterity”15,16.
Because administering the ACR-SLE battery is time-consuming in the clinic, several other cognitive assessment tools, including the HVLT-R and Montreal Cognitive Assessment (MoCA), have been studied in SLE16,17. The Hopkins Verbal Learning Test-Revised (HVLT-R), a word-list learning and memory test, is brief, easy to administer and to score, and yields indices of both recall and recognition. The HVLT-R assesses verbal learning efficiency, ability to access newly learned information, and retention18.
HVLT-R has been validated against the ACR-SLE battery by telephone interview16. HVLT-R was used in our study because of its psychometric properties; in particular, validity and ease of administration (< 30 min), and representation of testing in the domain of delayed recall, which is a domain of both MoCA17 and the Mini Mental State Examination (MMSE). The MoCA was designed to detect mild CI19, as was the MMSE, with some evidence showing that the latter is inferior to the MoCA for this purpose20,21,22. The Perceived Deficits Questionnaire 5-Item (PDQ-5) is an assessment of perceived CI from the patient’s perspective16,23,24. PDQ-5 was included in our study to investigate the possible association or discrepancy between patients’ self-reported and objective CI.
Our aims were (1) to determine the prevalence of CI as evaluated by MoCA, MMSE, relative to the HVLT-R, and PDQ-5; and (2) to define the association between patients’ demographic/psychosocial/clinical factors and cognitive function.
MATERIALS AND METHODS
Patients’ setting, clinical, and laboratory assessment
Adult patients with SLE (≥ 18 yrs, 4 or more of the ACR criteria or 3 ACR criteria plus a typical histological lesion of SLE on renal or skin biopsy) were recruited from the University of Toronto Lupus Clinic25. A complete history including demographics, disease manifestations, physical examination, and laboratory evaluation is obtained at each visit and entered into the lupus research databank. Patients were considered to have a positive smoking status if they were documented as a current smoker at any point in the 5 years preceding study start date. History of cardiovascular events (i.e., stroke, transient ischemic attack, myocardial infarction, congestive heart failure), diabetes, dyslipidemia, and hypertension (> 140 systolic or > 90 diastolic or taking any antihypertensive medication) was also available from the database. Antiphospholipid antibodies (aPL), including anticardiolipin antibodies (IgG or IgM) and lupus anticoagulant were considered present if patients had tested positive on 2 or more occasions since joining the SLE cohort. Antiphospholipid syndrome was defined as 2 or more aPL-positive tests at least 12 weeks apart, in the presence of at least 1 of the 2 major clinical manifestations (i.e., vascular thrombosis or pregnancy morbidity) of the syndrome26. SLE disease activity was measured with the SLE Disease Activity Index 2000 (SLEDAI-2K)27 and damage measured by the Systemic Lupus Erythematosus International Collaboration Clinics/ACR Damage Index28. Data on the use of psychotropic agents, pain medications, aspirin, and warfarin were obtained and analyzed. Fatigue was measured by the Fatigue Severity Scale (FSS). This project has been reviewed and approved by the University Health Network Research Ethics Board (REB#15-9582-BE). All patients provided informed consent.
Patient selection
Consecutive SLE adult patients who attended the Lupus Clinic between February 2014 and July 2015 and agreed to participate were recruited. Patients were excluded if they could not be reached by telephone, were non-English speaking, or were mentally/physically unwell and unable to participate as recommended by the Lupus Clinic healthcare team.
Cognitive testing
Three cognitive screening tests (HVLT-R, MoCA, and MMSE) were administered to patients by 1 of 2 trained assessors (SN and AD). Both assessors were trained by ZT on the administration of the cognitive tests. Although HVLT-R focuses on verbal learning and memory, this test showed adequate sensitivity (74%) in identifying patients with CI with acceptable specificity (68%) compared to ACR-SLE battery16. HVLT-R positive predictive value (PPV) was 43% and negative predictive value (NPV) was 89%16. The test-retest reliability of the HVLT-R with a mean interval of 6 weeks showed a reliability coefficient of 0.74 for total recall and 0.66 for delayed recall29. In our study, MoCA and MMSE were compared to HVLT-R. Each patient who agreed to participate completed 2 phases of our study: telephone assessment and in-clinic assessment. The time interval between study phases was limited to 2 weeks.
Phase 1 – telephone assessment
Patients were telephoned and verbal consent to participate in the study was obtained. Participants were then called at a time that suited their convenience within the 2-week time period and HVLT-R assessment test was conducted (the administration time for HVLT-R is 25–30 min). HVLT-R measured verbal learning16. Participants were asked to avoid writing anything down during the testing.
Phase 2 – in-clinic assessment
Patients completed the MoCA and MMSE through face-to-face testing directly before or after their regular clinic visit. MoCA and MMSE each took less than 10 min to complete. The order of administration of MoCA and MMSE was counterbalanced between patients.
Patient-reported outcomes (depression, anxiety, and PDQ-5 tests) and an intelligence screening test
SN and AD administered the Reynolds Intellectual Screening Test (RIST)30. The RIST is a screening measure of general intelligence that can be used to assess risk of functional intellectual impairment. It was included in our study to determine whether general intelligence affected performance on MoCA and MMSE as it has in previous studies31. The verbal component assesses vocabulary, language development, and general knowledge while the nonverbal component assesses nonverbal reasoning, spatial ability, and visual imagery30. All patients had normal or corrected-to-normal vision at the time of the study, which was relevant for the nonverbal intellectual assessment. Patients also completed the following self-report questionnaires: (1) Center of Epidemiologic Studies Depression Scale (CES-D: score range 0–60)32; (2) Beck Anxiety Inventory (BAI: score range 0–63)32; and (3) PDQ-5. RIST and each of the self-report questionnaires were completed in person on the same day as the MMSE and MoCA administration. PDQ-5 includes 5 questions representing 4 subscales: Attention/concentration, Retrospective Memory, Prospective Memory, and Planning/organization23. Patients rated their cognitive difficulties on a 4-point Likert scale (0 = never to 4 = almost always).
Scoring the tools
For HVLT-R, age-adjusted T scores were derived for total recall and delayed recall33. Impairment was assigned for HVLT-R if performance in either total recall or delayed recall categories fell below or equal to −1.5 SD of the population normative data. For MoCA and MMSE, CI was assigned if score fell below 2619,34. The RIST verbal and nonverbal tests were summed to produce a composite RIST index score (100 ± 15). A CES-D score ≥ 16 was considered positive for depressive symptoms. Patients tested positive for moderate to severe anxiety if their BAI score was ≥ 22. Total PDQ-5 score consisted of the sum of the raw scores on these 5 items and could range from 0–20 with higher scores indicating greater perceived deficit23. Prevalence of self-reported CI by PDQ-5 was determined based on the number of patients with difficulties occurring “often” or “almost always” in at least 1 of the PDQ-5’s 4 subscales.
Statistical analysis
Descriptive statistics were used to characterize the SLE population and to examine differences in demographic and clinical characteristics between patients with and without CI as defined by HVLT-R. Prevalence of CI was calculated for HVLT-R, MoCA, MMSE, and PDQ-5. MoCA subdomain analysis was performed to report scores on MoCA in patients with CI as defined by HVLT-R19,35. Sensitivity/specificity, PPV/NPV, and positive likelihood ratios (LR+) of MoCA and MMSE were calculated compared to HVLT-R. Correlation between HVLT-R with MoCA and MMSE was determined using Spearman correlation coefficient (r < 0.40 = weak; r 0.40–0.60 = moderate; r > 0.60 = strong). Sensitivity, specificity, PPV, and NPV of PDQ-5 in detecting CI (based on HVLT-R), depression (CES-D), and anxiety (BAI) were studied. Univariate regressions were performed on demographic and clinical variables with the outcome of CI (CI was defined by HVLT-R total recall or delayed recall). These variables included sex; race; age at time of study; age at SLE diagnosis; SLE disease duration; years of education; disease activity (SLEDAI-2K); disease damage (Systemic Lupus International Collaborating Clinics/ACR Damage Index); screenings of intelligence (RIST), depression (CES-D), anxiety (BAI), and fatigue (FSS); cardiovascular events; hypertension; diabetes; antiphospholipid status; smoking status; dyslipidemia; and medication use (psychotropic and pain medications, aspirin, and warfarin). Variables with p < 0.2 were entered into multivariate logistic regression using a stepwise variable selection method (i.e., p value had to be < 0.30 to be entered into next round, then < 0.15 to be retained in following round of variable selection). Hosmer-Lemeshow test was used in detecting evidence of model lack of fit in multivariate analysis (the model was rejected if p value was < 0.05 and it was accepted if p value was > 0.05).
A further analysis was conducted using receiver-operating characteristic curves to examine the predictive power of different MoCA and MMSE cutoffs for CI. Area under the curve of 0.7–0.9 indicates moderate accuracy, 0.5–0.7 indicates low accuracy, and ≤ 0.5 is equal to chance. Cutoffs with optimized sensitivity and specificity for MoCA and MMSE were identified by the Youden index. Data were analyzed using SAS version 9.3 (SAS Institute Inc.).
RESULTS
Patients
Demographic and clinical characteristics of the 98 patients with SLE recruited are shown in Table 1. Note that all patients completed HVLT-R, MoCA, and MMSE. There were no significant differences in the characteristics of the participants and the patients who attended the Lupus Clinic but did not participate in the study (n = 727), with the exception of years of education (15.2 ± 2.9 and 14.3 ± 2.6, respectively, p < 0.001) and dyslipidemia (82.7% and 68.9%, respectively, Appendix 1).
Demographics, clinical characteristics, cognition, patient-reported outcomes, and logistic regression to test the association of variables with outcome cognitive impairment (CI) in patients with SLE as defined by HVLT-R total recall or delayed recall.
Cognitive test performance
Prevalence of CI was highest using MoCA (48%) and lowest using MMSE (15%). The prevalence of CI by HVLT-R was 31%. Of the 15 patients who tested positive for CI using MMSE, 11 also tested positive using MoCA. In comparison with HVLT-R, sensitivity was higher for MoCA (73%) compared to MMSE (27%), though MMSE was more specific (90%) than MoCA (63%). PPV and LR+ were similar in both MoCA and MMSE (PPV: 47% and 53%; LR+: 2.0 and 2.6, respectively), but NPV was higher in MoCA (84%) than MMSE (74%).
Spearman analysis indicated moderate correlation (r = 0.42) between HVLT-R and MoCA, with lower correlation between HVLT-R and MMSE (Table 2). MoCA subdomain analysis showed that patients with CI as defined by HVLT-R scored significantly lower in the areas of attention, language, and delayed recall when compared to published data of normal controls19,35 (p values were significant). MoCA subdomains were also lower in areas of attention and delayed recall when compared to non-CI patients with SLE (Figure 1).
Scores on MoCA subdomains of patients with CI (n = 30; age 44 ± 12 yrs) as defined by HVLT-R compared to normative data from a previous study (n = 90; age 73 ± 7 yrs)20 and non-CI patients in this cohort. MoCA: Montreal Cognitive Assessment; HVLT-R: Hopkins Verbal Learning Test–Revised; CI: cognitive impairment.
Spearman correlation of MoCA and MMSE compared to HVLT-R.
Analysis of MoCA and MMSE cutoffs
The cutoff of 26 (by Youden index) for MoCA [area under the curve (AUC) 0.71] predicted CI with a sensitivity of 63% and specificity of 73%. MoCA cutoff of 25 predicted CI with a sensitivity of 71% and a specificity of 57%. For MMSE, the best cutoff, by Youden index, to predict CI was 28 (AUC 0.66) with a sensitivity of 74% and specificity of 57%. MMSE cutoff of 26 predicted CI with a sensitivity of 90% and specificity of 27%. These results should be interpreted with caution because of the relatively small sample size.
Results of intelligence screening, depression, anxiety, and PDQ-5 tests
Of 98 patients, 94% completed the RIST and 100% completed CES-D. One value is missing for BAI. PDQ-5 was completed by 82%. Mean RIST index score reached significance in patients with and without CI (p = 0.02), with lower scores in patients with CI. Patients with CI (defined using HVLT-R) had significantly higher mean CES-D scores than patients without CI, where higher scores indicated more depressive symptoms. In total, 45% of patients tested positive for depressive symptoms (60% of CI, 38% of no CI; p = 0.046) based on CES-D. Patients commonly reported symptoms of anxiety (40% of CI, 22% of no CI, p = 0.049), although mean BAI score was not statistically significantly different in patients with and without CI (19.0 ± 13.1 CI vs 14.7 ± 11.3 no CI, p = 0.11).
The PDQ-5 mean score was 8.9 ± 4.1 (80 patients had non-missing PDQ-5 and HVLT pairs). Among the patients with CI by HVLT-R (n = 30), 23 reported cognitive difficulties occurring “often” or “almost always” in at least 1 of PDQ-5’s 4 subscales (mean PDQ-5 10.0 ± 4.5). Among the patients without CI by HVLT-R (n = 68), 50 patients reported cognitive difficulties occurring “often” or “almost always” in at least 1 of PDQ-5’s 4 subscales (mean PDQ-5 8.3 ± 4.0). There was no significant difference in PDQ-5 mean scores between patients with and without CI (p = 0.10), and the PDQ-5 was inversely correlated with HVLT-R (Spearman correlation of −0.31, p = 0.006). Sensitivity and specificity of PDQ-5 were similar in both depression (CES-D: sensitivity 46%, specificity 69%, PPV 53%, NPV 62%) and anxiety (BAI: sensitivity 50%, specificity 67%, PPV 34%, NPV 80%). PDQ-5 predicted objective CI (HVLT-R: sensitivity 100%, specificity 89%, PPV 80%, NPV 100%).
Logistic regression analysis
In the univariate linear regression analysis, patients without CI also had more years of education (OR 0.87). A higher CES-D score was more prevalent in patients with CI compared to non-CI patients (60% vs 38%, OR 1.05). SLE patients without CI had significantly longer SLE disease duration (OR 0.95; Table 1). These variables had no statistically significant difference in univariate linear regression analysis: sex; race; age at time of study; age at SLE diagnosis; disease activity; disease damage; screenings of intelligence, depression, anxiety, and fatigue; cardiovascular events; hypertension; diabetes; antiphospholipid status; smoking status; dyslipidemia; and medication use (glucocorticoids, psychotropic and pain medications, aspirin, and warfarin).
In the multivariate analysis, more years of education (OR 0.81) and longer SLE disease duration (OR 0.94) remained significant in multivariate analysis and were associated with CI (Table 1). Each 1-year increase in the years of education reduces the probability of CI by 19%.
DISCUSSION
CI appears to be highly prevalent in patients with SLE and our study found that the prevalence of CI was 48% when screened using MoCA and 31% using HVLT-R. These results need to be carefully interpreted because screening cognitive tests might not be as specific as the ACR-SLE battery. We also found that PDQ-5 was predictive of CI identified by HVLT-R.
MoCA was a more sensitive screening measure for CI than MMSE, suggesting that MoCA may be identifying patients with mild CI. The sensitivity and NPV of the MoCA and MMSE are only fair using the HVLT-R as an external construct. A higher sensitivity for MoCA may have been reached if the external construct had been the ACR-SLE battery because HVLT-R focuses heavily on memory performance, which does not encompass all areas of cognitive function affected in SLE, let alone MoCA. It is also possible that a higher sensitivity of the MoCA may have been reached if the HVLT-R had been administered in person instead of over the telephone. Alternatively, perhaps the MoCA is not sufficiently sensitive in this population, and another cognitive screening assessment tool may prove to be more beneficial after further studies. MoCA subdomain analysis found that patients with SLE had particular difficulty in areas of attention, language, and delayed recall. One may worry that deficits in these 3 domains could be devastating for patients, perhaps affecting their self-esteem, limiting their ability to work or to complete daily tasks such as cooking, reading, or remembering recent conversations. Objective evidence of such deficits with further diagnostic testing in patients with SLE may help treating physicians and other healthcare providers focus their efforts into these specific areas of CI. The subgroup analyses focusing on determining the best cutoffs for MoCA and MMSE to predict CI (identified by HVLT-R) yielded the currently accepted cutoff, 26, for MoCA. The analysis for MMSE yielded a cutoff of 29 to predict CI, which is higher than the currently accepted cutoff of CI by MMSE, 26. This finding is not clinically meaningful, however, because a score of 29/30 on the MMSE only slightly deviates from a perfect score and therefore is unlikely to reflect true cognitive deficiency.
Owing to the cost and administrative burden, particularly ease of use and time needed to administer (< 10 min), and based on our analysis, MoCA can be the preferential screening test for an ambulatory clinic setting because it is available free in university settings. HVLT-R is more expensive and takes 25–30 min to administer. MMSE is less ideal because it tends to miss many patients with mild CI. When screening for CI, both high sensitivity (to avoid false-negative results) and high specificity (to identify with confidence patients who do not have the disease) are desired. Further studies are required to determine whether MoCA is effective as a screening test for CI, with clear benefits and negligible harms.
The self-report measure of CI (PDQ-5) predicted objective CI (identified by HVLT-R) and to a lesser degree underlying depression and/or anxiety. Julian, et al16 found self-reported cognitive complaints to be more associated with depression and/or anxiety in SLE and rheumatoid arthritis. Hanly, et al14 also found that self-reported cognitive complaints are influenced by the presence of depression and anxiety. Although there is often a discrepancy between patients’ self-reported and objective cognitive dysfunction, their perceived deficits can be quite disabling functionally and should be assessed clinically. Therefore, PDQ-5 should be administered alongside cognitive screening assessment tools, and further evaluation of depression and anxiety should be analyzed if necessary.
The risk factors for CI in patients with SLE are unclear, and past studies have yielded conflicting results36. Indeed, coping with a chronic illness such as SLE37 can become a psychological burden that may contribute to a pattern of diffuse and nonprogressive cognitive inefficiencies in everyday life. Our study also showed that higher education is inversely correlated with CI. More years of education may buffer the effect of cognitive decline. It has been suggested that life experiences such as educational exposure may increase cognitive reserve, allowing more-educated patients to cope better with factors or conditions that contribute to cognitive decline, compared to patients with less education38. We also found that patients with longer SLE duration were less likely to have CI. Of note is that the mean SLE disease duration was 14.2 ± 8.9 years versus 20.4 ± 12.7 years for those with and without CI, respectively. Thus, both groups had a long history with SLE. Our results should be interpreted with caution, especially because it is very possible that patients with more severe CI were not able to participate in our study. We found no significant association between self-reported social supports.
There were several limitations to our study. HVLT-R was considered the external construct instead of the ACR-SLE battery because of the lack of grants to administer this battery. It is possible that the 2-week time interval allowed between the telephone and the face-to-face administration of tests was enough to have variability in cognitive function. However, the mean time between the 2 interviews was 5 ± 5 days. Another limitation is the use of published normative data to classify patients with CI. A local matched control group may have been a better representation of the population as a whole. It is important to note that the norms for the HVLT-R are based on standardized, in-person administration and not on telephone administration.
The prevalence of CI in patients with SLE is high, thus the need for a brief, valid, sensitive cognitive screening test that is both cost-effective and easily administered in an ambulatory setting. MoCA can be used in a clinic setting to screen the SLE population for CI, but further studies are required to confirm its validity. Research will focus on patients who should be considered for further diagnostic cognitive testing.
Acknowledgment
The authors acknowledge the work of medical students Sonali de Chickera and Amy Miles, who worked on the preparation of the Research Ethics Board application and initial subject recruitment, and the support of the Toronto Lupus Clinic Research Program and directors Drs. Murray B. Urowitz and Dafna D. Gladman.
APPENDIX 1.
Demographics of participants (n = 98) and nonparticipant patients (who attended the Lupus Clinic but did not participate in the study; n = 727).
Footnotes
The Toronto Lupus Clinic Research Program is supported financially by the University Health Network, the Lou Rocca family, and the Lupus Foundation of Ontario. Dr. Touma’s research is supported by the Young Operating and the Young Investigator Salary Award of the Arthritis Society, and the New Investigator Research Grant of the Physicians’ Services Inc. Foundation.
- Accepted for publication June 23, 2017.
