Skip to main content

Main menu

  • Home
  • Content
    • First Release
    • Current
    • Archives
    • Collections
    • Audiovisual Rheum
    • COVID-19 and Rheumatology
  • Resources
    • Guide for Authors
    • Submit Manuscript
    • Payment
    • Reviewers
    • Advertisers
    • Classified Ads
    • Reprints and Translations
    • Permissions
    • Meetings
    • FAQ
    • Policies
  • Subscribers
    • Subscription Information
    • Purchase Subscription
    • Your Account
    • Terms and Conditions
  • About Us
    • About Us
    • Editorial Board
    • Letter from the Editor
    • Duncan A. Gordon Award
    • Privacy/GDPR Policy
    • Accessibility
  • Contact Us
  • JRheum Supplements
  • Services

User menu

  • My Cart
  • Log In

Search

  • Advanced search
The Journal of Rheumatology
  • JRheum Supplements
  • Services
  • My Cart
  • Log In
The Journal of Rheumatology

Advanced Search

  • Home
  • Content
    • First Release
    • Current
    • Archives
    • Collections
    • Audiovisual Rheum
    • COVID-19 and Rheumatology
  • Resources
    • Guide for Authors
    • Submit Manuscript
    • Payment
    • Reviewers
    • Advertisers
    • Classified Ads
    • Reprints and Translations
    • Permissions
    • Meetings
    • FAQ
    • Policies
  • Subscribers
    • Subscription Information
    • Purchase Subscription
    • Your Account
    • Terms and Conditions
  • About Us
    • About Us
    • Editorial Board
    • Letter from the Editor
    • Duncan A. Gordon Award
    • Privacy/GDPR Policy
    • Accessibility
  • Contact Us
  • Follow jrheum on Twitter
  • Visit jrheum on Facebook
  • Follow jrheum on LinkedIn
  • Follow jrheum on YouTube
  • Follow jrheum on Instagram
  • Follow jrheum on RSS
Research ArticleArticle

The Framingham Score and the Systematic Coronary Risk Evaluation at Low Cutoff Values Are Useful Surrogate Markers of High-risk Subclinical Atherosclerosis in Patients with Rheumatoid Arthritis

Patrick H. Dessein, Alfonso Corrales, Raquel Lopez-Mejias, Ahmed Solomon, Angela J. Woodiwiss, Javier Llorca, Gavin R. Norton, Fernanda Genre, Ricardo Blanco, Trinitario Pina, Carlos Gonzalez-Juanatey, Linda Tsang and Miguel A. Gonzalez-Gay
The Journal of Rheumatology March 2016, 43 (3) 486-494; DOI: https://doi.org/10.3899/jrheum.150510
Patrick H. Dessein
From the Cardiovascular Pathophysiology and Genomics Research Unit, School of Physiology, Faculty of Health Sciences, University of the Witwatersrand; Department of Rheumatology, Charlotte Maxeke Johannesburg Academic Hospital, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa; Epidemiology, Genetics and Atherosclerosis Research Group on Systemic Inflammatory Diseases, Rheumatology Division, Instituto de Investigación Santaria Valdecilla (IDIVAL); Department of Epidemiology and Computational Biology, School of Medicine, University of Cantabria, and CIBER Epidemiología y Salud Pública (CIBERESP), IDIVAL, Santander; Cardiology Division, Hospital Lucus Augusti, Lugo; Health Research Institute of Santiago de Compostela (IDIS), Division of Rheumatology, Clinical University Hospital of Santiago de Compostela, Santiago de Compostela, Spain.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: dessein@telkomsa.net
Alfonso Corrales
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Raquel Lopez-Mejias
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ahmed Solomon
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Angela J. Woodiwiss
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Javier Llorca
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gavin R. Norton
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Fernanda Genre
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ricardo Blanco
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Trinitario Pina
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Carlos Gonzalez-Juanatey
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Linda Tsang
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Miguel A. Gonzalez-Gay
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Supplemental
  • Info & Metrics
  • References
  • PDF
  • eLetters
PreviousNext
Loading

Abstract

Objective. We determined the performance of the Framingham score and the Systematic COronary Risk Evaluation (SCORE) in assessing high-risk atherosclerosis in patients with rheumatoid arthritis (RA).

Methods. We assembled 330 cases without established cardiovascular disease (CVD), diabetes, and moderate or severe chronic kidney disease among 451 consecutive Spanish patients who underwent CVD risk screening and carotid ultrasound-determined plaque assessment. The findings were validated in 90 black and 97 white African patients.

Results. When sensitivity for the Framingham score was set at 80% in receiver-operator curve analysis [area under the curve (AUC) = 0.799], the corresponding cutoff value and specificity were 7.3% and 63%, respectively. At a specificity of 80%, the cutoff value and sensitivity were 10.8% and 65%, respectively. When sensitivity for SCORE (AUC = 0.747) was set at 80%, the cutoff value and specificity were 0.5% and 58%, respectively. At a specificity of 80%, the cutoff value and sensitivity were 1.5% and 50%, respectively. Upon applying a cutoff value of 7.3% for the Framingham and 0.5% for SCORE in African white patients with RA, the corresponding sensitivities and specificities were 67% and 72%, and 67% and 55%, respectively. CVD risk equations did not discriminate between black African patients with and without plaque (AUC = 0.544 and 0.549 for Framingham score and SCORE, respectively).

Conclusion. The Framingham score and SCORE at markedly low cutoff values of 7.3% to 10.8% and 0.5% to 1.5%, respectively, can usefully estimate plaque presence in RA. Effective population-specific CVD risk assessment strategies are needed in black African patients with RA.

Key Indexing Terms:
  • RHEUMATOID ARTHRITIS
  • CARDIOVASCULAR DISEASE RISK STRATIFICATION
  • FRAMINGHAM SCORE
  • SYSTEMATIC CORONARY RISK EVALUATION

Traditional and nontraditional cardiovascular (CV) risk factors including disease characteristics and genetic factors mediate the enhanced atherosclerosis and CV disease (CVD) event rates in rheumatoid arthritis (RA)1,2,3,4,5,6,7,8. In the population at large, the use of multiple major traditional risk factor equations including the Framingham score and the Systematic COronary Risk Evaluation (SCORE) is currently recommended to assess CVD risk9,10. The updated version of the Framingham score and SCORE estimate the absolute 10-year risk for any CVD event and fatal CVD events, respectively10,11. Thresholds of a Framingham score of 20% or more and a SCORE of 5% or more are arbitrarily considered indications for intensified risk factor management mostly with CV drugs and particularly statins9,10.

CVD risk assessment remains inadequate in RA at present12. Thus, both the SCORE and Framingham score reportedly underestimate incident CVD event rates in RA13. Accordingly, the European League Against Rheumatism (EULAR) recommended multiplying CVD risk equation results by 1.5 in patients with RA when 2 of the 3 following disease characteristics are present: disease duration > 10 years, rheumatoid factor or/and anticyclic citrullinated positivity, and the presence of severe extraarticular manifestations, thereby providing the modified SCORE (mSCORE)3.

Ultrasound (US)-determined carotid plaque represents very high CV risk and strongly predicts incident CVD events independent of traditional CVD risk factors as included in the Framingham score, SCORE, and RA characteristics9,10,14,15. In this regard, we previously found that the Framingham score was as low as 7% in patients with RA with plaque4. Congruently, studies reported that most patients with RA who were estimated not to be at high risk by the SCORE or mSCORE actually had carotid artery plaque16,17. The Framingham score in patients with RA is nevertheless significantly associated with carotid artery atherosclerosis4, as well as electron beam tomography-determined coronary artery calcification (CAC) scores18, another marker of subclinical CVD that can enhance CVD risk stratification9,10.

Guidelines on CVD prevention recommend vascular imaging to identify carotid plaque or high CAC scores upon refining CV risk assessment9,10. In RA, a SCORE of ≥ 5% or a Framingham score of ≥ 20% is too insensitive in identifying patients with high-risk atherosclerosis who require intervention with lipid-lowering agents4,12,16,17. However, whether the application of the respective CVD risk equations at lower cutoff values can reliably assess the presence of carotid plaque and thereby preclude the need for vascular imaging in patients with RA has, to our knowledge, not been reported. In our study, we explored this possibility using receiver-operator characteristic (ROC) curve analysis in a large group of Spanish patients with RA. In addition, we validated findings obtained among Spanish patients in white and black African patients with RA.

MATERIALS AND METHODS

Patients

A total of 330 patients with RA were enrolled at the Hospital Universitario Marques de Valdecilla in Santander, Spain. These participants originated from a group of 451 consecutive patients who underwent CVD risk factor recording and carotid US as previously reported16,17. Patients were excluded when they had established CVD, including ischemic heart disease, cerebrovascular accident, peripheral arterial disease, and/or heart failure. We also excluded patients with diabetes and moderate [estimated glomerular filtration rate (eGFR) = 30–59 ml/min/1.73 m2] or severe chronic kidney disease (eGFR < 30 ml/min/1.73 m2) because these comorbidities represent high or very high CVD risk.

The external validation group consisted of 187 (90 black and 97 white) African patients with RA. These participants were derived from 243 (121 black and 122 white) consecutive patients who underwent CVD risk factor recording and carotid US at the Charlotte Maxeke Johannesburg Academic Hospital and Milpark Hospital in Johannesburg, South Africa19,20. Exclusion criteria were applied as in the Spanish patients. The study was approved by the local ethics committees and each participant gave written informed consent.

CV risk factors

Recorded characteristics are shown in Table 1. Dyslipidemia was diagnosed when patients had a total high-density lipoprotein (HDL) cholesterol ratio of > 4 and/or were using lipid-lowering agents. Other conventional CVD risk factors were defined using our previously reported methods16,17. Based on age, sex, smoking status, systolic blood pressure, total cholesterol, and HDL cholesterol concentrations, the SCORE was calculated to determine the 10-year risk of fatal CVD in a European population at low CVD risk, as recommended in persons living in Spain10. Based on the variables included in the SCORE, as well as treatment for hypertension (HTN), we calculated the Framingham score to determine the 10-year risk of any CVD events consisting of coronary heart disease, stroke, peripheral artery disease, or heart failure as reported by D’Agostino, et al in 200811 and recommended for use in South Africa21.

View this table:
  • View inline
  • View popup
Table 1.

Recorded characteristics in 330 Spanish patients with RA. Data were analyzed by Student t test, Mann-Whitney U test, or logistic regression models as appropriate. Results are expressed as mean (SD), median (interquartile range), or percent unless otherwise specified.

Recorded CV drugs included antihypertensive agents and statins. Extraarticular manifestations consisted of nodular disease, Felty syndrome, pulmonary fibrosis, rheumatoid vasculitis, and secondary Sjögren syndrome.

Carotid US

In Spanish patients, arterial atherosclerotic plaques in the extracranial carotid tree were identified using the commercially available scanner Mylab 70 Esaote equipped with a 7–12 MHz linear transducer and the automated software-guided technique radio frequency-Quality Intima-media Thickness in real time (Esaote) as previously reported16. Images were obtained of at least 1-cm length of the common carotid arteries for measurement of the intima-media thickness (IMT) of the far wall from an optimal angle of incidence defined as the longitudinal angle of approach where both branches of the internal and external carotid artery are visualized simultaneously22. Carotid artery plaque was identified as recommended in the Mannheim consensus, i.e., when a focal structure that encroaches into the arterial lumen of at least 0.5 mm or 50% of the surrounding carotid IMT (cIMT) value or demonstrates a thickness of > 1.5 mm as measured from the media-adventitia interface of the intima-lumen interface, is present22.

Among South African patients, Belinda A. Stevens, B Tech (BAS) performed the carotid artery US measurements in private healthcare patients, and AS performed the measurements in public healthcare patients. Both operators obtained images for measurement of the IMT of the carotid arteries using the same methodology as was applied in the Spanish patients (see above) and with high-resolution B-mode US (Image Point, Hewlett Packard, and SonoCalc IMT, Sonosite Inc., used in private care and public care, respectively) using linear array 7.5-MHz probes. The details of the methodology used by BAS were reported previously4. The equipment used by AS involves the application of a unique semiautomated border detection program that was previously found to provide highly reproducible results23. The IMT in the left and right common carotid arteries was measured and the cIMT was defined as the mean of these. As in Spanish patients, carotid artery plaque was defined as recommended in the Mannheim consensus22. Both operators were blinded to the CV risk profiles of the patients. Repeat US examinations by both operators on 23 patients revealed Spearman correlations between repeat cIMT measurements of 0.983 and 0.956 for BAS and AS, respectively, and the correlation between measurements made by BAS and AS was 0.926. Both operators identified carotid artery bulb or/and internal carotid artery plaque in 11 of these 23 patients with full agreement.

All assessments were made on the same day in each patient.

Data analysis

Results are expressed as mean (SD), median (interquartile range), or proportions, as appropriate. Because CV risk factor-CVD relations differ by population grouping among African patients with RA19, we analyzed the data separately in black and white patients who were recruited in Johannesburg.

The associations of recorded patient characteristics with plaque were assessed using the Student t test, Mann-Whitney U test, or univariate logistic regression models, as appropriate.

The performance of the SCORE and Framingham score in assessing plaque presence was determined by the area under the curve (AUC) in ROC curve analysis. The AUC [standard errors (SE)] of the CVD risk equations were compared within and between groups by the paired or unpaired Student t test, as appropriate. To determine clinically useful cutoff values in assessing the presence of plaque, we set the sensitivity and specificity of each CVD risk equation at 80% in sequential analyses in Spanish patients. We subsequently applied these obtained cutoff values in African patients when appropriate (see below). We also determined the sensitivity and specificity when the conventional cutoff values of ≥ 5% and ≥ 20% were used for the SCORE and Framingham score, respectively, in assessing plaque presence.

Statistical computations were made using the GB Stat program (Dynamic Microsystems Inc.) and SPSS software, version 21 (SPSS).

RESULTS

Recorded characteristics in Spanish patients with and without carotid plaque

Table 1 shows the recorded characteristics in the 330 Spanish patients with RA and univariate associations of patient characteristics with carotid plaque. Participants with plaque compared to those without were older.

Each of these traditional CVD risk factors was associated with plaque: HTN, systolic blood pressure, total cholesterol and triglyceride concentrations, antihypertensive and lipid-lowering agent use, and poor blood pressure control.

Nontraditional CVD risk factors that were related to plaque included only the erythrocyte sedimentation rate (ESR) and C-reactive protein (CRP) concentrations. The SCORE and Framingham score were each related to plaque. The mean cIMT was increased by 0.126 mm in patients with compared to those without plaque.

ROC curve analysis on the CVD risk equation-carotid plaque relations

Figure 1 gives the ROC curves for the CVD risk equation-carotid plaque relations in all 330 Spanish patients. The AUC (95% CI) of the ROC curves was 0.747 (0.639–0.800) and 0.799 (0.752–0.846) for the SCORE and Framingham score, respectively. In assessing plaque presence, the AUC (SE) of the ROC curve for the Framingham score was larger than that for the SCORE (p = 0.003).

Figure 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1.

Receiver-operator characteristic curves for predicting plaque presence by the Framingham score and SCORE. P values are given for the AUC-plaque relations. SCORE: Systematic COronary Risk Evaluation; AUC: area under the curve.

Table 2 shows that when sensitivity for the Framingham score was set at 80%, the cutoff value was 7.3% with a corresponding specificity, positive predictive value (PPV), negative predictive value (NPV), and correct classification of 63%, 66%, 76%, and 71%, respectively. When specificity was set at 80%, the cutoff value was 10.8% and the corresponding sensitivity, PPV, NPV, and correct classification were 65%, 75%, 70%, and 72%, respectively. Upon using the conventional cutoff value of 20%, the corresponding specificity and PPV increased to 95% and 85%, respectively, but the sensitivity, NPV, and correct classification decreased to 25%, 57%, and 61%, respectively.

View this table:
  • View inline
  • View popup
Table 2.

Classification of Spanish patients with RA at CVD risk equation cutoff values based on sensitivity or specificity set at 80% in ROC analysis, or conventional recommendations.

Also given in Table 2, when sensitivity for the SCORE was set at 80%, the cutoff value was 0.5% with a corresponding specificity, PPV, NPV, and correct classification of 58%, 65%, 76%, and 72%, respectively. When specificity was set at 80%, the cutoff value was 1.5% and the corresponding sensitivity, PPV, NPV, and correct classification were 50%, 67%, 59%, and 64%, respectively. Upon using the conventional cutoff value of 5%, the corresponding specificity and PPV increased to 99% and 86%, respectively, but the sensitivity, NPV, and correct classification decreased to 8%, 52%, and 54%, respectively.

Patients included in the Framingham study were aged 30 to 74 at enrollment. When we repeated the analysis in Figure 1 among Spanish patients with RA within the respective age group (n = 282), the results were materially unaltered (data not shown).

External validation of findings on CVD risk equation-carotid plaque relations

Recorded characteristics in 97 white and 90 black African patients are given in Table 3. Among white African patients, these characteristics were each associated with carotid plaque: age, male sex, HTN, systolic blood pressure, HDL cholesterol concentrations, cholesterol–HDL cholesterol ratio, the use of antihypertensives, and poor blood pressure control. Among nontraditional CVD risk factors, disease duration, extraarticular disease, and CRP concentrations were related to plaque. The SCORE and Framingham score were each related to plaque.

View this table:
  • View inline
  • View popup
Table 3.

Recorded characteristics in 97 white and 90 black African patients with RA. Data were analyzed by Student t test, Mann-Whitney U test, or logistic regression models as appropriate. Results are expressed as mean (SD), median (interquartile range), or percent unless otherwise specified.

Among black African patients with RA, age was associated with plaque. Systolic blood pressure was paradoxically lower in patients with plaque compared to those without plaque. In this regard, body mass index (BMI) was associated with systolic blood pressure (r = 0.348, p = 0.001) and concurrently, numerically lower in black African patients with plaque compared to those without plaque. In an age, sex, BMI, and antihypertensive agent use–adjusted logistic regression model, systolic blood pressure was no longer significantly associated with plaque (OR 0.98, 95% CI 0.95–1.00, p = 0.07). None of the other traditional and nontraditional CVD risk factors were associated with plaque in black African patients with RA. Also, the SCORE and Framingham score were each unrelated to high-risk atherosclerosis in the respective group.

In white African patients with RA, the AUC (95% CI) of the ROC curves were 0.694 (0.584–0.804) and 0.737 (0.694–0.876) for the SCORE and Framingham score, respectively. Within the white African group in identifying plaque presence, the AUC of the ROC curve for the Framingham score was numerically larger by 0.053 than that for the SCORE. This difference was not significant, presumably because of the overall relatively small sample size. Indeed, by comparison, the respective difference was similar at 0.052 in Spanish patients (Figure 1). Among white African patients with RA in identifying plaque presence, the AUC (SE) of the ROC curves for the SCORE and Framingham score did not differ from those in the Spanish patients (p = 0.4 and 0.3, respectively).

In black African patients with RA, the AUC (95% CI) of the ROC curves were small at 0.549 (0.427–0.672) and 0.544 (0.425–0.664) for the SCORE and Framingham score, respectively, and with insignificant corresponding p values (0.4 and 0.5, respectively). Therefore, no further analysis was performed in this group.

In Table 4, the obtained cutoff values in Spanish patients were applied in white Africans with RA. Upon using a cutoff value of 7.3% for the Framingham score (at which sensitivity was 80% in Spanish patients), the corresponding sensitivity, specificity, PPV, NPV, and correct classification were 67%, 67%, 57%, 75%, and 67%, respectively. When a cutoff value of 10.8% (at which specificity was 80% in Spanish patients), the corresponding sensitivity, specificity, PPV, NPV, and correct classification were 44%, 90%, 74%, 71%, and 71%, respectively. For the conventional Framingham score cutoff value of 20%, the corresponding sensitivity, specificity, PPV, NPV, and correct classification were 21%, 100%, 100%, 65%, and 68%, respectively.

View this table:
  • View inline
  • View popup
Table 4.

Classification of white African patients with RA at CVD risk equation cutoff values as determined in ROC analysis in Spanish cases, or conventional recommendations.

Upon using a cutoff value of 0.5% for the SCORE (at which sensitivity was 80% in Spanish patients), the corresponding sensitivity, specificity, PPV, NPV, and correct classification were 72%, 55%, 52%, 75%, and 62%, respectively. When a cutoff value of 1.5% (at which specificity was 80% in Spanish patients), the corresponding sensitivity, specificity, PPV, NPV, and correct classification were 44%, 86%, 69%, 69% and 69%, respectively. None of the white African patients had a SCORE of > 5%.

ROC curve analysis findings on CVD risk equation-large cIMT and -atherosclerosis extent relations, and their validation in African white patients

Among Spanish patients, 79 (23.9%) had a cIMT of ≥ 0.728 mm (upper quartile), which was strongly associated with plaque presence (OR 6.05, 95% CI 3.30–11.06, p < 0.0001). The AUC (95% CI) of the ROC curves were 0.741 (0.682–0.801) and 0.746 (0.684–0.808) for the Framingham score and SCORE, respectively. Supplementary Table 1 (available online at jrheum.org) shows that when sensitivity for the Framingham score was set at 80%, the cutoff value was 7.4 with a corresponding specificity of 51%; when specificity was set at 80%, the cutoff value was 14.1% with a corresponding sensitivity of 53%. Upon using the conventional cutoff value of 20%, the sensitivity was reduced to 33%. As also given in Supplementary Table 1 (available online at jrheum.org), when sensitivity for the SCORE was set at 80%, the cutoff value was 0.5 with a corresponding specificity of 56%; when specificity was set at 80%, the cutoff value was 1.5% with a corresponding sensitivity of 60%. Upon using the conventional cutoff value of 5%, the sensitivity was reduced to 13%.

Carotid plaque was present in 162 (49.1%) of Spanish participants; 58 (35.8%) and 104 (64.2%) of them had unilateral and bilateral plaque, respectively. The AUC (95% CI) of the ROC curves were 0.690 (0.605–0.775) and 0.670 (0.585–0.755) for the Framingham score and SCORE, respectively. Supplementary Table 2 (available online at jrheum.org) shows that when sensitivity for the Framingham score was set at 80%, the cutoff value was 8.8 with a corresponding specificity of 53%; when specificity was set at 80%, the cutoff value was 17.0% with a corresponding sensitivity of 38%. Upon using the conventional cutoff value of 20%, the sensitivity was further reduced to 31%. As also given in Supplementary Table 1 (available online at jrheum.org), when sensitivity for the SCORE was set at 80%, the cutoff value was 0.5 with a corresponding specificity of 26%; when specificity was set at 80%, the cutoff value was 1.5% with a corresponding sensitivity of 53%. Upon using the conventional cutoff value of 5%, the sensitivity was reduced to 12%.

Among African white patients, 36 (37.1%) had a cIMT of ≥ 0.728 mm; of the 39 (40.2%) with plaque, 14 (75.9%) and 25 (64.1%) had unilateral and bilateral plaque, respectively. Upon external validation, application of the cutoff values at which sensitivity was set at 80% in Spanish patients had corresponding specificities of 78% and 50% for a large cIMT and bilateral (versus unilateral) plaque, as given in Supplementary Tables 3 and 4 (available online at jrheum.org), respectively.

Alternative predictive models for plaque

CVD risk factors, including traditional, nontraditional, and disease characteristics that are not included in the Framingham score and SCORE, can also contribute to increased atherogenesis in RA4,5,6,7,8. In this regard, disease duration, DAS28, extraarticular disease, triglyceride and CRP concentrations, and the ESR were related to carotid plaque at p ≤ 0.2 in Spanish patients (Table 1). Notably, BMI was not associated with plaque (p = 0.9; Table 1). When these characteristics were entered in a multivariable logistic regression model, only the ESR remained significantly associated with plaque (OR 1.03, 95% CI 1.00–1.06, p = 0.025). Upon forcing BMI into the latter model, the respective risk factor was also not independently associated with plaque (OR 0.96, 95% CI 0.78–1.01, p = 0.2), whereas the ESR remained related to plaque (OR 1.03, 95% CI 1.00–1.06, p = 0.024). Supplementary Table 5 (available online at jrheum.org) shows the AUC in ROC analysis for the association of ESR, a high ESR, Framingham score, and SCORE, and combinations of the respective predictors with plaque presence in the 330 Spanish patients with RA. The largest AUC was obtained when a Framingham score ≥ 10.8 was entered as a predictor for plaque presence; compared with the AUC for Framingham score ≥ 10.8, these were similar: the AUC for Framingham score ≥ 7.3, SCORE ≥ 0.5, SCORE ≥ 1.5, ESR ≥ 17 mm/h and Framingham score ≥ 7.3, ESR ≥ 17 mm/h and Framingham score ≥ 10.8, ESR ≥ 17 mm/h and SCORE ≥ 0.5, and ESR ≥ 17 mm/h and SCORE ≥ 1.5 (p = 0.9, 0.9, 0.06, 0.8, 0.9, 0.8, and 0.06, respectively), whereas the AUC for ESR and ESR ≥ 17 mm/h were smaller (p = 0.02 and 0.0004, respectively). Taken together, the inclusion of the ESR in addition to the Framingham score or SCORE did not increase the AUC in ROC analysis. The addition of CVD risk equations to ESR did, however, markedly increase the AUC.

In all ROC analyses, upon multiplication of the SCORE and Framingham score as recommended by EULAR3, results were materially unaltered (data not shown).

DISCUSSION

A Framingham score of ≥ 20 and SCORE of ≥ 5, as well as the presence of carotid plaque, are currently considered to represent high CVD risk even in the absence of established CVD, chronic kidney disease, or diabetes9,10. Carotid US can substantially improve CVD risk assessment, but is not routinely performed and is inaccessible to many patients with RA12,14,15,16,17. The main finding in our present study is that upon using the Framingham score and SCORE cutoff values that are markedly low at 7.3% to 10.8% and 0.5% to 1.5%, respectively, a high Framingham score or SCORE can assess the presence of high-risk subclinical atherosclerosis to an extent that appears clinically and fairly useful in RA. By contrast, upon using conventional cutoff values of 20% for Framingham score and 5% for SCORE as recommended in the general population, the corresponding sensitivities were reduced to 25% and 8%, respectively. The latter result further validates previously reported findings4,16,17. Additionally, we found that the low Framingham score and SCORE cutoff values were preferable to their conventional ones in identifying patients with a large cIMT and extensive atherosclerosis, as determined by bilateral compared with unilateral carotid plaque.

The SCORE and Framingham score were each associated with carotid plaque in univariate analysis (p < 0.0001). The Framingham score demonstrated better discrimination between patients with and without plaque, with significantly larger AUC of the ROC curves compared with those for the SCORE in all patients and the respective subgroups (AUC = 0.799 vs 0.747, p = 0.003 for difference). The SCORE estimates the 10-year risk of a first fatal atherosclerotic event10 whereas the Framingham score estimates that of any CVD event11. The demographic characteristics and modifiable CVD risk factors included in the Framingham score and SCORE equations are mostly similar. However, whereas for a given blood pressure, CVD risk remains larger in patients receiving treatment for HTN than in persons without HTN9, this is accounted for in the Framingham score, but not the SCORE. In this regard, uncontrolled HTN was frequent and further strongly associated with plaque in our present study. Whether the Framingham score is preferable to the SCORE in assessing plaque presence should be subject to further study.

Our external validation groups were substantially smaller than those consisting of Spanish patients with RA. Despite this limitation, the main findings in our analysis among Spanish patients were replicated among white Africans with RA. As determined by the AUC of the ROC curve, the performance of both the Framingham score and the SCORE in assessing plaque presence did not differ significantly between Spanish and white Africans with RA. More importantly, upon using the cutoff values for the Framingham score and SCORE as determined in white Spanish patients in ROC analysis among African whites with RA, the corresponding sensitivity, specificity, PPV, and NPV appeared clinically useful, particularly when the lower obtained values of 7.3% and 0.5%, respectively, were applied. The results were similar for large cIMT and atherosclerosis extent.

We previously reported that traditional risk factors and disease characteristics are related to carotid atherosclerosis in white but not black Africans with RA10,24,25. Our current study revealed that neither the Framingham nor the SCORE risk equations discriminated between African black patients with RA with and without plaque. This strongly supports the need for population-specific CV risk stratification in RA, as we recently suggested26. In this regard, we recently reported that eGFR equations are useful in identifying black African patients with RA with carotid artery plaque27. Also, circulating concentrations of adipokines consisting of adiponectin, leptin, resistin, chemerin, and retinol-binding protein 4 were associated with atherosclerosis in patients with RA independent of potential confounders, including population origin28,29,30,31,32.

We assessed the performance of traditional risk factor assessment equations that are widely used and currently recommended in guidelines for CVD prevention in either of the 2 countries where patients were recruited. Interestingly, the 2013 American College of Cardiology/American Heart Association CV risk score, as well as the Framingham score and Reynolds Risk Score, were recently shown to each perform poorly in identifying patients with RA with high CAC scores as a marker of increased CVD risk33. The Reynolds score consists of a model that includes high-sensitivity CRP concentrations in addition to traditional CVD risk factors13. We found that CAC scores are less sensitive than carotid US in detecting high-risk atherosclerosis among patients with RA17.

Given that CVD risk equations were developed to stratify patients for incident CV events rather than prevalent atherosclerosis, our findings require further validation in epidemiological studies. Although many associations were assessed in our present study, the Spanish patients formed a large group. Further, the CVD equations used include a range of CV risk factors, and relationships in ROC analyses were strong and consistent, reproduced in white African patients, and would persist upon setting significance at p < 0.01.

Our present study indicates that a Framingham score of 7.3% to 10.8% and a SCORE of 0.5% to 1.5% comprise measures that can usefully assess plaque presence in patients with RA without established CVD, chronic kidney disease, and/or diabetes. This finding could assist in refining the need for enhanced CVD risk stratification by carotid US12,16,17 or intensified risk management with CV drugs, including statins among patients with RA who have no access to CV imaging. Delineation of effective population-specific CVD risk stratification strategies is required in black African patients with RA.

ONLINE SUPPLEMENT

Supplementary data for this article are available online at jrheum.org.

Footnotes

  • Supported by grants from “Fondo de Investigaciones Sanitarias” PI06/0024, PS09/00748, and PI12/00060 (Spain). Partially supported by RETICS Program, RD08/0075, and RD12/0009/0013 (RIER) from “Instituto de Salud Carlos III” (ISCIII; Spain). Research performed by Patrick Dessein was supported by the South African Medical Research Council and the National Research Foundation.

  • Accepted for publication October 31, 2015.

REFERENCES

  1. 1.↵
    1. Gonzalez-Gay MA,
    2. Gonzalez-Juanatey C,
    3. Martin J
    . Rheumatoid arthritis: a disease associated with accelerated atherogenesis. Semin Arthritis Rheum 2005;35:8–17.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Aviña-Zubieta JA,
    2. Choi HK,
    3. Sadatsafavi M,
    4. Etminan M,
    5. Esdaile JM,
    6. Lacaille D
    . Risk of cardiovascular mortality in patients with rheumatoid arthritis: a meta-analysis of observational studies. Arthritis Rheum 2008;59:1690–7.
    OpenUrlCrossRefPubMed
  3. 3.↵
    1. Peters MJ,
    2. Symmons DP,
    3. McCarey D,
    4. Dijkmans BA,
    5. Nicola P,
    6. Kvien TK,
    7. et al.
    EULAR evidence-based recommendations for cardiovascular risk management in patients with rheumatoid arthritis and other forms of inflammatory arthritis. Ann Rheum Dis 2010;69:325–31.
    OpenUrlAbstract/FREE Full Text
  4. 4.↵
    1. Dessein PH,
    2. Joffe BI,
    3. Veller MG,
    4. Stevens BA,
    5. Tobias M,
    6. Reddi K,
    7. et al.
    Traditional and nontraditional cardiovascular risk factors are associated with atherosclerosis in rheumatoid arthritis. J Rheumatol 2005;32:435–42.
    OpenUrlAbstract/FREE Full Text
  5. 5.↵
    1. del Rincón I,
    2. Freeman GL,
    3. Haas RW,
    4. O’Leary DH,
    5. Escalante A
    . Relative contribution of cardiovascular risk factors and rheumatoid clinical manifestations. Arthritis Rheum 2005;52:3413–23.
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Solomon DH,
    2. Kremer J,
    3. Curtis JR,
    4. Hochberg MC,
    5. Reed G,
    6. Tsao P,
    7. et al.
    Explaining the cardiovascular risk associated with rheumatoid arthritis: traditional risk factors versus markers of rheumatoid arthritis severity. Ann Rheum Dis 2010;69:1920–5.
    OpenUrlAbstract/FREE Full Text
  7. 7.↵
    1. Gonzalez-Gay MA,
    2. Gonzalez-Juanatey C,
    3. Lopez-Diaz MJ,
    4. Piñeiro A,
    5. Garcia-Porrua C,
    6. Miranda-Filloy JA,
    7. et al.
    HLA-DRB1 and persistent chronic inflammation contribute to cardiovascular events and cardiovascular mortality in patients with rheumatoid arthritis. Arthritis Rheum 2007;57:125–32.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Rodríguez-Rodríguez L,
    2. González-Juanatey C,
    3. Palomino-Morales R,
    4. Vázquez-Rodríguez TR,
    5. Miranda-Filloy JA,
    6. Fernández-Gutiérrez B,
    7. et al.
    TNFA −308 (rs1800629) polymorphism is associated with a higher risk of cardiovascular disease in patients with rheumatoid arthritis. Atherosclerosis 2011;216:125–30.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. Greenland P,
    2. Alpert JS,
    3. Beller GA,
    4. Benjamin EJ,
    5. Budoff MJ,
    6. Fayad ZA,
    7. et al;
    8. American College of Cardiology Foundation; American Heart Association
    . 2010 ACCF/AHA guideline for assessment of cardiovascular risk in asymptomatic adults: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2010;56:e50–103.
    OpenUrlCrossRefPubMed
  10. 10.↵
    1. Perk J,
    2. De Backer G,
    3. Gohlke H,
    4. Graham I,
    5. Reiner Z,
    6. Verschuren WM,
    7. et al;
    8. Fifth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice; European Association for Cardiovascular Prevention and Rehabilitation
    . European Guidelines on cardiovascular disease prevention in clinical practice (version 2012): The Fifth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of nine societies and by invited experts). Atherosclerosis 2012;223:1–68.
    OpenUrlCrossRefPubMed
  11. 11.↵
    1. D’Agostino RB Sr,
    2. Vasan RS,
    3. Pencina MJ,
    4. Wolf PA,
    5. Cobain M,
    6. Massaro JM,
    7. et al.
    General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117:743–53.
    OpenUrlAbstract/FREE Full Text
  12. 12.↵
    1. Dessein PH,
    2. Semb AG
    . Could cardiovascular disease risk stratification and management in rheumatoid arthritis be enhanced? Ann Rheum Dis 2013;72:1743–6.
    OpenUrlFREE Full Text
  13. 13.↵
    1. Arts EE,
    2. Popa C,
    3. Den Broeder AA,
    4. Semb AG,
    5. Toms T,
    6. Kitas GD,
    7. et al.
    Performance of four current risk algorithms in predicting cardiovascular events in patients with early rheumatoid arthritis. Ann Rheum Dis 2015;74:668–74.
    OpenUrlAbstract/FREE Full Text
  14. 14.↵
    1. Gonzalez-Juanatey C,
    2. Llorca J,
    3. Martin J,
    4. Gonzalez-Gay MA
    . Carotid intima-media thickness predicts the development of cardiovascular events in patients with rheumatoid arthritis. Semin Arthritis Rheum 2009;38:366–71.
    OpenUrlCrossRefPubMed
  15. 15.↵
    1. Evans MR,
    2. Escalante A,
    3. Battafarano DF,
    4. Freeman GL,
    5. O’Leary DH,
    6. del Rincón I
    . Carotid atherosclerosis predicts incident acute coronary syndromes in rheumatoid arthritis. Arthritis Rheum 2011;63:1211–20.
    OpenUrlCrossRefPubMed
  16. 16.↵
    1. Corrales A,
    2. González-Juanatey C,
    3. Peiró ME,
    4. Blanco R,
    5. Llorca J,
    6. González-Gay MA
    . Carotid ultrasound is useful for the cardiovascular risk stratification of patients with rheumatoid arthritis: results of a population-based study. Ann Rheum Dis 2014;73:722–7.
    OpenUrlAbstract/FREE Full Text
  17. 17.↵
    1. Corrales A,
    2. Parra JA,
    3. González-Juanatey C,
    4. Rueda-Gotor J,
    5. Blanco R,
    6. Llorca J,
    7. et al.
    Cardiovascular risk stratification in rheumatic diseases: carotid ultrasound is more sensitive than Coronary Artery Calcification Score to detect subclinical atherosclerosis in patients with rheumatoid arthritis. Ann Rheum Dis 2013;72:1764–70.
    OpenUrlAbstract/FREE Full Text
  18. 18.↵
    1. Chung CP,
    2. Oeser A,
    3. Avalos I,
    4. Gebretsadik T,
    5. Shintani A,
    6. Raggi P,
    7. et al.
    Utility of the Framingham risk score to predict the presence of coronary atherosclerosis in patients with rheumatoid arthritis. Arthritis Res Ther 2006;8:R186.
    OpenUrlCrossRefPubMed
  19. 19.↵
    1. Solomon A,
    2. Woodiwiss AJ,
    3. Abdool-Carrim AT,
    4. Stevens BA,
    5. Norton GR,
    6. Dessein PH
    . The carotid artery atherosclerosis burden and its relation to cardiovascular risk factors in black and white Africans with established rheumatoid arthritis: a cross-sectional study. J Rheumatol 2012;39:1798–806.
    OpenUrlAbstract/FREE Full Text
  20. 20.↵
    1. Solomon A,
    2. Christian BF,
    3. Norton GR,
    4. Woodiwiss AJ,
    5. Dessein PH
    . Risk factor profiles for atherosclerotic cardiovascular disease in black and other Africans with established rheumatoid arthritis. J Rheumatol 2010;37:953–60.
    OpenUrlAbstract/FREE Full Text
  21. 21.↵
    1. Klug E;
    2. South African Heart Association (S A Heart); Lipid and Atherosclerosis Society of Southern Africa (LASSA)
    . South African dyslipidaemia guideline consensus statement. S Afr Med J 2012;102:178–87.
    OpenUrlPubMed
  22. 22.↵
    1. Touboul PJ,
    2. Hennerici MG,
    3. Meairs S,
    4. Adams H,
    5. Amarenco P,
    6. Bornstein N,
    7. et al.
    Mannheim carotid intima-media thickness consensus (2004–2006). An update on behalf of the Advisory Board of the 3rd and 4th Watching the Risk Symposium, 13th and 15th European Stroke Conferences, Mannheim, Germany, 2004, and Brussels, Belgium, 2006. Cerebrovasc Dis 2007;23:75–80.
    OpenUrlCrossRefPubMed
  23. 23.↵
    1. Gepner AD,
    2. Korcarz CE,
    3. Aeschlimann SE,
    4. LeCaire TJ,
    5. Palta M,
    6. Tzou WS,
    7. et al.
    Validation of a carotid intima-media thickness border detection program for use in an office setting. J Am Soc Echocardiogr 2006;19:223–8.
    OpenUrlCrossRefPubMed
  24. 24.↵
    1. Solomon A,
    2. Norton GR,
    3. Woodiwiss AJ,
    4. Dessein PH
    . Obesity and carotid atherosclerosis in African black and Caucasian women with established rheumatoid arthritis: a cross-sectional study. Arthritis Res Ther 2012;19:R67.
    OpenUrl
  25. 25.↵
    1. Dessein PH,
    2. Norton GR,
    3. Joffe BI,
    4. Abdool-Carrim AT,
    5. Woodiwiss AJ,
    6. Solomon A
    . Metabolic cardiovascular risk burden and atherosclerosis in African black and Caucasian women with rheumatoid arthritis: a cross-sectional study. Clin Exp Rheumatol 2013;31:53–61.
    OpenUrlPubMed
  26. 26.↵
    1. Solomon A,
    2. Tsang L,
    3. Woodiwiss AJ,
    4. Millen AM,
    5. Norton GR,
    6. Dessein PH
    . Cardiovascular disease risk amongst African black patients with rheumatoid arthritis: the need for population specific stratification. Biomed Res Int 2014;2014:826095.
    OpenUrlPubMed
  27. 27.↵
    1. Dessein PH,
    2. Hsu HC,
    3. Tsang L,
    4. Millen AM,
    5. Woodiwiss AJ,
    6. Norton GR,
    7. et al.
    Kidney function, endothelial activation and atherosclerosis in black and white Africans with rheumatoid arthritis. PLoS One 2015;10: e0121693.
    OpenUrlCrossRefPubMed
  28. 28.↵
    1. Dessein PH,
    2. Tsang L,
    3. Solomon A,
    4. Woodiwiss AJ,
    5. Millen AM,
    6. Norton GR
    . Adiponectin and atherosclerosis in rheumatoid arthritis. Mediators Inflamm 2014;2014:358949.
    OpenUrlPubMed
  29. 29.↵
    1. Dessein PH,
    2. Tsang L,
    3. Woodiwiss AJ,
    4. Solomon A
    . Effect of traditional cardiovascular risk factors on the independent relationship of leptin with atherosclerosis in rheumatoid arthritis. J Rheumatol 2014;41:2087–9.
    OpenUrlFREE Full Text
  30. 30.↵
    1. Dessein PH,
    2. Tsang L,
    3. Woodiwiss AJ,
    4. Hsu HC,
    5. Norton GR,
    6. Solomon A
    . Leukocyte count influences the relationship of circulating resistin concentrations with advanced atherosclerosis in rheumatoid arthritis. Clin Exp Rheumatol 2014;32:989–90.
    OpenUrlPubMed
  31. 31.↵
    1. Dessein PH,
    2. Tsang L,
    3. Woodiwiss AJ,
    4. Norton GR,
    5. Solomon A
    . Circulating concentrations of the novel adipokine chemerin are associated with cardiovascular disease risk in rheumatoid arthritis. J Rheumatol 2014;41:1746–54.
    OpenUrlAbstract/FREE Full Text
  32. 32.↵
    1. Dessein PH,
    2. Tsang L,
    3. Norton GR,
    4. Woodiwiss AJ,
    5. Solomon A
    . Retinol binding protein 4 concentrations relate to enhanced atherosclerosis in obese patients with rheumatoid arthritis. PLoS One 2014;9:e92739.
    OpenUrlCrossRefPubMed
  33. 33.↵
    1. Kawai VK,
    2. Chung CP,
    3. Solus JF,
    4. Oeser A,
    5. Raggi P,
    6. Stein CM
    . The ability of the 2013 American College of Cardiology/American Heart Association risk score to identify rheumatoid arthritis patients with high coronary artery calcification scores. Arthritis Rheumatol 2015;67:381–5.
    OpenUrlCrossRefPubMed
PreviousNext
Back to top

In this issue

The Journal of Rheumatology
Vol. 43, Issue 3
1 Mar 2016
  • Table of Contents
  • Table of Contents (PDF)
  • Index by Author
  • Editorial Board (PDF)
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word about The Journal of Rheumatology.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
The Framingham Score and the Systematic Coronary Risk Evaluation at Low Cutoff Values Are Useful Surrogate Markers of High-risk Subclinical Atherosclerosis in Patients with Rheumatoid Arthritis
(Your Name) has forwarded a page to you from The Journal of Rheumatology
(Your Name) thought you would like to see this page from the The Journal of Rheumatology web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
The Framingham Score and the Systematic Coronary Risk Evaluation at Low Cutoff Values Are Useful Surrogate Markers of High-risk Subclinical Atherosclerosis in Patients with Rheumatoid Arthritis
Patrick H. Dessein, Alfonso Corrales, Raquel Lopez-Mejias, Ahmed Solomon, Angela J. Woodiwiss, Javier Llorca, Gavin R. Norton, Fernanda Genre, Ricardo Blanco, Trinitario Pina, Carlos Gonzalez-Juanatey, Linda Tsang, Miguel A. Gonzalez-Gay
The Journal of Rheumatology Mar 2016, 43 (3) 486-494; DOI: 10.3899/jrheum.150510

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero

 Request Permissions

Share
The Framingham Score and the Systematic Coronary Risk Evaluation at Low Cutoff Values Are Useful Surrogate Markers of High-risk Subclinical Atherosclerosis in Patients with Rheumatoid Arthritis
Patrick H. Dessein, Alfonso Corrales, Raquel Lopez-Mejias, Ahmed Solomon, Angela J. Woodiwiss, Javier Llorca, Gavin R. Norton, Fernanda Genre, Ricardo Blanco, Trinitario Pina, Carlos Gonzalez-Juanatey, Linda Tsang, Miguel A. Gonzalez-Gay
The Journal of Rheumatology Mar 2016, 43 (3) 486-494; DOI: 10.3899/jrheum.150510
Reddit logo Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One
Bookmark this article

Jump to section

  • Article
    • Abstract
    • MATERIALS AND METHODS
    • RESULTS
    • DISCUSSION
    • ONLINE SUPPLEMENT
    • Footnotes
    • REFERENCES
  • Figures & Data
  • Supplemental
  • Info & Metrics
  • References
  • PDF
  • eLetters

Keywords

RHEUMATOID ARTHRITIS
CARDIOVASCULAR DISEASE RISK STRATIFICATION
FRAMINGHAM SCORE
SYSTEMATIC CORONARY RISK EVALUATION

Related Articles

Cited By...

More in this TOC Section

  • Tocilizumab in Behçet Disease: A Multicenter Study of 30 Patients
  • The National Prevalence of Clinically Diagnosed Psoriatic Arthritis in Sweden in 2017
  • Biological Sex Inequality in Rheumatology Wait Times During the COVID-19 Pandemic
Show more Article

Similar Articles

Keywords

  • rheumatoid arthritis
  • CARDIOVASCULAR DISEASE RISK STRATIFICATION
  • FRAMINGHAM SCORE
  • SYSTEMATIC CORONARY RISK EVALUATION

Content

  • First Release
  • Current
  • Archives
  • Collections
  • Audiovisual Rheum
  • COVID-19 and Rheumatology

Resources

  • Guide for Authors
  • Submit Manuscript
  • Author Payment
  • Reviewers
  • Advertisers
  • Classified Ads
  • Reprints and Translations
  • Permissions
  • Meetings
  • FAQ
  • Policies

Subscribers

  • Subscription Information
  • Purchase Subscription
  • Your Account
  • Terms and Conditions

More

  • About Us
  • Contact Us
  • My Alerts
  • My Folders
  • Privacy/GDPR Policy
  • RSS Feeds
The Journal of Rheumatology
The content of this site is intended for health care professionals.
Copyright © 2022 by The Journal of Rheumatology Publishing Co. Ltd.
Print ISSN: 0315-162X; Online ISSN: 1499-2752
Powered by HighWire