Assessment of interrater reliability for each CV quality indicator in 48 randomly selected charts. The interrater reliability for 2 of 3 chart reviewers was reported for each QI by calculating a κ score. The interpretation of the score is as follows: below 0.0 = poor, 0.00–0.20 = slight agreement, 0.21–0.40 = fair agreement, 0.41–0.60 = moderate agreement, 0.61–0.80 = substantial agreement, and 0.81–0.99 = almost perfect agreement between reviewers. Values are n (%) unless otherwise specified.
| Quality Indicator | κ | Percent Agreement, n = 48 | Interpretation | ||
|---|---|---|---|---|---|
| 1. Communication of increased CV risk in RA | 0* | 47 (98) | Falsely low because of low performance on the indicator | ||
| 2A. CV risk assessment | 1.0 | 48 (100) | Perfect agreement | ||
| 2B. IF intermediate or high risk according to guidelines, THEN recommended that treatment of risk factors be initiated | 0* | 41 (85) | Falsely low due to low performance on the indicator | ||
| Y1 | Y2 | Y1 | Y2 | ||
| 3A. Smoking status and cessation counseling | 0.28 | 0.59 | 31 (65) | 39 (81) | Fair agreement Y1 and moderate agreement Y2 |
| 3B. IF smoker counseled to quit smoking | 0.88 | 0.57 | 46 (96) | 44 (92) | Almost perfect agreement Y1 and moderate agreement Y2 |
| 4. Screening for HTN | 0.91 | 0.50 | 46 (96) | 38 (79) | Almost perfect agreement Y1 and moderate agreement Y2 |
| 5. Communication to PCP about a documented high blood pressure | 0.70 | 0.70 | 41 (85) | 41 (85) | Substantial agreement |
| 6. Measurement of a lipid profile | 0.91 | 46/48 (96) | Almost perfect agreement | ||
| 7A. Screening for diabetes | 0.84 | 44/48 (92) | Almost perfect agreement | ||
| Y1 | Y2 | Y1 | Y2 | ||
| 7B. Yearly screening for diabetes in patients at intermediate or high risk** | 0.75 | 0.61 | 40 (83) | 38 (79) | Substantial agreement |
| 8. Exercise | 0.49 | 0.33 | 37 (77) | 37 (77) | Moderate agreement Y1 and fair agreement Y2 |
| 9A. BMI screening and lifestyle counseling | 0.85 | 0.63 | 47 (98) | 44 (92) | Almost perfect agreement Y1 and substantial agreement Y2 |
| 9B. IF they are overweight or obese according to national guidelines, THEN they should be counseled to modify their lifestyle | 0.71 | 0.75 | 41 (85) | 42 (88) | Substantial agreement |
| 10. Minimizing corticosteroid usage | 0.79 | 0.57 | 43 (90) | 40 (83) | Substantial agreement Y1 and moderate agreement Y2 |
| 11. Communication about risks/benefits of antiinflammatories in patients at high risk of CV events | 0.51 | 0.69 | 42 (88) | 45 (94) | Moderate agreement Y1 and substantial agreement Y2 |
↵* Because of the way κ scores are calculated, these indicators have a high agreement but a low κ score and are falsely 0.
↵** Patients at high or intermediate risk for diabetes include patients with the following risk factors: family history of type 2 diabetes in a first-degree relative, history of metabolic syndrome, obesity or overweight (BMI ≥ 25 kg/m2), steroid use, history of gestational diabetes or a macrosomic infant, history of impaired fasting glucose (≥ 6.1 mmol/l) or HbA1C ≥ 6.0%, history of HTN (blood pressure ≥ 140/90 mmHg), member of a high-risk population (e.g., Aboriginal, Asian, Hispanic, South Asian, African, Pacific Islanders), or high risk based on validated diabetes risk calculators or high or intermediate CV risk based on CV risk calculators (e.g., Framingham Risk Score). CV: cardiovascular; QI: quality indicator; RA: rheumatoid arthritis; HTN: hypertension; PCP: primary care physician; BMI: body mass index; Y1: Year 1; Y2: Year 2.