Abstract
Objective. Rheumatologists triage referrals to assess those patients who may benefit from early intervention. We describe a referral tool and formally evaluate its sensitivity for urgent and early inflammatory arthritis (EIA) referrals.
Methods. All referrals received on a standardized referral tool were reviewed by a rheumatologist and, based on the information conferred, assigned a triage grade using a previously described triage system. Each referral was also dichotomized as suspected EIA or not. After the initial rheumatologic assessment, the diagnosis was recorded and a consultation grade, blinded to referral grade, was assigned to each case. Agreement between referral and consultation grades was assessed. A regression analysis was performed to determine factors that predicted truly urgent referrals including EIA.
Results. We evaluated 696 referrals. A total of 210 (30.2%) were categorized as urgent at the time of consultation. The referral tool was able to successfully detect 169 of these referrals (sensitivity 80.5%, specificity 79.4%). EIA occurred in 95 (13.6%); of those referrals, 86 were correctly classified as urgent at the time of triage (sensitivity 90.5%, specificity 69.6%). Items that helped correctly discriminate urgent or EIA referrals included patient age < 60, duration of disease, morning stiffness, patient-reported joint swelling, a personal or family history of psoriasis, urgency as rated by referring physician, prior assessment by a rheumatologist, elevated C-reactive protein, and a positive rheumatoid factor.
Conclusion. A 1-page referral tool that includes parts completed by the referring physician and patient has good sensitivity to detect urgent referrals including EIA.
The prevalence of musculoskeletal pain in the general population has been estimated at more than 20%1, and at least 10% of visits to family physicians are attributable to rheumatologic disorders2. Primary care physicians, however, frequently express low levels of confidence in their ability to diagnose and manage such disorders3. Given the burden of musculoskeletal disease, as well as the increasing wait times for rheumatologist consultation, various practice management and referral triage strategies have been developed4,5,6. These programs have focused on changes to appointment scheduling, the development of specialized care streams, or preappointment screening of referrals for appropriateness. The success of these programs, however, requires the accurate transfer of clinical information between the primary caregiver and rheumatologist.
The importance of triage strategies for rheumatologic referrals is highlighted by the now well-accepted concept that rheumatoid arthritis (RA) in its early stages is a relative emergency. There is good evidence that early introduction of disease-modifying agents is associated with improved clinical and radiographic outcomes7,8,9. Further, there is growing evidence that a brief delay in therapy can affect disability, development of erosions, and the achievement of remission7,10,11. Triage strategies focusing on identifying potential cases of early inflammatory arthritis (EIA), and examining them in a timely manner, have the potential to significantly improve patient outcomes. In general, screening tools should have high sensitivity.
A previous study found unstructured rheumatology referral letters lacked basic details of an inflammatory history, physical examination, and laboratory evaluation12. Moreover, this paucity of referral information led to inappropriate patient triage. Based on the information provided in the unstructured referring letter, triage of these referrals found the sensitivity for detecting urgent referrals at 59%. This is unacceptably low for the accurate and reliable triage of referrals. Further evidence for a lack of information in rheumatology and orthopedic referrals can be found in the United Kingdom where critical information, including duration of symptoms, level of function, examination and laboratory findings, and presumptive diagnosis, were absent in more than 50% of cases13. An analysis of referral letters to an outpatient rheumatology clinic in Norway yielded slightly more optimistic findings, where 95% of referrals were said to outline the clinical problem appropriately, and 76% of referrals included a physical examination14. Similarly, good agreement was found in a UK study between the “paper” priority, based on the grade provided by the consultant rheumatologist upon review of physician referral letters, and the “clinical” priority made after clinical assessment15.
Moreover, there remain numerous anecdotal reports from rheumatologists who describe inaccurate referral information, resulting in inappropriate triage of patients. Many countries have too few rheumatologists, so triaging is paramount to seeing the right patient at the right time.
The development of a standardized rheumatologic referral tool for family physicians provides an opportunity to educate at the primary care level, as well as improve triage, treatment, and outcomes for patients with early RA. While other triage and referral tools exist, some contain inadequate referral information, or they may be costly or take too long to complete. The Comprehensive Arthritis Referral Tool (CART) is a simple, single-page referral tool that includes relevant questions for both the patient and referring physician. It seeks to streamline the referral and triage process by helping physicians ask the proper questions and provide the most effective information to the rheumatologist. The goals of our study were to describe a prototype referral tool for rheumatology and to formally evaluate this tool for its ability to discriminate urgent versus non-urgent rheumatology referrals, including EIA.
MATERIALS AND METHODS
The institutional review board of Western University approved this research. Our study involved 5 outpatient rheumatology practices located at St. Joseph’s Hospital, London, Ontario, Canada, affiliated with Western University, Ontario, Canada. Collectively, these rheumatologists receive about 120 new referrals per month and provide service to a referral population of about 1.3 million. These rheumatologists work in the same center, improving the feasibility of our study.
CART (Appendix 1) was developed through surveys of practicing rheumatologists and focus groups with family physicians. This tool was developed with the intention of differentiating inflammatory, degenerative, and chronic pain conditions, with the aim of identifying urgent rheumatologic referrals. Copies of the referral tool (as a tear-off pad) were mailed to all physicians who had referred to the rheumatology center within the last 2 years along with all local emergency departments, urgent care centers, and specialists’ offices.
All patient referrals received on the referral tool between September 2007 and August 2008 were evaluated. All referrals were initially triaged for urgency by a consultant rheumatologist (AT). If a referral form were missing more than 25% of the information, it was excluded from our study. At the time a referral was received, triage criteria12 were used to assign each case a triage grade (TG) between A+ and C. A copy of the triage criteria can be found in Appendix 2. The referrals were then further categorized into urgent (rating A+/A) versus non-urgent (rating B/C). Upon receipt of each referral, the consultant rheumatologist reviewed the referral information, made a broad presumptive diagnosis, and assigned an initial TG as described above.
All referrals were left to the individual rheumatologist to arrange an initial rheumatologic consultation consisting of patient history, as well as a general and musculoskeletal physical examination. After the initial rheumatologic consultation, each patient was given a presumptive diagnosis and assigned a consultation grade (CG) by the consultant rheumatologist (AT) based upon the information gathered at the time of consultation (posthoc). This CG was based on the factors present at the time of rheumatologic assessment. While the same rheumatologist assigned each of the grades (triage and consult), he was blinded to the referral’s initial grade at the time he was assigning the CG. The CG was assumed to be the most accurate reflection of urgency, and agreement between the TG and CG was assessed.
Outcome measures
The primary outcome measures for our study were the sensitivity and specificity of the referral tool to screen for those cases that were deemed truly urgent and for those cases classified as EIA, defined as an inflammatory arthritis of less than 12 months’ duration. The diagnosis of inflammatory arthritis was left to the consulting rheumatologist.
To identify variables associated with truly urgent referrals or confirmed EIA, univariate logistic regression analyses were performed for each of the 20 variables comprising the CART referral tool using urgent/not urgent and EIA/not EIA as the dependent variables for each regression analysis. Multivariate regression analysis was then performed only on those variables that were significant in the univariate analyses. OR and 95% CI were calculated for each variable. A p value of < 0.05 was considered statistically significant.
RESULTS
Referrals
A total of 895 referrals were reviewed and 135 referrals were excluded (Figure 1). Of those excluded, 38 referrals did not receive appointments for a number of reasons, including appointment was deemed to be no longer required, patient had been given an appointment with another rheumatologist, or referral was deemed more appropriate for another clinic (i.e., orthopedics, pain clinic, etc.). Among the 760 referrals that were evaluated, 64 referrals contained less than 25% of the information on the triage tool and were excluded. That left 696 referrals (77%) for evaluation.
Participant disposition of patient referrals received using the Comprehensive Arthritis Referral Tool (CART) between September 2007 and August 2008.
Baseline demographics
The baseline demographics as reported on the referral form are listed in Table 1.
Baseline demographics based on grade at consultation (%).
Sensitivity for detecting urgent referrals
Table 2 displays the distribution of grades assigned. A total of 210 referrals (30.2%) were deemed urgent at the time of consultation. Of those, 169 were correctly identified at triage, resulting in a sensitivity of 80.5% for detecting urgent referrals. There were 486 referrals (69.8%) scored as non-urgent at consultation. Of those, 386 were correctly identified as non-urgent at triage resulting in a specificity of 79.4%.
Urgent versus non-urgent referrals at triage versus consultation.
Of the 210 referrals found to be urgent at consultation, only 169 were correctly identified at triage (41 missed). Reviewing the data reveals that these referrals were inappropriately triaged for 3 reasons: (1) patient had known inflammatory rheumatologic disease that was flaring; (2) the referring physician thought the referral was non-urgent; and (3) important laboratory data were missing.
Sensitivity for detecting EIA
Table 3 displays the distribution of grades assigned. By definition, EIA was graded as urgent at triage. A total of 95 referrals (13.6%) were classified as EIA at the time of consultation. Of those, 86 were correctly classified as urgent at the time of triage, resulting in a sensitivity of 90.5% for detecting EIA referrals. There were 601 referrals (86.4%) scored as not EIA at consultation. Of those, 418 were correctly identified as non-urgent at triage resulting in a specificity of 69.6%.
Triage urgency grading versus consultation EIA grading.
Of the 9 EIA referrals missed, 7 were not correctly identified at triage because the information in the referral stated the duration of illness as > 12 months. The remaining 2 EIA referrals were overlooked because pertinent laboratory details [elevated rheumatoid factor (RF)] were missing and 1 was misidentified as a degenerative problem.
Logistic regression analysis to determine predictors of urgent versus non-urgent referrals at consultation
Twenty independent variables were included in the logistic regression model.
The variables that predicted an urgent referral are found in Table 4. Those variables that did not predict an urgent consult included sex, age, morning stiffness, physician-reported joint swelling, difficulty with activities of daily living, stopping work, physician presumed diagnosis of inflammatory versus noninflammatory, low hemoglobin, elevated white blood cell count, thrombocytosis, elevated erythrocyte sedimentation rate (ESR), or positive antinuclear antibodies (ANA).
Predictors of urgent versus non-urgent referrals.
Logistic regression analysis to determine predictors of EIA
Among the 20 independent variables included in the logistic regression model, those that predicted an urgent referral are found in Table 5. Those variables that did not predict EIA included sex, physician-reported joint swelling, family history of RA, personal or family history of psoriasis, difficulty with activities of daily living, stopping work, low hemoglobin, elevated white blood cell count, thrombocytosis, elevated ESR, elevated C-reactive protein (CRP), or positive ANA.
Predictors of early inflammatory arthritis.
DISCUSSION
The purpose of our study was to evaluate the ability of a single-page rheumatology referral tool (Appendix 1) to assist in the identification of urgent and EIA referrals. By definition, EIA is urgent; however, not all urgent referrals are EIA.
Detection of urgent referrals
When used to detect urgent referrals, the sensitivity of this tool was 80.5%. This is significantly higher than our previous study that found a sensitivity of 59% when using only unstructured referring physicians’ referral notes12. The reason for the improved sensitivity was a significant increase in referral information on which to base the TG. Other methods such as a priority referral score (PRS) have been evaluated for rating the level of urgency of rheumatology referrals and have shown improvements with standardizing information and guiding referrals16. This tool rates urgency on a linear scale (range 0–100) using 8 criteria that are defined and contain sublevels. While the PRS showed acceptable reliability when tested, its accuracy was not measured in clinical practice. Further, the PRS would require referring physicians to spend time reviewing the definitions to ensure accuracy of scoring. The tool is also not disease-specific. The CART tool is a single-page, easy to use, referral form. In addition to physician input, it allows for patient input. As such, it contains clear, concise questions that do not require definition, and it also includes a diagram where the patient can indicate areas of pain and stiffness. The CART tool also differentiates between inflammatory, degenerative, and chronic pain conditions, thereby improving triage of urgent cases such as EIA.
In a systematic literature review evaluating early referral strategies for inflammatory arthritis, the combination of triage systems and referral forms was shown to increase referrals and reduce delay, whereas triage or referral forms alone only improved identification17. Similar to the combination of the CART tool and triage strategy, a centralized referral database and triage system for rheumatology was found to accurately categorize referrals, including urgent referrals, in 90% of cases18. This system focused on centralizing all referrals using a common form screened by a specialist nurse, the assignment of a triage category, and referral to a rheumatologist. The quality of referral was rated as moderate or high in 91% of referrals.
Previous studies have found that unstructured rheumatology referrals lack important information other than joint pain12,19. For example, any mention of joint swelling was absent from 64% of referrals in a previous study using only physicians’ notes12. In our study, joint swelling (either physician- or patient-reported) was present in 100% of referrals. Further, the presence of patient-reported joint swelling was found to be a predictor of an urgent referral (OR 1.77, p = 0.021). In a study of diagnostic triage and referrals, joint stiffness and swelling were the key clinical features resulting in physicians assessing EIA20.
How do we improve the detection of urgent referrals? As reported, there were 41 missed urgent referrals at the time of triage. The modifiable causes of this misclassification include missing laboratory data and unrecognized diagnoses of inflammatory disease that was flaring. Moving forward, an additional item could be added to the CART, questioning the presence of known rheumatic disease. The necessity to include appropriate laboratory information should also be further emphasized. While waiting for laboratory results could result in a delay in the referral process, depending on the area of practice and resources available, in many cases the additional information could assist in more accurate triage.
Detection of EIA
When used to detect EIA referrals, the sensitivity of this tool was 90.5%. The sensitivity of CART was similar to a triage system that correctly identified patients with inflammatory arthritis with 91% accuracy21. This system was based on the consulting rheumatologist’s evaluation alone. Evaluations were made from physician letters and reports, and an urgency category was assigned. That study did not use a standardized referral form or scoring system. While the accuracy of diagnosis was high, only 55% of patients newly diagnosed with inflammatory arthritis were appropriately triaged as “soon” and 37% were incorrectly triaged as “routine.” The authors acknowledged that the addition of specific diagnostic criteria could further improve diagnosis of EIA using the triage system21. The CART tool was accurate in diagnosing EIA, and showed high sensitivity in detecting urgent referrals. It also allows physicians to assess specific diagnosis criteria and differentiate inflammatory conditions. In addition, our analysis of the CART tool included an evaluation of the independent variables that were the strongest predictors of EIA, such as joint swelling or duration < 12 months, as well as the urgency of referrals.
Out of a total of 95 referrals classified as EIA at consultation, only 9 were missed at referral. The challenge with detecting EIA is that many patients have other preexisting musculoskeletal conditions. Therefore, patients may report longer disease duration believing that the de novo development of EIA is actually an extension of a preexisting musculoskeletal condition. This is difficult to modify and accounted for 7 of the 9 missed EIA cases.
The absence of critical laboratory data was important in the misclassification of 2 cases of EIA. As in the case for urgent referrals, the necessity to include appropriate laboratory information could be further emphasized on the CART.
Examining independent variables
Multivariate logistic regression analysis is useful in identifying variables that are helpful in forming a prediction that a referral may be urgent or EIA or both. The process of triaging referrals is based on experience and pattern recognition; this depends on the perception of interrelationships between separate observations. For example, a family history of psoriasis in a young patient with no swollen joints and a negative RF is less effective than a family history of psoriasis in a young patient with 2 swollen knees and a negative RF. Gormley, et al concluded that history and examination are critical for EIA assessment and should be emphasized20. The CART tool takes these clinical factors into account.
During the initial development of the CART, a new significant functional decline (i.e., loss of work or significant difficulties with activities of daily living) was identified as being potentially helpful for detecting urgent referrals. Similarly, the previously described PRS also identified self-reliance and role/work as potentially important predictors of urgency16. Another observational study of a patient-directed and doctor-directed prereferral questionnaire for an early arthritis clinic included a question about physical function and work22. A decline in function, whether it is secondary to difficulty with activities of daily living or loss of work, was not predictive of an urgent referral or EIA in our analysis. The reality is patients stop working and encounter functional decline for a litany of reasons. A functional decline is not necessarily reflective of the urgency of a medical problem.
Morning stiffness lasting more than 60 min is a feature that is commonly sought to support a diagnosis and is used as an outcome measure for inflammatory arthritis. The 1987 classification criteria for RA included the presence of more than 60 min of morning stiffness23. Moreover, a question about morning stiffness is present in all current preappointment rheumatology questionnaires, including the one used for this analysis20,22,24. In our study, patient-reported morning stiffness lasting more than 60 min was a very weak predictor of EIA and had no value in determining an urgent referral. Prolonged morning stiffness is commonly reported in many noninflammatory rheumatic conditions, including fibromyalgia (FM) and even osteoarthritis (OA). This is supported by a previous study where morning stiffness was not found to be predictive of a diagnosis of early RA22. The revised 2010 classification criteria for RA do not include morning stiffness as a variable25.
A strong family history of RA was more suggestive of a non-urgent referral than an urgent one. Because there is a genetic predisposition for RA, this finding was not anticipated. The presence of real or apparent “rheumatoid disease in the family” likely has an effect on whether a patient seeks help from a primary care physician or the primary care physician seeks help from a rheumatologist. This likely far outweighs any true genetic influence. Counterintuitively, the presence of a family history of RA is a better predictor of noninflammatory disease. This finding has been supported in previous research26.
A personal or family history of psoriasis was a very weak risk factor for an urgent referral and played little value in assisting in the discernment of EIA. As mentioned, the presence of psoriasis is contextually useful in situations where the clinical presentation is consistent with an inflammatory arthritis.
Duration less than 12 months was the strongest predictor of EIA, but it was not perfect. Although duration is important for EIA, it must not be overlooked when considering other potentially urgent referrals (i.e., flaring in EIA).
The CART was developed to detect urgent rheumatology referrals and EIA. A central theme in these referrals is the presence of arthritis defined as inflamed joints (warm, swollen, tender joints that may or may not be erythematous). Therefore, to accurately triage patients with arthritis, it is important to determine if there is, indeed, true joint swelling. This concept is supported through the inclusion of questions about joint swelling in all currently published referral tools16,22,24. The next questions are (1) who should be asked about the presence of joint swelling; and (2) how useful is this information in triage?
A systematic review and metaanalysis of patient-reported joint counts found self-reported tender joint counts to have moderate to marked correlation with those performed by a trained assessor27. In contrast, swollen joint counts demonstrated lower levels of correlation. No studies were identified specifically examining the ability of a primary care physician to detect swollen joints.
Interestingly, patient-reported joint swelling was one of the strongest predictors of EIA and a significant but weaker predictor of an urgent referral. Referring physician-reported joint swelling was not a statistically significant predictor of urgency. This raises several questions, including (1) did the referring physician actually examine the joints? (2) If yes, how confident were they in their findings? A previous study examining early RA referrals suggests deficits and uncertainties among referring physicians, especially in interpreting patient history and clinical findings20. Gormley, et al reported, “Following our data, provided information about joint swelling of the hands by unskilled doctors must be handled with caution.” What is clear from our data is that a patient who reports no joint swelling or is unsure about joint swelling is unlikely to have EIA.
The pattern of joint involvement is also important. The CART included a pain diagram with enlarged hands and feet. Scoring this diagram for inclusion into the logistic regression analysis was too difficult as a result of the sheer number of permutations and combinations. Previous research concluded that pain diagram patterns may help increase the likelihood of various rheumatic diagnoses including polyarticular pattern and inflammatory arthritis28. Pain diagrams require further study to better understand their utility.
Although the probability of a referral being urgent is significantly increased when the referring physician reports it as being “urgent,” there are many pitfalls when relying on this data. In a study examining time to treatment in RA, 1 of the only factors influencing triage allocation was a physician assessment of urgency: swollen joints and an elevated CRP were the 2 factors that resulted in this urgency rating by physicians29. Asking a referring physician to determine the “perceived urgency” of a referral is unfair. It is clear that referring physicians have difficulty discerning what is meant by the term “urgent.” This was apparent in the fact that 61% of truly urgent referrals at consultation were not thought to be urgent by the referring physician as reported on the CART at the time of referral. Perhaps a time frame for the initial consultation (i.e., within 2 weeks, 2 to 4 weeks, 2 to 4 mos, etc.) rather than an urgency rating would be more useful for the referring physician.
It is important to try to educate referring physicians regarding the conceptual approach a rheumatologist uses in the diagnosis of rheumatic disease. Rheumatologists work by categorizing disease into patterns of inflammatory, degenerative, and chronic pain. The CART tool includes a section for the referring physician to also categorize in this manner.
Our data shows that referring physicians still struggle to understand what is meant by the terms “inflammatory” and “noninflammatory.” To support this point, 50% of the referrals that were truly noninflammatory (OA, back pain, arthralgia, etc.) were actually reported as inflammatory by the referring physician. Referring physicians did somewhat better with truly inflammatory diseases, reporting 74% correctly as inflammatory on the CART. However, it is clear that more education is needed to improve the conceptual approach to arthritis by primary care physicians. For example, a past workshop on musculoskeletal conditions for physicians in Canada resulted in positive changes in arthritis management for attendees and increased their level of knowledge30.
From a laboratory perspective, an elevated CRP was a predictor of an urgent referral, but an ESR was not. Persistent elevations of acute-phase reactants are likely of more use than single isolated values; however, this is impractical. A positive RF was a predictor of an urgent referral and EIA. A positive ANA was not a predictor of an urgent referral or EIA. Admittedly, we did not stratify the ANA into titers.
Many autoimmune diseases including EIA occur most often in people aged 30 to 60 years. One might expect that patients in this age range would more likely be urgent or EIA. Indeed, this was the case as those older than 60 years were less likely to have a diagnosis of EIA. However, age cannot be an isolated determinant because 27 of 95 confirmed cases (28%) of EIA were older than 60 years.
Inflammatory and autoimmune disease is considered more common in women. However, female sex was not a predictor of urgency or EIA. A possible explanation is that noninflammatory diseases (i.e., FM, OA), which are also more common in women, balance out the effect of sex. Secondly, women are more likely to seek medical attention for health-related complaints than men31.
Study strengths and limitations
Our study has a number of limitations. First and foremost was the difficulty in using the pain diagram in the multivariate analysis. The vast number of combinations and permutations of scoring the pain diagram made this challenging. In future, the concordance or discordance of the pattern of pain with the presumed diagnosis may be explored. Admittedly, the grading of the referrals at triage and at consultation was performed by the same physician, and this may have introduced bias into our study. However, potential bias was minimized by blinding the consultant rheumatologist to the triage scores at the time of grading consult scores. In addition, we have highlighted only a small snapshot of dysfunction within our current model of care because the current study included only those referrals that included a completed referral tool at a single center. Finally, we did not explore the referrals (n = 64; 8%) that were not included in the analysis because of a lack of data, nor the reasons why > 25% of information was missing (e.g., too much time to complete the referral tool, information not readily available, etc.). Understanding the reasons why these forms were not adequately completed, as well as the outcomes of these patients, could be informative to assess the feasibility of the longterm acceptance and use of CART by referring physicians.
The strengths of our study included a population from a large referral area with an emphasis on EIA. A heterogeneous population of patients was included with multiple diagnoses. Multiple referring physicians were involved, some of whom gave input on the development of the referral tool.
Moving forward: revision of the referral tool
Our study has highlighted the importance of the transfer of appropriate information from the referring physician or healthcare specialist. However, not all of the information on the referral tool was useful in screening for urgent or early inflammatory disease. Items such as new difficulty with activities of daily living, energy levels, or even a family history of RA should have a lower level of importance. In contrast, patient-reported joint swelling, known diagnosis of an inflammatory condition, and laboratory investigations should be further emphasized.
Our study has highlighted the importance of the transfer of appropriate information from the referring physician. The sensitivity to detect urgent referrals increased from 59% (in the past using the referring physicians’ letters) to 81% using this referral tool. Further, the sensitivity to detect EIA was 91% using this referral tool. Future refinements to the referral tool are planned along with a multicenter study to evaluate the generalizability of a refined tool.
Acknowledgment
We acknowledge the entire Division of Rheumatology at Western University and our support staff, who made this study possible.
APPENDIX 1.
The Comprehensive Arthritis Referral Tool (CART), a single-page rheumatology referral questionnaire.
APPENDIX 2.
Triage grading system.
Footnotes
-
Funded through a peer-reviewed grant awarded by the Canadian Investigators for Outcomes in Rheumatoid Arthritis (CIORA).
- Accepted for publication June 17, 2014.
REFERENCES
In this issue
