Abstract
Objective. The purpose of the 2007 Osteoarthritis Measurement in Routine Rheumatology Outpatient Practice survey was to describe practice styles, instrument usage, and perceptions of responder criteria and state-attainment criteria in osteoarthritis (OA) management in routine clinical rheumatology practice.
Methods. A 16-item questionnaire (65 subcomponents) was developed, pretested, revised, formatted, and mailed to rheumatologists residing in Australia. Responses were obtained from 136 rheumatologists (response rate 58%).
Results. Approximately half the Australian respondents did not follow up their patients with hip and knee OA and two-thirds did not follow up their patients with hand OA. Health status measures (HSM) were infrequently used, even by those respondents who followed their patients with OA longitudinally, and the scores from those HSM that were used, were rarely if ever formally recorded. Respondents rated the following 6 requirements of a measure for use in clinical practice as very important: validity, reliability, responsiveness, simplicity, quick completion, and easy scoring. One-fifth to one-quarter of respondents indicated they did not know quantitatively what constituted a clinically important improvement, or a health state acceptable to patients with OA. The majority of the remainder selected values not closely aligned with published values in the peer review literature.
Conclusion. While simply describing the health status of the patient is interesting, the more strategic applications are in benchmarking, and using the data to inform shared decision-making and therapeutic goal-setting. The OMIRROP survey suggests that further investigation of interpretation issues are essential, before evaluating the role of quantitative measurement in routine OA clinical practice.
In the last 25 years, there has been steady progress in the development of measurement techniques1 for osteoarthritis (OA), and agreement on core set measures2–4. Recently, attention has focused on the elaboration of definitions of response4–13 and state-attainment13–17 for patients with hip and knee OA. The concepts of standardized measurement in OA using valid, reliable, and responsive instruments has been recognized in US Food and Drug Administration18, the European Agency for the Evaluation of Medicinal Products19, Osteoarthritis Research Society International (OARSI)3,4, IMMPACT20,21, and OMERACT2 guidelines. These guidelines principally concern research-based applications.
In contrast, there have been no recommendations specifically regarding outcome measures for routine use in the clinical care of patients with OA. Emerging definitions of individual patient response include the OMERACT-OARSI Responder Criteria7, Minimum Perceptible Clinical Improvement (MPCI)6, Minimum Clinically Important Improvement (MCII75)9,12, and Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC 20–50–70)8. Comparable developments in defining the attainment of a satisfactory state of health include the Patient Acceptable Symptom State (PASS75)14,17, and the Bellamy et al Low Intensity Symptom State-attainment (BLISS) Index13,15,16. While these emerging definitions are more recent, their publication in the peer-review literature preceded the conduct of this Osteoarthritis Measurement in Routine Rheumatology Outpatient Practice (OMIRROP) survey by 2 to 7 years. All the aforementioned response and state-attainment criteria/proposals in the rheumatology OA clinical trials literature are based in part or in whole on the WOMAC22 and Australian/Canadian OA Hand Index (AUSCAN)23 indices.
Three previous OMIRROP surveys24–26 have been conducted, one in Canada24 and 2 in Australia25,26. Those surveys indicated very low levels of uptake, in routine clinical practice, of the use of patient reported outcomes, based on standardized health status measures. It was concluded that measurement developments in OA clinical research had not found implementation in routine clinical practice. The first 2 OMIRROP surveys predated the emergence of response and state-attainment criteria, and none of the previous OMIRROP surveys asked questions regarding response or state-attainment.
The objectives of the 2007 OMIRROP survey were (1) to describe current OA followup practices; (2) to ascertain the required characteristics of instruments suitable for use in clinical practice; (3) to gauge the extent to which several well known generic, general arthritis, and OA-specific HSM are currently being used in the clinical care setting; and (4) to ascertain the perception of rheumatologists regarding response and state-attainment, given the aforementioned publications in this area.
MATERIALS AND METHODS
A 16-item (65-subcomponent) questionnaire was developed, pretested with 5 rheumatologists, then revised, formatted, and mailed by Australia Post, with the assistance of the Australian Rheumatology Association (ARA), to rheumatologists residing in Australia. The sample was ascertained from the 2007 ARA directory. We excluded any ARA registrants who identified themselves as pediatric rheumatologists, or who had previously informed the ARA they did not wish to participate in surveys. In order to maintain respondent anonymity, questionnaires were coded and sent to the ARA for the initial mailing. The ARA matched the coded questionnaires to their membership list. A letter of introduction, with an accompanying note of invitation from the ARA, was sent to 236 eligible practising rheumatologists. The first-round questionnaires were sent a week later. Second and third mailings of the OMIRROP questionnaire were sent, by the ARA, to nonrespondents at intervals of about 1 month, with a personal letter accompanying each mailing to maximize the response rate. For the second and third mailings, the ARA was sent a list of the codes for both respondents and nonrespondents. ARA personnel then performed each subsequent mailing based on this information. Using this methodology, the identity of the respondents and nonrespondents was unknown to the investigators, and the individual responses of members were unknown to the ARA.
Participants were questioned in the OMIRROP survey regarding their measurement practices in the longitudinal followup (serial assessments over time) of their adult outpatients with OA of hand, hip, and knee. For the purpose of the survey, an outpatient was defined as a nonhospitalized (i.e., ambulatory) patient, seen either in private clinical practice or in the outpatient clinic, of a healthcare facility. Although a large number of outcome measures are currently available, the OMIRROP survey focused only on the use of Health Status Measurement (HSM) tools used in OA. Participants were also questioned regarding their perception of the relative and absolute improvement (minimum clinically important improvement; MCII) that they regarded as clinically important, and the clinical state attained with treatment that they regarded as satisfactory (patient acceptable symptom severity; PASS).
Following receipt of questionnaires, data were entered into Excel and data quality was assured for out-of-range values and transcription errors. The data were analyzed using Statistica software (version 8; StatSoft Inc., Tulsa, OK, USA).
RESULTS
Response data
Responses were obtained from 136 rheumatologists (response rate 58%; men 67%, women 33%) from each Australian state and from the Australian Capital Territory. In answer to the question regarding type of practice, the following responses were obtained: full-time hospital 29%; visiting medical officer (VMO), teaching hospital 49%; VMO, non-teaching hospital 7%; private practice 80%. The majority of the respondents engaged in more than one type of practice. The mean year of graduation from medical school of respondents was 1982 (range 1963–1998) and the mean year of starting practice in rheumatology (based on year of FRACP qualification in rheumatology) of respondents was 1992 (range 1973–2007). The majority of respondents (77%) had experience participating in at least one prior clinical research project, in which they had been required to make or supervise clinical measurements on study subjects. Respondents were more likely to longitudinally follow patients with knee OA (53%) and hip OA (48%) than patients with hand OA (34%).
Basic measurement procedures
To assess use of basic measurement procedures, participants were asked firstly to respond to the question, “How often do you serially use health status assessment techniques for longitudinally monitoring the efficacy of anti-rheumatic drug therapy in your adult patients with Osteoarthritis?”. Only 4% responded in the “usually” (4%) or “always” (1%) categories, the remaining 95% responding either “never” (74%) or “occasionally” (21%).
Health status measures
No major health status instrument evaluated was routinely used in clinical practice in patients with OA. Only 21% (n = 29) of respondents reported using any health status measures (HSM) routinely in clinical practice. The Health Assessment Questionnaire (HAQ; n = 18) and the WOMAC (n = 15) were the 2 most frequently used HSM. All other instruments, that is, the Arthritis Impact Measurement Scales (AIMS), AIMS2, Lequesne Index (Hip and Knee), KOOS Index, HOOS Index, AUSCAN Hand OA Index, Functional Index for Hand OA (Dreiser Index), Cochin Hand Index, Health Utilities Index (HUI), Nottingham Health Profile (NHP), Short Form-36 (SF-36), European Quality of Life Index (EuroQol), and the McGill Pain Questionnaire, were either not used at all, or were used by ≤ 4 respondents.
Data recording
Sixty-three respondents provided details of where they recorded the actual scores for the HSM above. The following locations were identified: written notes in the patient’s chart (27%), dictated in the patient’s charts (9%), recorded on rough notes (6%), recorded on a flow sheet (3%), or other (3%). Fifty-one percent of respondents reported not recording the scores anywhere.
Characteristics of a measure for use in adult outpatient practice
Respondents were asked to rate the importance of 6 characteristics (simplicity, quick completion, easy scoring, reliability, validity, and responsiveness) relevant to the use of measurement techniques in routine clinical practice, according to the following scale: “extremely important,” “very important,” “moderately important,” “somewhat important,” and “not important at all.” The majority of respondents identified each of the 6 characteristics as being “extremely important” (57%–71%) or “very important” (23%–33%). The remaining respondents rated the same 6 characteristics as “moderately important” (4%–10%) or “somewhat important” (1%–4%). No respondent rated any of the 6 characteristics as “not important at all.”
Ranking preference for commonly used scaling methods
Use of the Likert-type scale (i.e., descriptive, adjectival) was the first rank preference for 50% (n = 65) of respondents. A smaller percentage (35%, n = 45) preferred the visual analog scale for measuring symptom severity in routine clinical care. Only 16% of respondents (n = 20) ranked a numerical rating scale (11-boxex labelled 0–10) as their first preference.
Response and state-attainment criteria
The final group of 6 questions in the OMIRROP survey concerned response and state-attainment. In particular, rheumatologists were asked their opinion as to what relative and absolute critical values for pain, stiffness, function, and patient global assessment (PGA) they considered of clinical value to patients, when evaluating the importance of the clinical response achieved and the clinical state attained. Responses were made on categorical scales graded in 10-unit increments from 0 to 100. A “Don’t know” option was provided for each question.
Minimum clinically important improvement (MCII); hip/knee OA — absolute
Nineteen percent to 24% of respondents stated that they did not know what constituted the absolute value (0–100 normalized units, nu) for the MCII for hip and knee OA (Table 1). The majority of the remaining rheumatologists responded in the 20–29 nu category, for pain, stiffness, function, and PGA, with percentage of respondents as follows: pain 38%, stiffness 34%, function 30%, and PGA 31% (Table 1).
MCII; hip/knee OA — relative
Seventeen percent to 23% of respondents stated that they did not know what constituted the relative (0–100%) value for the MCII for hip and knee OA (Table 1). The majority of the remaining rheumatologists responded in the 20%–29% category, for pain, stiffness, function and PGA, with percentage of respondents as follows: pain 35%, stiffness 30%, function 29%, and PGA 29% (Table 1).
MCII; hand OA — absolute
Twenty-two percent to 28% of respondents stated that they did not know what constituted the absolute (0–100 nu) value for the MCII for hand OA (Table 1). The majority of the remaining rheumatologists responded in the 20–29 nu category, for pain, stiffness, function and PGA, with percentage of respondents as follows: pain 32%, stiffness 27%, function 26%, and PGA 27% (Table 1).
MCII; hand OA — relative
Twenty percent to 25% of respondents stated that they did not know what constituted the relative (0–100%) value for the MCII for hand OA (Table 1). The majority of the remaining rheumatologists responded in the 20%–29% category, for pain, stiffness, function and PGA, with percentage of respondents as follows: pain 32%, stiffness 27%, function 29%, and PGA 29% (Table 1).
Patient acceptable symptom severity (PASS); hip/knee OA
Twenty-five percent to 30% of respondents stated that they did not know what constituted the absolute (0–100 nu) value for PASS for hip/knee OA (Table 1). The majority of the remaining rheumatologists responded in the 20–29 nu category for pain, and in the 30–39 nu category for stiffness, function and PGA, with percentage of respondents as follows: pain 22%, stiffness 17%, function 20%, and PGA 21% (Table 1).
PASS; hand OA
Twenty-six percent to 30% of respondents stated that they did not know what constituted the absolute (0–100 nu) value for PASS for hand OA (Table 1). The majority of the remaining rheumatologists responded in the 20–29 nu category for function, and in the 30–39 nu category for pain, stiffness, and PGA, with percentage of respondents as follows: pain 23%, stiffness 24%, function 21%, and PGA 23% (Table 1).
DISCUSSION
Quantification of the clinical effects of interventions by patient-reported outcome (PRO) measures is a standard procedure in clinical research. The methods used are well recognized, and employ valid, reliable, and responsive measurement techniques. Numerous clinical research publications in the OA pharmaceuticals literature attest to the adjudication of clinical benefit through the application of PRO measures. In contrast, there is a paucity of evidence in the OA clinical practice literature to support their use in routine clinical care. Quantitative clinical measurement in routine clinical care offers several potential advantages: (1) it can provide information regarding the severity of the patient’s disease, and place the patient on the spectrum of disease; (2) it can provide information to both physician and patient that can be used to inform shared goal-setting, and in decision-making regarding the necessity to initiate, continue, modify, or terminate a particular therapy; (3) it can provide information to disability insurers regarding the severity of disease and the outcome of treatment programs; (4) it can provide information to litigators regarding the patient’s health status and may provide some insight into attribution issues; (5) it can allow healthcare agencies to understand the clinical effect of their expenditures, and therefore the appropriateness of ongoing payment for clinical interventions; and (6) where normative values are available, it can permit the patient’s health status to be benchmarked against their age and sex-matched peer group.
In the OMIRROP 2007 survey, a response rate of 58% was achieved. This is comparable with other surveys of this type. Where a submaximal response rate is achieved, concerns regarding nonresponse bias arise. It is difficult to assess the magnitude and direction of that bias. However, in some areas of medicine it has been noted that practice style is a function of year of graduation or year of entering practice27. There was no significant difference in the year of FRACP certification between respondents and nonrespondents, suggesting that the observations are likely generalizable to all Australian rheumatologists. Further, the majority of rheumatologists engaged in both hospital and private practice. These factors together with representation from both male and female rheumatologists from all Australian states and the capital territory support the contention that the results are likely generalizable to all Australian rheumatologists.
Given that many Australian rheumatologists do not routinely follow their patients with OA, there is limited opportunity for them to use HSM in longitudinal followup. While the HAQ and WOMAC were the most frequently used measures, the general lack of formal recording and archiving of quantitative information was noteworthy. The reasons why quantitative information is not routinely recorded, archived, analyzed, and used by rheumatologists to inform decision making in OA therapeutics are unknown. Reasons could include the following: absence of evidence that the use of quantitative information to inform shared decision-making and goal-setting results in superior patient outcomes; lack of information/guidance on the interpretation of HSM data; lack of formal education in outcome measurement techniques in undergraduate and postgraduate curricula in rheumatology, leading to a lack of familiarity with HSM; lack of obligation by colleges, societies, and regulatory authorities to record and report quantitative health status information; absence of a measurement culture; logistic difficulties in recording, archiving, and analyzing HSM data; and lack of international agreement, until recently, on core sets of outcome measures in OA. Intuitively, the absence of evidence explanation seems the most likely reason. Recently published Royal Australian College of General Practitioners OA Guidelines28, recommending the establishment of patient care plans and the recording of patient responses, suggest that at least one authority in Australia has recognized the importance of quantitative measurement in clinical practice environments.
It is clear from each of the OMIRROP surveys, including the latest (2007), that rheumatologists not only place high importance on clinimetric aspects of their measurement tools (validity, reliability, and responsiveness), but also on practical aspects such as simplicity, quick completion, and easy scoring. Many, but not all existing HSM meet all of these requirements, and therefore it is more likely that factors other than the logistics of data capture are more influential in the lack of utilization. While respondents expressed a preference for Likert-type scales for recording health status, Likert, visual analog, and numeric rating scales are all responsive29, and current Australian National Health and Medical Research Council acute pain management guidelines suggest the use of a numeric rating scale for pain30.
The last section of the OMIRROP 2007 survey questionnaire concerned response and state-attainment. In the last 8 years, responder criteria have emerged5–9 and state-attainment criteria have been proposed14,16. A limited analysis comparing Response Status Assignment (RSA) by experts, patient perception, and OMERACT responder criteria showed high levels of between-method agreement in RSA31. However, a recent international evaluation of MCII75 and PASS75 based on the WOMAC Numeric Rating Scale 3.1 Index and the AUSCAN Numeric Rating Scale 3.1 Index showed considerable between-country differences in the estimates. Given the emerging data in this field, caution should be exercised in interpreting the OMIRROP response and state-attainment data. They were collected on categorical, not continuous scales, for reasons of convenience and familiarity, and are not directly comparable to estimates based on continuous data5–9,11,12,14–17.
Despite the publication of full reports in the peer review literature since 2000, it is clear that 17%–29% of respondents did not provide estimates, and instead selected the “Don’t know” option for one or more estimates. This may suggest either that the “Don’t know” respondents were unaware of the literature on response and state-attainment in OA, or that they remain unconvinced there is, as yet, an acceptable and applicable specification for response and state-attainment in hip, knee, and hand OA. Those estimates that were provided by respondents were often at variance with published response and state-attainment criteria proposed in the OA literature (Table 2). It should be noted, however, that even the previously published estimates differ from one another, probably due to conceptual, methodological, and geographical differences (Table 2).
Limitations of our study include the potential for nonresponse bias. Based on a response rate > 50%, response from all 4 practice types, both sexes, all Australian states, and a lack of difference in year of FRACP certification, a nonresponse bias is unlikely. Whether the results can be generalized outside Australia is unknown. Based on between country differences observed in the REFLECT study17, it should not necessarily be assumed that the results are globally generalizable. Further study is therefore recommended.
OMIRROP responses suggest that many patients with OA are not followed longitudinally, and that standardized patient-reported outcomes are infrequently utilized in tracking patient progress. While there is consensus on the qualities required of a measurement tool for evaluating PRO, existing PRO that meet these requirements are not used routinely. Further, since Australian rheumatologists are not generally aware of what constitutes a clinically important improvement, educational programs may be needed before patient responses can be appropriately interpreted. Of greatest strategic importance is the application of quantitative PRO data in benchmarking, and to inform shared decision-making and therapeutic goal-setting. Whether the benchmark is Minimum Perceptible Clinical Improvement, Minimum Clinically Important Improvement, OMERACT-OARSI responder criteria, Patient Acceptable Symptom State, the BLISS Index, or recently developed WOMAC and AUSCAN-derived age and sex-specific population-based normative data collected from 13,000 randomly selected members of the Australian general public32,33, opportunities to interpret PRO data are emerging. Investigation of data interpretation issues is essential in further evaluating the role of quantitative PRO measurement in routine clinical practice in OA.
Acknowledgments
The valuable contribution of the Australian Rheumatology Association is recognized and much appreciated.
Footnotes
-
Supported in part by an unrestricted educational grant from Genzyme Biosurgery Inc., Cambridge, Massachusetts, USA.
- Accepted for publication December 24, 2008.