Original ArticleComputerized adaptive test for patients with knee impairments produced valid and responsive measures of function
Introduction
Measurement of patient-reported outcomes (PRO) in health care is evolving with psychometricians capitalizing on the administrative efficiency and measure precision of computerized adaptive tests (CAT) [1], [2], [3]. The primary advantages of CAT administrations [4], [5], [6] are that the adaptive tests tailor PRO item administration, so each patient responds to items whose location, that is, item's position on the latent continuum [7], [8], or difficulty approximates the patient's ability estimates, and the number of items administered is minimized [8], [9], [10], [11]. In this era of clinical efficiency that is increasingly influenced by the need for the collection of PROs that may influence payment policy [12], [13], [14], measure precision and efficiency are important.
CAT has its origins in mental [15], educational [16], and military [5] testing. Recently, CAT applications have been introduced in health care [3], [17]. As described by Jette et al. [18], application of CAT in health care have been recommended for nearly a decade [19], [20], [21], but only recently have CAT been used in data simulations [22], [23], [24], [25], [26], [27], research demonstrations [24], [28], [29], and prospective data collections [18], [30], [31]. Although development of CAT in health care in general is growing [1], [2], [3], [10], [32], development and use of CAT in outpatient rehabilitation specifically [18], [25], [26], [27], [30], [33], [34] is rapidly advancing. Several factors positively influence that growth. Therapists commonly measure PRO because national associations recommend outcomes collection and analysis [35]. Outcomes measures are common in outpatient rehabilitation [35]. Most patients have functional limitations that improve with therapy and can be measured [18], [36], [37]. Payers expect justification for treatment [38].
The current study builds on previous work where we developed and simulated body part specific CAT [25], [26], [27] using retrospective data analyses. Here, we evaluated the practical and psychometric adequacy of a CAT used to assess functional status (FS) of patients with knee impairments seeking rehabilitation in outpatient therapy clinics participating with Focus On Therapeutic Outcomes, Inc. (FOTO), an international medical rehabilitation outcomes database management company [39], [40]. As previously described [26], for the purpose of this study of patients with knee impairments, the latent trait of interest is lower extremity FS, which we operationally define as the patient's perception of their ability to perform functional tasks described in the FS items. Therefore, FS represents the level of ability of the patient, which represents the patient's location on the underlying latent trait of FS [7], [8]. The item bank for the knee CAT [26] was developed using items from the Lower Extremity Functional Scale (LEFS) [41], a scale with strong psychometrics [41], [42], [43], [44] and broad clinical and research acceptance [45]. FS, as assessed using LEFS items, represents the “activity” dimension of the World Health Organization's International Classification of Functioning, Disability and Health [46].
Our purposes were to (1) assess the practicality of using a CAT in routine clinical practice; (2) perform a psychometric evaluation of content range coverage and test precision; and (3) assess known group construct validity, sensitivity to change, and responsiveness of the FS measures estimated using the knee CAT. Analyses will be used to determine strengths and weaknesses of the CAT and help direct future research designed to improve the CAT's practicality and psychometric properties.
Section snippets
Design and setting
We conducted a secondary analysis of retrospective data collected from patients with knee impairments prior to and at conclusion of rehabilitation. Focus On Therapeutic Outcomes, Inc. Institutional Review Board for the Protection of Human Subjects approved the project.
Subjects
Patients with knee impairments, who represent a sample of convenience, were analyzed (Table 1). Identification of medical or surgical diagnoses was optional in the data collection (described below), but of the patients with
Efficiency
Scale distributions, number of treatment visits, and treatment duration are displayed in Table 2. Patients who had both intake and discharge data, compared to patients with just intake data, were older (52 years [SD = 17] vs. 49 years [SD = 16], t = 13.6, df = 18,807, P < 0.001), had more chronic symptoms (chi-square = 19.8, df = 2, P < 0.001), had more comorbidities (chi-square = 9.1, df = 3, P = 0.029), took more medication at intake for their condition (chi-square = 30.9, df = 1, P < 0.001), were more likely to be
Discussion
CAT are common in standardized testing for licensure, certification, and admissions tests [4], [6] but have only recently begun to be used to collect routine clinical data in busy outpatient rehabilitation clinics in the United States [18], [31] and Israel [30]. Results of the current study support the conclusions of Jette et al. [18] that the potential is good for CAT to efficiently generate estimates of FS that are precise, valid, and sensitive to change. These conclusions are encouraging for
Acknowledgments
The authors would like to thank Karon F. Cook, PhD for her insightful comments regarding statistical analyses, results, and manuscript edits.
References (92)
- et al.
An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact
J Clin Epidemiol
(2006) A computer adaptive testing simulation applied to the FIM instrument motor component
Arch Phys Med Rehabil
(2003)- et al.
Computer adaptive testing improved accuracy and precision of scores over random item selection in a physical functioning item bank
J Clin Epidemiol
(2006) - et al.
Simulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function
J Clin Epidemiol
(2006) - et al.
Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments
J Clin Epidemiol
(2005) - et al.
Simulated computerized adaptive test for patients with lumbar spine impairments was efficient and produced valid measures of function
J Clin Epidemiol
(2006) - et al.
Score comparability of short forms and computerized adaptive testing: simulation study with the activity measure for post-acute care
Arch Phys Med Rehabil
(2004) - et al.
A 37-item shoulder functional status item pool had negligible differential item functioning
J Clin Epidemiol
(2006) - et al.
The development of a comorbidity index with physical function as the outcome
J Clin Epidemiol
(2005) - et al.
A taxonomy for responsiveness
J Clin Epidemiol
(2001)