Simulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function

Abstract

Background and Objective

To test unidimensionality and local independence of a set of shoulder functional status (SFS) items, develop a computerized adaptive test (CAT) of the items using a rating scale item response theory model (RSM), and compare discriminant validity of measures generated using all items (θ_IRT) and measures generated using the simulated CAT (θ_CAT).

Study Design and Setting

We performed a secondary analysis of data collected prospectively during rehabilitation of 400 patients with shoulder impairments who completed 60 SFS items.

Results

Factor analytic techniques supported that the 42 SFS items formed a unidimensional scale and were locally independent. Except for five items, which were deleted, the RSM fit the data well. The remaining 37 SFS items were used to generate the CAT. On average, 6 items were needed to estimate precise measures of function using the SFS CAT, compared with all 37 SFS items. The θ_IRT and θ_CAT measures were highly correlated (r = .96) and resulted in similar classifications of patients.

Conclusion

The simulated SFS CAT was efficient and produced precise, clinically relevant measures of functional status with good discriminating ability.

Introduction

A recent study by Hart et al. [1] showed how computerized adaptive testing (CAT) can transform the process of estimating latent traits, using simulated CATs to estimate measures of functional abilities for patients with lower extremity impairments by analyzing a patient's performance on a set of items. The CATs precisely estimated measures of lower extremity functional status, with good discriminating ability. Latent traits and abilities (or, as defined in the Hart et al. [1] study, measures of lower extremity functional status) cannot be directly observed but can be estimated by analyzing a person's performance on a set of items [2]. CATs have recently emerged in the medical [3], [4] and rehabilitation [1], [5], [6] fields, and development of CAT measures of function in rehabilitation has been recommended [7], [8], [9].

The advantages of CATs compared to a computer-administered or paper-and-pencil outcomes instruments have been enumerated [10], [11]. From a clinical perspective, however, the most important advantages of CATs are that they provide an efficient alternative to traditional paper-and-pencil or computer-administered tests, allow outcomes data to be collected during the clinical encounter with reduced patient burden, and produce precise estimates of patient ability [1], [6].

Here we describe the construction of a CAT using responses to the original items that were administered in the development of the Flexilevel Scale of Shoulder Function (FLEX-SF) [12], a flexilevel [13], [14] outcomes instrument for patients with shoulder impairments receiving rehabilitation. No studies have assessed the developmental FLEX-SF items for unidimensionality, local independence, or generated a CAT application from these items. Our overall objective was to develop a clinically relevant CAT specific to patients with shoulder impairments using the original developmental FLEX-SF items. Specific objectives were to (i) test unidimensionality and local independence of the developmental FLEX-SF items, (ii) develop a CAT using these items, and (iii) compare how well simulated CAT measures discriminated among patients compared to measures based on the all the FLEX-SF items.