Abstract
Objectives
Cognitive Interviewing (CI) is a technique increasingly used to obtain respondent feedback on potential items during questionnaire development. No standard guidelines exist by which to incorporate CI feedback in deciding to retain, revise, or eliminate potential items. We used CI in developing fatigue items for the National Institutes of Health (NIH) Patient-Reported Outcomes Measurement Information System (PROMIS) Roadmap initiative. Our aims were to describe the CI process, formally evaluate the utility of decisions made on the basis of CI, and offer suggestions for future research.
Methods
Participants were 22 patients with a diverse range of chronic health conditions. During CI, each participant provided feedback on a series of items. We then reviewed the CI data and decided whether to retain, revise, or eliminate each potential item. Following this, we developed or adopted three quantitative methods to compare retained versus eliminated items.
Results
Retained items raised fewer serious concerns, were less likely to be viewed as non-applicable, and were less likely to display problems with clarity or to make incorrect assumptions about respondents.
Conclusions
CI was useful in developing the PROMIS fatigue items and the methods used to judge CI for the present item set may be useful for future investigations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cella, D., Yount, S., Rothrock, N., et al. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH roadmap cooperative group during its first two years. Medical Care, 45, S3–S11. doi:10.1097/01.mlr.0000258615.42478.55.
Bartley, S. H., & Chute, E. (1947). Fatigue and impairment in man. New York: McGraw-Hill.
Christodoulou, C. (2005). The assessment and measurement of fatigue. In J. DeLuca (Ed.), Fatigue as a window to the brain (pp. 19–35). New York: MIT Press.
Yellen, S. B., Cella, D. F., Webster, K., et al. (1997). Measuring fatigue and other anemia-related symptoms with the Functional Assessment of Cancer Therapy (FACT) measurement system. Journal of Pain and Symptom Management, 13, 63–74. doi:10.1016/S0885-3924(96)00274-6.
Wessely, S., Hotopf, M., & Sharpe, D. (1998). Chronic fatigue and its syndromes. New York: Oxford University Press.
Stewart, A. L., Hays, R. D., & Ware, J. E. (1992). Health perceptions, energy/fatigue, and health distress measures. In A. L. Stewart & J. E. Ware (Eds.), Measuring functional status and well-being: The medical outcomes study approach (pp. 143–172). Durham, NC: Duke University Press.
North American Nursing Diagnosis Association. (1996). Nursing diagnoses: Definition and classification, 1997–1998. Philadelphia, PA: McGraw-Hill.
Glaus, A. (1998). Fatigue in patients with cancer: Analysis and assessment. Heidelberg, Germany: Springer.
DeWalt, D. A., Rothrock, N., Yount, S., et al. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45, S12–S21. doi:10.1097/01.mlr.0000254567.79743.e2.
Reeve, B. B., Hays, R. D., Bjorner, J. B., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45, S22–S31. doi:10.1097/01.mlr.0000250483.85507.04.
Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks: Sage.
Bartenfeld, T. A. (2003). Department of health and human services: Center for disease control and prevention: Proposed data collections submitted for public comment and recommendations. Federal Register, 68, 35227.
Wu, H. S., & McSweeney, M. (2004). Assessing fatigue in persons with cancer—An instrument development and testing study. Cancer, 101, 1685–1695. doi:10.1002/cncr.20540.
Hyde, M., Wiggins, R. D., Higgs, P., et al. (2003). A measure of quality of life in early old age: The theory, development and properties of a needs satisfaction model (CASP-19). Aging & Mental Health, 7, 186–194. doi:10.1080/1360786031000101157.
Carbone, E. T., Campbell, M. K., & Honess-Morreale, L. (2002). Use of cognitive interview techniques in the development of nutrition surveys and interactive nutrition messages for low-income populations. Journal of the American Dietetic Association, 102, 690–696. doi:10.1016/S0002-8223(02)90156-2.
Murtagh, F. E. M., Addington-Hall, J. M., & Higginson, I. J. (2007). The value of cognitive interviewing techniques in palliative care research. Palliative Medicine, 21, 87–93. doi:10.1177/0269216306075367.
Napoles-Springer, A. M., Santoyo-Olsson, J., O’Brien, H., et al. (2006). Using cognitive interviews to develop surveys in diverse populations. Medical Care, 44, S21–S30. doi:10.1097/01.mlr.0000245425.65905.1d.
Willis, G. B., & Lessler, J. T. (1999). Question appraisal system: QAS-99. National Cancer Institute.
Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71, 287–311. doi:10.1093/poq/nfm006.
Conrad, F., & Blair, J. (1996). From impressions to data: Increasing the objectivity of cognitive interviews. In Proceedings of the Section on Survey Research Methods. Alexandria, VA: American Statistical Association, pp. 1–9.
Acknowledgements
The Patient-Reported Outcomes Measurement Information System (PROMIS) is a National Institutes of Health (NIH) Roadmap initiative to develop a computerized system measuring patient-reported outcomes in respondents with a wide range of chronic diseases and demographic characteristics. PROMIS was funded by cooperative agreements to a Statistical Coordinating Center (Evanston Northwestern Healthcare, PI: David Cella, PhD, U01AR52177) and six Primary Research Sites (Duke University, PI: Kevin Weinfurt, PhD, U01AR52186; University of North Carolina, PI: Darren DeWalt, MD, MPH, U01AR52181; University of Pittsburgh, PI: Paul A. Pilkonis, PhD, U01AR52155; Stanford University, PI: James Fries, MD, U01AR52158; Stony Brook University, PI: Arthur Stone, PhD, U01AR52170; and University of Washington, PI: Dagmar Amtmann, PhD, U01AR52171). NIH Science Officers on this project are Deborah Ader, Ph.D., Susan Czajkowski, PhD, Lawrence Fine, MD, DrPH, Louis Quatrano, PhD, Bryce Reeve, PhD, William Riley, PhD, and Susana Serrate-Sztein, PhD. This manuscript was reviewed by the PROMIS Publications Subcommittee prior to external peer review. See the web site at http://www.nihpromis.org for additional information on the PROMIS cooperative group. We thank the team at the University of North Carolina, Chapel Hill that conducted the cognitive interviews. We also thank Pamela Calvanese and Maria Bolger for helping to review the CI feedback on the fatigue items. A.A.S. is the chair of the scientific advisory board of invivodata, inc. and a senior scientist at the Gallup Organization.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Christodoulou, C., Junghaenel, D.U., DeWalt, D.A. et al. Cognitive interviewing in the evaluation of fatigue items: Results from the patient-reported outcomes measurement information system (PROMIS). Qual Life Res 17, 1239–1246 (2008). https://doi.org/10.1007/s11136-008-9402-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-008-9402-x