Abstract
Objective. Much attention has been placed upon decreasing costs of clinical research. However, little has been studied about the effects on research completion.
Methods. A survey was sent to all registry investigators and coordinators to determine the cost of enrollment in a national registry, whether sites had to supplement using their own funds, and whether the cost affected enrollment.
Results. Results indicate that a majority of sites supplemented enrollment with their own funding (88%) and diagnoses requiring a lot of time to enroll were avoided.
Conclusion. This survey showed that reimbursement rates were well below the costs of enrollment.
- REGISTRIES
- JUVENILE RHEUMATOID DISEASES
- SURVEY
- TRIAL COSTS
In recent decades, much attention has been placed upon the cost of clinical research trials, how to reduce those costs, and whether trials are cost-effective1,2,3,4,5,6,7,8. However, there has been little research into the cost of trials at the site level. Additionally, to our knowledge no work has focused on the effect that decreasing reimbursement has on enrollment.
The Childhood Arthritis and Rheumatology Research Alliance (CARRA) registry commenced in 2009 and closed to enrollment in late 2013. The registry was an observational longitudinal study that enrolled 9497 children across all major pediatric diseases at 51 sites. The registry was originally supported by a grant from the US National Institutes of Health, and subsequently through support from The Arthritis Foundation and Friends of CARRA.
Reimbursement for the registry was completed as an initial site payment upon site institutional review board (IRB) approval, and included enrollment of the first 15 subjects. Sites were then paid quarterly for subject enrollment at a rate of $44 per subject. Additionally, milestone payments were made for completion of IRB amendments and continuing reviews.
The primary aim of our survey was to determine whether the reimbursement costs associated with the CARRA registry were enough to cover the costs of enrollment and data collection, or whether outside funds were used in the completion of the research. An additional aim was to determine whether certain diagnoses were avoided for enrollment because of complexity or the time required to enroll.
MATERIALS AND METHODS
The survey was sent to all principal investigators and study staff in the study contact list (n = 134) through a commercially available online survey tool. The survey included 11 questions asking about respondents’ role in the study, enrollment numbers, which team member(s) enrolled most subjects, whether any diagnoses were avoided for enrollment and the reasons they were avoided, eligible subjects approached and enrolled, and whether individual research funds or divisional money was used to supplement the cost of completing the research (Table 1).
The survey was open for about 3 months, and several reminders were sent, asking study teams to complete the survey. Respondents had the option to remain anonymous.
We obtained enrollment data by site and for each condition from CARRA. Per CARRA guidelines, we defined small sites as those with 1 or 2 rheumatologists and large sites as having 3 or more rheumatologists. Statistics were completed using SPSS version 20.
RESULTS
A total of 52 people completed the survey (39% response rate). Respondents included principal investigators (PI; 44%), clinic nurses not specifically doing research (6%), and study coordinators (SC; 50%). The number of subjects enrolled at the sites ranged from 0 to 453 (mean 126, SD 120, median 90).
Most case report forms (CRF) were completed by the PI or SC (64% and 73%, respectively). The average hourly salary of the person completing the CRF was $34 and it took, on average, 3 h to enroll a single subject (SD 6.10 h) from recruitment through data entry. This enrollment period was an average, from recruitment and consent, through to data completion, medical record abstraction, and data entry. This means the average patient was enrolled at a cost of $102.
One-third of respondents reported avoiding certain diagnoses for enrollment into the registry, primarily systemic lupus erythematosus (SLE) and juvenile primary fibromyalgia. The primary reasons for avoiding these diagnoses were that the chart review and CRF completion were too time-consuming. We compared 4 diagnoses from the CARRA registry to that of a 3-year collection of new rheumatology patients from 25 sites in the United States reported in 1996 to see whether there was a possible selection bias (Table 2)9. There were a comparable number of children with arthritis between the 2 studies, but disparity when considering the other 3 conditions. There were relatively more children with SLE and dermatomyositis (JDMS) in the CARRA registry, and a significantly relative decrease in the number of children with fibromyalgia (FM) enrolled.
When asked whether sites supplemented the funds they received from the registry, with either their own research funds or divisional money, 88% of respondents answered affirmatively.
DISCUSSION
The CARRA registry is the first to attempt to record longitudinal data on pediatric subjects with rheumatologic diseases. It is considered an important first step in the natural observation of the course of pediatric rheumatic disease, as well as a bank of potential subjects to approach for new studies. However, the implementation of the registry was fraught with difficulties, largely because of costs that were not realized at inception. This survey supported what we expected: that the cost to enroll a subject into the registry far exceeded what was provided in the grants, and therefore most sites had to supplement what was given to them with either their own research money or divisional funds.
Because we allowed responders to keep the sites anonymous, we were unable to determine whether there was a cost difference between small and large sites. Our large site, for example, was able to hire a research assistant to recruit and enter data using our own funds, resulting in a reduced hourly cost and increased number of children entered (453). Our calculated per-patient cost was $88 per subject enrolled, which is a good deal less than the average of $102. Although we had a lower per-subject cost, we still had a total deficit of $19,932. We speculate that small centers would not have access to a research assistant, and would therefore incur higher costs and enroll fewer subjects. The use of resources varies from site to site owing to several factors, including the role of the research person. The use of a dedicated research person facilitates recruitment, although that was not specifically studied. Most rheumatology centers are stretched too thin regarding personnel to justify using their time for studies that are in addition to their clinical responsibilities. Adequate funding could help support paid research assistants whose job it is to carry out most of the research activities.
We knew that more complicated diseases tended to take longer to abstract, yet payment was the same. As a result, we postulated there would be an enrollment bias favoring subjects whose data were more easily abstracted because of their disease diagnosis or length of disease. Indeed, 33% of responders reported subject selection, most commonly against enrolling subjects with SLE and FM. However, children with SLE and JDMS were enrolled at a higher rate than the survey by Bowyer and Roettcher would predict9. This higher enrollment rate may have been observed because these children came to the clinic more often, were more willing to participate in research, or were more interesting to rheumatologists as potential research subjects. More notable is the marked dearth of children entered into the CARRA registry with FM, an absence that may reflect either the long and complicated histories of children with FM, or researcher bias against noninflammatory conditions.
This registry brings to the forefront what is often the case in academic research: that government and foundation funding is inadequate to cover the actual costs to the site for successful implementation. This is unfortunate because it prevents potentially beneficial research from being properly completed and may result in underpowered studies. It also creates an ethical dilemma in the recruitment of subjects because of the possibility that subject data may ultimately be unusable as a result of recruitment and enrollment issues. Underfunding studies creates significant barriers to investigators doing clinical research who need additional funds, either from their research monies or from divisional funds, to complete sponsored studies. Losing money to participate in funded research cannot be long tolerated in this economic climate, nor should it be. There are advantages to participation in collaborative studies, including advances in disease understanding, comparison of care and treatment, and publications. However, relying on colleagues to be good citizens in the research community is not an adequate reason to perpetuate underfunding recruitment and enrollment costs.
We did not compare our enrollment in our study to industry sponsors. It may be that reimbursement is greater in studies in which industry has vested interests and thus recruitment may be enhanced. Using a budget template that is based on industry-sponsored trials may provide a more realistic appraisal when working on budgets for registry and other non-industry trials.
In the future, all parties must make an increased mutual effort to ensure that clinical research requiring the recruitment and enrollment of subjects is properly funded so that valid and evaluable results are obtained. Investigators need to more accurately estimate the time and effort that will be necessary for the successful implementation of a trial. If the grant is a multisite protocol, sites should be required to fully estimate the costs for their own site, because these can vary greatly. We speculate that smaller sites may have increased costs. Additionally, if a grant comes in for a specific amount, the funder needs to understand that budget justifications for recruitment and enrollment are likely to vary significantly from site to site based on the circumstances that exist among pediatric rheumatology centers. Ultimately, these steps will improve the implementation of studies and the quality of the data obtained.
Acknowledgment
We thank all participants and hospital sites that recruited patients for the CARRA Registry. The authors thank the following CARRA Registry site principal investigators and research coordinators: L. Abramson, E. Anderson, M. Andrew, N. Battle, M. Becker, H. Benham, T. Beukelman, J. Birmingham, P. Blier, A. Brown, H. Brunner, A. Cabrera, D. Canter, D. Carlton, B. Caruso, L. Ceracchio, E. Chalom, J. Chang, P. Charpentier, K. Clark, J. Dean, F. Dedeoglu, B. Feldman, P. Ferguson, M. Fox, K. Francis, M. Gervasini, D. Goldsmith, G. Gorton, B. Gottlieb, T. Graham, T. Griffin, H. Grosbein, S. Guppy, H. Haftel, D. Helfrich, G. Higgins, A. Hillard, J.R. Hollister, J. Hsu, A. Hudgins, C. Hung, A. Huttenlocher, N. Ilowite, A. Imlay, L. Imundo, C.J. Inman, J. Jaqith, R. Jerath, L. Jung, P. Kahn, A. Kapedani, D. Kingsbury, K. Klein, M. Klein-Gitelman, A. Kunkel, S. Lapidus, S. Layburn, T. Lehman, C. Lindsley, M. Macgregor-Hannah, M. Malloy, C. Mawhorter, D. McCurdy, K. Mims, N. Moorthy, D. Morus, E. Muscal, M. Natter, J. Olson, K. O’Neil, K. Onel, M. Orlando, J. Palmquist, M. Phillips, L. Ponder, S. Prahalad, M. Punaro, D. Puplava, S. Quinn, A. Quintero, C. Rabinovich, A. Reed, C. Reed, S. Ringold, M. Riordan, S. Roberson, A. Robinson, J. Rossette, D. Rothman, D. Russo, N. Ruth, K. Schikler, A. Sestak, B. Shaham, Y. Sherman, M. Simmons, N. Singer, S. Spalding, H. Stapp, R. Syed, E. Thomas, K. Torok, D. Trejo, W. Upton, R. Vehe, E. von Scheven, L. Walters, J. Weiss, P. Weiss, N. Welnick, A. White, J. Woo, J. Wootton, A. Yalcindag, C. Zapp, L. Zemel, and A. Zhu.
Footnotes
-
Funded by the Division of Rheumatology, The Children’s Hospital of Philadelphia. The following organizations aided this work: US National Institute of Arthritis and Musculoskeletal and Skin Diseases, Friends of the Childhood Arthritis and Rheumatology Research Alliance, and the Arthritis Foundation.
- Accepted for publication December 17, 2014.
REFERENCES
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.