Abstract
Objective. To describe the development of the Systemic Lupus Erythematosus Disease Activity Index 2000 Responder Index-50 (S2K RI-50) Website (www.s2k-ri-50.com) and to assess satisfaction with its training and examination modules among rheumatologists and rheumatology fellows.
Methods. The development of the Website occurred in 3 phases. The first was a deployment phase that consisted of preparing the site map along with its content. The content included the S2K RI-50 training manual, the tests and corresponding question bank, and the online adaptive training module, along with the extensive site testing. The second phase included the participation of rheumatologists and trainees who completed the Website modules. The third was a quality assurance phase in which an online survey was developed to determine the satisfaction level of its users. Further modifications were implemented per participants’ recommendations.
Results. The site has been online since it was registered in September 2010. Fourteen rheumatologists and rheumatology trainees from different centers reviewed and completed the material contained in the Website. The survey revealed acceptance among rheumatologists for the Website’s content, design, and presentation. The Website was rated as user-friendly and useful in familiarizing investigators with the S2K RI-50. After completion of the training and examination modules, participants reported a suitable level of preparation to implement the S2K RI-50 in clinical trials and research settings in a timely manner.
Conclusion. The Website includes training and examination modules that familiarize rheumatologists with the S2K RI-50 and assesses their competence to use the index. This prepares them for the use of the S2K RI-50 in clinical trials and research settings.
The Systemic Lupus Erythematosus Disease Activity Index (SLEDAI-2K) Responder Index-50 (SRI-50) is a novel, valid, and reliable index able to measure ≥ 50% improvement in disease activity1,2. SRI-50 is based on SLEDAI-2K, but it allows one to discern a signal toward improvement and measure it, as compared to complete recovery in the descriptors within the SLEDAI-2K3,4,5.
The current acronym SRI-50 was modified to SLEDAI-2K Responder Index-50 (S2K RI-50). Similarly, the SRI-50.com Website was transferred to the URL www.s2k-ri-50.com. This modification was made to avoid confusion between the new index S2K RI-50 and other indices (such as the SLE Responder Index)6.
In the S2K RI-50, each of the 24 descriptors has a definition that measures ≥ 50% improvement, resulting in an appropriate score for the corresponding descriptor and a total score describing disease activity overall, as in SLEDAI-2K1. A recent study showed that S2K RI-50 captures clinically important improvements in patients with active disease between visits when patients are reevaluated within a 3-month period7. It has also been shown that the S2K RI-50 has sensitivity to response at 6 and 12 months, mirroring the timeframe of clinical trials8. In fact, S2K RI-50 identified patients with SLE who had an improvement in disease activity more than SLEDAI-2K and SLE Responder Index at 6 and 12 months8. Thus, S2K RI-50 may be used as an outcome measure in clinical trials to identify responders.
We have learned from previous clinical trials in SLE how important it is to provide rheumatologists with appropriate training for a specific measure. Indeed, the majority of the current clinical trials mandate that investigators be certified on specific measures that will be used in the study. This led us to develop the S2K RI-50 Website.
Our aims were to (1) describe the development of the S2K RI-50 Website (www.s2k-ri-50.com); and (2) assess user satisfaction with its training and examination modules among rheumatologists and rheumatology fellows.
MATERIALS AND METHODS
First phase: development of www.s2k-ri-50.com
The first phase was a deployment phase that consisted of preparing the site map along with its content. In this phase, 3 rheumatologists (ZT, DDG, and MBU), a Website developer, and a coordinator (AM) participated. The content of the S2K RI-50 Website was evaluated on several occasions to refine and establish areas for improvement, mostly related to the content and the presentation of the Website, until a final version was accepted.
Second phase: participation of rheumatologists in the training and completion of examination modules
This phase involved the invited participation of rheumatologists and rheumatology fellows from different academic centers, most of whom were unfamiliar with S2K RI-50. An invitation letter for participation in this study was sent to rheumatologists in different academic centers. Participants were provided with access to the Website and were requested to review the training module and to complete 5 randomly assigned case scenarios in the examination module. Success was determined by the participants correctly documenting the SLE-related clinical and laboratory findings on the S2K RI-50 data retrieval form and scoring the baseline and followup visit for each of the 5 cases.
The Website tracked and saved the work, and participants were able to logout and return to the examination module at their convenience and continue from the point they left off. Cases were grouped into 5 levels that differed by degree of difficulty. After completing the data retrieval form for each case, the status of the case changed from “pending” to either “incorrect” or “correct.” Participants were able to modify their answers on the data retrieval form until they achieved the right score.
Third phase: evaluation of the Website with an online satisfaction survey
In this phase, the utility of the S2K RI-50 Website was assessed. Toward this end, an online survey was developed and included the evaluation of both the training and examination modules, as well as the overall performance of the Website, including (1) feasibility: download and switch speed, ease of use, and the time required to complete the training and examination modules; and (2) content: presentation of the Website and overall impression, and usefulness of the Website in familiarizing rheumatologists with S2K RI-50 and preparing them to use the S2K RI-50 in clinical trials. The online satisfaction survey was composed of 15 questions. Eleven were answered on a 5-point Likert scale (1 strongly agree, 2 somewhat agree, 3 neither agree nor disagree, 4 somewhat disagree, and 5 strongly disagree), 2 reflected the time spent on the Website to accomplish each of the training and examination modules, 1 question had a binary outcome (yes or no), and 1 gave subjects the opportunity to leave written comments (Figure 1). In addition to the satisfaction survey, participants were encouraged to give suggestions for improving the Website. After reviewing the comments, changes were incorporated in the Website whenever applicable.
Systemic Lupus Erythematosus Disease Activity Index 2000 Responder Index-50 (S2K RI-50) training and examination modules satisfaction survey.
RESULTS
First phase: development of www.s2k-ri-50.com
The Website is composed of static and interactive parts. The static parts include (1) S2K RI-50 manual; (2) the introductory section on the 24 descriptors of SLEDAI-2K and S2K RI-50 and keywords on the S2K RI-50 data retrieval form to determine whether there is ≥ 50% improvement; and (3) a link to the PDF files of the SLEDAI-2K, S2K RI-50 definitions, and the S2K RI-50 data retrieval form9.
The interactive parts of the Website include a computer-adaptive training module with 2 cases and their solutions and a databank of case scenarios with a spectrum of disease complexity to be used as the source for examination of investigators. The objective of the S2K RI-50 manual and the training module is to familiarize investigators with the S2K RI-50 definitions and the S2K RI-50 data retrieval form. The examination module includes 5 randomly assigned case scenarios from the databank. The successful completion of any case is obtained by correctly documenting the relevant clinical and laboratory findings and scoring the baseline and the followup visit on the S2K RI-50 data retrieval form. Investigators are required to successfully complete the examination module to be certified in the use of S2K RI-50.
Second phase: participation of rheumatologists to complete the training and examination modules
In April 2011, eighteen invitations were sent to rheumatologists and rheumatology fellows to participate in our study. Ten of 12 rheumatologists and 4 of 6 rheumatology fellows accepted. Participants reviewed the Website and completed the training and examination modules. Rheumatologists were from Canada, USA, UK, Australia, Israel, Argentina, Sweden, and China. All 14 participants accomplished this phase.
Third phase: evaluation and modification of the Website
The overall response rate was 78%. The satisfaction survey documented whether participants either strongly agreed or somewhat agreed with the Website design and presentation, ease of use, and usefulness to familiarize them with S2K RI-50 (Figure 2). The mean time needed to complete the training and examination modules was 26 ± 15 and 50 ± 21 min, respectively. After completion of the training and examination modules, participants reported a suitable level of preparation to implement the S2K RI-50 in clinical trials and research settings. Participants provided suggestions and where appropriate, changes were made in accordance with those suggestions. For example, the Website was modified to provide more precise instruction on how to complete the forms and what should be documented on the electronic S2K RI-50 data retrieval form.
Survey response from 11 participants. S2K RI-50: Systemic Lupus Erythematosus Disease Activity Index 2000 Responder Index-50.
DISCUSSION
SLE is a chronic, complex disease characterized by remissions and exacerbations. The assessment of patients with SLE includes the determination of 5 domains: disease activity, chronic damage resulting from disease activity or its treatment, adverse events of drugs, health-related quality of life, and economic effect10. The ability to quantify and grade disease activity whether in clinical practice or in research settings is critical. Disease activity can be assessed with the use of global and organ-specific indices; of these, the most commonly used include the SLEDAI-2K and the British Isles Lupus Assessment Group (BILAG) index, respectively3,11. Nevertheless, even with the use of organ-specific indices, there is a trend toward determining global scores.
After almost 5 decades without approval of new drugs for SLE, the most significant breakthrough has been the US Food and Drug Administration (FDA) approval of the first biological treatment for SLE: belimumab12. The FDA approved hydroxychloroquine and corticosteroids in 1955, and aspirin was approved to treat SLE in 1948. The failure of previous trials in SLE could be due to an actual drug inefficacy, although other possibilities need to be considered, including study design, inclusion criteria, sample size, and the use of insensitive outcome measures.
Researchers have long realized the need to develop an optimal outcome measure in clinical trials of new drugs in patients with SLE. One of the most commonly adopted global indices in research settings and clinical trials is the SLEDAI, which was initially developed and introduced in 1985 in Toronto4. SLEDAI-2K is a modified version of SLEDAI that was introduced in 2002 and validated against SLEDAI3. SLEDAI-2K was modified to allow the documentation of ongoing disease activity in the descriptors: skin rash, alopecia, mucosal ulcers, and proteinuria3. Thus SLEDAI-2K includes the presence of any inflammatory rash, alopecia, or mucosal ulcers and new, recurrent, or persistent proteinuria > 0.5 g/24 h. As in the original SLEDAI, all the descriptors in SLEDAI-2K must be attributed to SLE activity3,4. The practical applicability of SLEDAI-2K in clinical settings, ease of administration, and simplicity in scoring are some of its fundamental properties. Nevertheless, SLEDAI and its modified version, SLEDAI-2K, share the same disadvantages by missing a signal toward improvement as the instrument records only complete resolution of a disease manifestation. In response to this deficit, we developed the SLEDAI-2K Responder Index-50 (S2K RI-50), an instrument that identifies partial (≥ 50%) but clinically important improvement1,13. The performance of S2K RI-50 needs to be evaluated in clinical trials.
The S2K RI-50 data retrieval form allows the identification of the change in disease activity in both directions (improvement and worsening). Nevertheless, the weighted score for a persistently active descriptor does not change on the followup visits even if a worsening has occurred in a particular descriptor. For instance, the weighted score for active skin rash is 2, and on followup visits, if the percentage and/or activity of the lesions are getting worse, the weighted score is also 2. Certainly, this is a very important research question that will require a further refinement of S2K RI-50 in the future. As mentioned, with the standardized S2K RI-50 data retrieval form it became more straightforward to measure the worsening in active descriptors over time.
Recent trials highlighted the importance of investigators’ familiarity in using a specific outcome measure to achieve valid results14. The reanalysis of the data from the epratuzumab phase IIb trial showed disagreement in the rate of responders with different composite outcome measures. This difference was driven by the incorrect scoring of the individual items on SLEDAI-2K and BILAG14. Thus, authors recommended that SLEDAI and BILAG items be monitored for consistency14. This goal can only be accomplished with appropriate training of the investigators in specific measurement and with the supervision of a monitoring committee in drug trials. Currently, one of the accepted methods of preparing investigators for specific assessment tools is training and examination modules that can be applied on paper or online.
Online training is becoming an important method for the training of physicians, healthcare providers, and investigators in the use of specific outcome measures. Currently, the trend in clinical trials in SLE is to prepare investigators in the use of an outcome measure and then have them certified in its use for each trial. The use of online material has been proposed as a feasible and effective method for preparing investigators. There are now many important SLE-related drugs under development, similar to what we have witnessed in rheumatoid arthritis and spondyloarthritis in the last decade. S2K RI-50 is a promising novel responder index currently being used in ongoing clinical trials.
We developed a dedicated Website for S2K RI-50 that includes training and examination modules. The objective of this training is to introduce physicians and researchers to the definitions of S2K RI-50 and the S2K RI-50 data retrieval form. The second part of the training module is interactive and includes cases each composed of baseline and followup visits along with their solutions. Participants document only SLE-related clinical and laboratory findings and derive the appropriate scores. The Website is interactive, and provides appropriate directions and corrections throughout the case. The training module ensures the preparedness of physicians for the examination module. In the examination module, investigators are required to successfully complete 5 cases. The 5 cases are randomly selected from a databank of case scenarios with a spectrum of disease activity. The databank includes more than 40 cases that are sorted into 5 groups based on degree of difficulty. Additional cases for the databank are added periodically. A successful completion of any case is obtained by correctly scoring the baseline and followup visit. After completing the examination module, investigators are granted certifications, making them eligible as an investigator to use S2K RI-50 in a specific clinical trial. Recertification is required for each trial and requires the same process.
This Website will be the sole resource for investigators’ preparation and will provide them with certification to use the S2K RI-50. Thus it was essential to evaluate its acceptance and performance by physicians from different centers. We assessed user satisfaction with the S2K RI-50 Website through an online survey of rheumatologists and rheumatology fellows. The results of the survey showed that the Website has good overall acceptance and confirm that the Website is very helpful in preparing rheumatologists for use of the S2K RI-50 in clinical trials and research settings. In addition, this survey allowed for further changes to the Website as recommended by participants. More importantly, the time needed to complete the training and examination modules was about 75 min.
The completion of the training and examination modules at www.s2k-ri-50.com should be mandatory before use of the S2K RI-50 in clinical trials. Currently, 105 participants are registered on the Website and the site is accepting online registration, self-training, and examination. We believe that the S2K RI-50 has great potential because previous measures of disease activity have led to difficulties and failed to perform adequately in clinical trials, therefore hampering the development and testing of novel treatments for SLE. Currently, an ongoing trial has adopted SRI-50 as an outcome measure and this will allow us to validate this new index for potential use in future trials.
Footnotes
-
Dr. Touma is a recipient of the Lupus Ontario Geoff Carr Fellowship and the University of Toronto Arthritis Centre of Excellence Fellowship. The Lupus Clinic is supported by The Lupus Flare Foundation, the Arthritis and Autoimmune Centre Foundation, the Toronto General-Toronto Western Hospital Foundation, and the Smythe Foundation.
- Accepted for publication September 25, 2012.
