Abstract
Objective Advances in the field of axial spondyloarthritis (axSpA) and the methodology to develop core sets have led the Assessment of SpondyloArthritis international Society (ASAS) group to update the ASAS–Outcomes in Rheumatology (OMERACT) core set. An important aspect was to ensure it would be applicable to the entire spectrum of axSpA. The first step was to define the most relevant disease domains.
Methods A 3-round Delphi survey was conducted to gather opinions of 188 patients and 188 axSpA experts to define the most relevant disease domains to be included in the core set. The Delphi survey evaluated 2 separate research settings: (1) studies assessing symptom-modifying therapies; and (2) studies evaluating disease-modifying therapies. Importance of the domains was rated on a 1–9 Likert scale. A domain was considered for inclusion if, for both stakeholder groups, ≥ 70% of participants scored the domain as critical (7–9) and ≤ 15% scored it as not important (1–3) after 3 rounds.
Results A total of 132 (70%) patients and 135 (72%) experts completed at least 1 round. After 3 rounds, 7 domains (pain, physical function, stiffness, disease activity, mobility, overall functioning and health, peripheral manifestations) were selected for the symptom-modifying therapies setting. For the disease-modifying therapies setting, 6 domains (physical function, disease activity, mobility, structural damage, extramusculoskeletal manifestations, peripheral manifestations) were selected. All domains selected by experts were also selected by patients. Patients selected all offered domains except emotional function.
Conclusion This study provides the domains selected by patients and axSpA experts that should be considered for the core set for axSpA.
The Assessment of SpondyloArthritis international Society (ASAS) collaborated with Outcome Measures in Rheumatology (OMERACT) to develop a core outcome set for ankylosing spondylitis (AS) in 1999.1 The core set has been well implemented in the field in the past 20 years.2 Nevertheless, since the development of the original core set, it has become apparent that AS belongs to the broader disease spectrum of axial spondyloarthritis (axSpA), which consists of 2 subtypes: radiographic axSpA (also known as AS) and nonradiographic axSpA.3 Further, there have been major advances in outcome instruments in the field of axSpA, such as the use of magnetic resonance imaging,4 and the development of the Ankylosing Spondylitis Disease Activity Score (ASDAS),5 validated enthesitis scores,6 the ASAS Health Index,7 and the ASAS flare definition.8
In addition, the methodology to develop core outcome sets has improved. Although there is no gold standard for the development or update of a core set, in the last few years OMERACT and the Core Outcome Measures in Effectiveness Trials (COMET) have worked exhaustively to provide specific guidance on how a core set should be developed (e.g., OMERACT handbook9 and OMERACT Filter 2.0,10 COMET handbook,11 and Core Outcome Set–Standards for Development12). Because of all these advances, the ASAS group decided it was necessary to update the original ASAS-OMERACT core set for AS according to the current recommended methodology, to ensure the core set will be applicable to the entire spectrum of axSpA.
An important step in the process of updating the core outcome set was determining which disease domains (outcomes) are relevant. In order to establish these, a 3-round Delphi survey was employed to gather opinions from relevant stakeholders. The results of this 3-round Delphi survey formed the basis of the proposal for a final core set according to the new format of the OMERACT Onion.13 Subsequently, the proposal was presented to OMERACT to seek endorsement for the proposed core domain set. A detailed description of the entire process that led to the selection of domains for the updated core set will be published separately. The methods used to compose and execute the Delphi survey, as well as its results, are described in the current paper. The aim of this study was to select the domains that should be considered for inclusion in the core set for axSpA.
METHODS
Preparation of the Delphi survey. The original core set14 was developed for 3 different scenarios: (1) therapies that improve the symptoms and clinical features of inflammatory manifestations of the disease (symptom-modifying antirheumatic therapy [SMART]; this includes physical therapy); (2) therapies that change the course of disease by decreasing inflammatory manifestations (thereby improving function) and by preventing or decreasing structural damage (disease-modifying antirheumatic drugs [DMARDs]); and (3) clinical record keeping in daily practice, to facilitate uniform clinical record keeping to enable research from clinical records and to monitor patient care in a standardized way.
The core set update focused only on the first 2 scenarios. Thus, the Delphi survey consisted of 2 separate sections: one focused on the outcomes to be included in the core set for studies assessing symptom-modifying therapies, the other on the outcomes to be included in the core set for studies evaluating disease-modifying therapies.
A list of candidate domains to include in the Delphi survey was computed using 3 sources: (1) the current core set for AS14; (2) all domains assessed in studies evaluating pharmacological and nonpharmacological interventions identified in the systematic literature review (SLR) that assessed the implementation of the original core set2 (to ensure the most recent studies were included, the search strategy from the SLR was used to identify studies published thereafter, i.e., between 2011 and 2018); and (3) information collected on the qualitative studies and patient focus group interviews conducted as part of the development of the ASAS/World Health Organization Comprehensive and Brief Core sets of the International Classification of Functioning, Disability and Health (ICF) for AS.15,16 All aspects of health identified in this process were considered when defining candidate domains for the core set for axSpA.
After eliminating duplicates, the list of candidate domains was grouped and finalized by 3 of the authors (DvdH, VNC, AB) and later agreed on by the steering committee. The first round of the Delphi survey contained 11 candidate domains for symptom-modification therapies and 12 candidate domains for disease-modification therapies (the same domains with 1 additional domain representing structural damage). For this first round, participants had the opportunity to suggest additional domains.
Participants. The invited participants were divided in 2 main stakeholder groups: 1 group consisted of patients with axSpA and the other group consisted of a variety of expert stakeholders (all ASAS members, including rheumatologists, other healthcare professionals, methodologists, and researchers, as well as representatives from the pharmaceutical industry and drug regulatory agencies), labeled as axSpA experts. The ASAS members were informed they would be invited to participate in the Delphi survey to update the current core set in an annual meeting prior to commencement of the project. Representatives of the pharmaceutical industry and drug regulatory agencies were informed of the project by email and invited to participate prior to commencement of the project. Patients were recruited through 3 national patient societies (Spondylitis Association of America, National Ankylosing Spondylitis Society, and Canadian Spondylitis Association) and were eligible to participate if they were aged ≥ 18 years and had a diagnosis of axSpA from their rheumatologist. Information regarding the Delphi survey and its purpose was posted on the websites of each of the organizations, and patients were invited to participate by email through their associations. Recruitment ceased once the group of patients was equal in size to the group of experts (n = 188). Ethical approval and consent to participate in the Delphi survey was not required based on the Dutch Medical Research Involving Human Subjects Act (WMO).
Content of the Delphi survey. An explanatory text was provided at the beginning of the survey in each round, containing information on the purpose of the Delphi and relevant information to fully understand the content and scoring system. This information was adapted per stakeholder group, using lay wording and more extensive explanations for the patients.
The main objective of this Delphi survey was to select the most relevant disease domains to be included in the core set for axSpA. Simultaneously, this survey was used to investigate the effect of invitation approach on the response rate and final outcome of a Delphi survey. The methods and results of this experiment are published separately.17 In summary, the participants were not aware of the experiment and received identical information regarding the Delphi survey. All participants knew from the start that this was a 3-round Delphi but did not know that for half of the participants, an invitation for the second and third rounds was conditional on responding to the first round. The experiment on the 2 different ways of inviting participants showed no effect on the final results of the Delphi survey.17 For the purpose of the domain selection, it was predetermined that the information from all participants regardless of invitation approach would be used. Here we present the results of the Delphi for the 2 different stakeholders that will be used for the core set.
The Delphi survey was split according to the 2 established scenarios (i.e., SMART and DMARD) and grouped by domain (i.e., participants who were invited to vote on the relevance of a specific domain in symptom-modifying therapies first and immediately thereafter on the same domain in disease-modifying therapies). This procedure was maintained for all domains except structural damage, which was offered for voting only in the disease-modifying therapies section of the survey. A definition was provided for each domain in all rounds, including a brief explanation and examples (Supplementary Table 1, available with the online version of this article).
In each round, the participants received summarized information of the previous round, including their individual score and aggregated scores from their respective stakeholder group. Participants who responded for the first time to the invitation for round 2 received only aggregated scores of the first round, and the same procedure applied to round 3.
Each round was open for 2–3 weeks and a single reminder was sent after 1 week to those who did not yet complete the round. Data were collected online using SurveyMonkey (www.surveymonkey.com) between November 2 and December 30, 2018.
Domain selection. To identify the importance of each of the domains for the core set, each participant was asked to provide 1 score per domain using a 9-point Likert scoring system. Domains were graded according to their level of importance. Following the OMERACT handbook, a score of 1–3 signified an outcome as not important, 4-6 as important but not critical, and 7–9 as critical.9 The aggregated scores per domain were analyzed separately for each of the stakeholder groups. If a domain was scored as critical by ≥ 80% of the participants in a stakeholder group, the domain was selected for consideration in the core set and was not offered for voting in subsequent rounds for this stakeholder group. If a domain did not achieve this score, the predefined criteria to include a domain in the next round of the Delphi per stakeholder group were as follows: ≥ 50% of the participants scored the domain as critical; and ≤ 20% scored the domain as not important.
Finally, a domain was considered for inclusion in the core set if, for both stakeholder groups (experts and patients), ≥ 70% of participants scored the domain as critical and ≤ 15% scored it as not important after 3 rounds; this is in line with the guidelines provided in the OMERACT handbook.9
Statistical analysis. For the purpose of this study, we used descriptive statistics to present the data. To determine which domains fulfilled the criteria to be considered for inclusion, the proportion of participants voting critical, important but not critical, and not important were calculated.
RESULTS
In total, 376 participants were invited to participate: 188 patients and 188 axSpA experts. Patients were from 3 countries in 2 continents, and axSpA experts were from 41 countries from 5 continents (Supplementary Table 2, available with the online version of this article). The axSpA experts who completed at least 1 round consisted of 123 rheumatologists (of whom 10 were also methodologists and 2 were also patient representatives), 4 physiotherapists, 4 representatives from pharmaceutical companies, 2 radiologists, and 2 researchers.
Participants. The overall response rate was 49% for the patients and 58% for the axSpA experts after the final round. In addition, rounds 1 and 2 were completed by 63% and 52% of patients and 60% and 55% of axSpA experts, respectively.
Content of the Delphi survey. In round 1, stiffness was mentioned by multiple axSpA experts and was therefore added to the list of domains from round 2 onward for both scenarios (i.e., symptom- and disease-modifying therapies). Supplementary Table 3 (available with the online version of this article) provides an overview of the domains that were offered for voting in each round for each of the stakeholder groups.
Domain selection. Table 1 and Table 2 present the proportion of critical votes per domain after the final round (split by stakeholder group) for the symptom- and disease-modifying therapy scenarios, respectively, wherein the domains voted as critical by ≥ 70% and not important by ≤ 15% are printed in bold. Supplementary Tables 4 and 5 (available with the online version of this article) present additional information on the proportions of critical, important but not critical, and not important votes per domain per round.
Proportion of critical votes per domain after the final round for the symptom-modifying therapies scenario, split by stakeholder group.
Proportion of critical votes per domain after the final round for the disease-modifying therapies scenario, split by stakeholder group.
For the symptom-modifying therapies, 7 domains were voted as critical by ≥ 70% of patients and axSpA experts after 3 rounds. These were as follows: disease activity, pain, overall functioning and health, physical function, mobility, peripheral manifestations, and stiffness (Table 1). An additional 4 domains were voted as critical by ≥ 70% of patients only; in fact, the domain emotional function was the only domain voted critical by < 70% of patients. There were no domains voted as critical by ≥ 70% of axSpA experts only.
For the disease-modifying therapies, 6 domains were selected by ≥ 70% of patients and axSpA experts after the final round. These were as follows: disease activity, physical function, mobility, peripheral manifestations, extramusculoskeletal manifestations, and structural damage (Table 2). An additional 6 domains were voted as critical by ≥ 70% of patients only, who selected all domains except emotional function. Identical to the symptom-modifying therapies scenario, there were no domains voted as critical by experts only.
The domains that were voted as critical by ≥ 70% and voted not important by ≤ 15% in both stakeholder groups are presented in Figure 1.
Domains selected after 3 rounds by patients (dark) and axSpA experts (light) in the setting assessing (A) symptom-modifying therapies and (B) disease-modifying therapies, including the percentage of critical votes. AxSpA: axial spondyloarthritis.
DISCUSSION
This 3-round Delphi survey was an important step in the process to update the core outcome set and aimed to determine which domains should be considered for inclusion according to patients and axSpA experts. Only 1 additional domain (stiffness) was added from round 2 onward, indicating that the candidate domains identified in the preparatory steps were a good representation of the domains of interest in the field.
In our study, patients selected more domains to be included in the core set compared with the axSpA experts. Specifically, domains such as fatigue, sleep, and work and employment were deemed very important by patients, but less so by experts. These domains have a major effect on the daily life of the patient, but are not necessarily specific to the disease; this could explain the difference in importance between patients and experts. In general, axSpA experts deemed the more objectively measurable domains such as structural damage and mobility as most critical to be measured in settings investigating disease-modifying therapies, whereas the more subjective domains such as pain, stiffness, and overall functioning and health were limited to the settings investigating symptom-modifying treatments.
The domain with the highest percentage of critical votes in the group of axSpA experts in both settings was disease activity, indicating that this domain is most important to measure in all trials investigating therapies for axSpA according to experts; in patients, however, the highest percentage of critical votes in both settings was for pain. There was a noticeable difference in the voting for the domain pain in the disease-modifying therapies setting, wherein 95% of patients voted it as critical, yet only 65% of the experts deemed this domain important enough to be measured in all trials investigating DMARDs.
A large panel of international axSpA experts and patients were invited to participate in this study. The use of an electronically distributed Delphi ensured no travel was required and anonymity was guaranteed. Further, no public speaking was required, which is known to increase patient participation.18 Despite these measures, not all continents were equally represented, as the majority of axSpA experts who responded were from Europe and America, and invitations to patients were restricted to native English speakers to ensure understanding of the survey and its components. Nevertheless, all stakeholder groups who will benefit from an updated core outcome set were included in its development, which we hope will increase uptake. Finally, OMERACT and COMET methodology were followed as closely as possible.
In conclusion, this Delphi survey study identified 7 domains that should be considered for the core set evaluating the efficacy of symptom-modifying therapies, and 6 domains that should be considered for the core set investigating disease-modifying therapies, according to patients and axSpA experts. The results from this study will be used to compose the core outcome set for axSpA, in which a distinction will be made for the domains mandatory for studies assessing symptom-modifying therapies and studies evaluating disease-modifying therapies. After finalizing the core outcome set, the next step for ASAS will be to identify appropriate instruments to measure the chosen domains.
ACKNOWLEDGMENT
We would like to thank all patients from the Spondylitis Association of America, Canadian Spondylitis Association, and National Axial Spondyloarthritis Society for their participation in the Delphi survey. We would also like to thank all ASAS members as well as the representatives of the pharmaceutical companies and drug regulatory agencies for their contribution to the project.
Footnotes
The Assessment of SpondyloArthritis international Society (ASAS) funded AB and VNC to work on the project to update the core outcome set.
The authors declare no conflicts of interest relevant to this article.
- Accepted for publication July 14, 2021.
- Copyright © 2021 by the Journal of Rheumatology
