Weighting Composite Endpoints in Clinical Trials: Essential Evidence for the Heart Team

doi:10.1016/j.athoracsur.2012.05.027

The Annals of Thoracic Surgery

Volume 94, Issue 6, December 2012, Pages 1908-1913

Presented at the Forty-eighth Annual Meeting of The Society of Thoracic Surgeons, Fort Lauderdale, FL, Jan 28–Feb 1, 2012.

https://doi.org/10.1016/j.athoracsur.2012.05.027 Get rights and content

Background

Coronary revascularization trials often use a composite endpoint of major adverse cardiac and cerebrovascular events (MACCE). The usual practice in analyzing data with a composite endpoint is to assign equal weights to each of the individual MACCE elements. Noninferiority margins are used to offset effects of presumably less important components, but their magnitudes are subject to bias. This study describes the relative importance of MACCE elements from a patient perspective.

Methods

A discrete choice experiment was conducted. Survey respondents were presented with a scenario that would make them eligible for the Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery (SYNTAX) trial three-vessel disease cohort. Respondents chose among pairs of procedures that differed on the 3-year probability of MACCE, potential for increased longevity, and procedure/recovery time. Conjoint analysis derived relative weights for these attributes.

Results

In all, 224 respondents completed the survey. The attributes did not have equal weight. Risk of death was most important (relative weight 0.23), followed by stroke (0.18), potential increased longevity and recovery time (each 0.17), myocardial infarction (0.14), and risk of repeat revascularization (0.11). Applying these weights to the SYNTAX 3-year endpoints resulted in a persistent, but decreased margin of difference in MACCE favoring coronary artery bypass graft surgery compared to percutaneous coronary intervention. When labeled only as “procedure A” and “procedure B,” 87% of respondents chose coronary artery bypass graft surgery over percutaneous coronary intervention. When procedures were labeled as “coronary stent” and “coronary bypass surgery,” only 73% chose coronary artery bypass graft surgery. Procedural preference varied with demographics, sex, and familiarity with the procedures.

Conclusions

The MACCE elements do not carry equal weight in a composite endpoint, from a patient perspective. Using a weighted composite endpoint increases the validity of statistical analyses and trial conclusions. Patients are subject to bias by labels when considering coronary revascularization.

Section snippets

Patients and Methods

A discrete choice experiment was conducted to define the relevance and relative weights of four MACCE elements and two other considerations in coronary revascularization. These results were applied to the Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery (SYNTAX) trial to illustrate the importance of considering individual element weights in a real-world setting. In addition, the experiment tested whether choices for coronary revascularization procedures are

Results

Of 225 persons screened for the survey, 224 (84 women and 140 men) met the eligibility criteria and completed the online survey. Table 2 shows the baseline demographic and self-reported medical history of the survey respondents. The mean age of survey respondents was 65 years (range, 34 to 90). The median time to complete the survey was 22 minutes (range, 5 to 5,536) for 28 questions and 13 choices.

A majority of the survey respondents (136 of 224, 60.7%) had prior coronary revascularization. Of

Comment

Coronary revascularization is one of the most common procedures performed in the United States, with more than 1 million performed each year [5]. Several multicenter, randomized trials, most recently, the Arterial Revascularization Therapies Study II and SYNTAX, have demonstrated superior short- and long-term clinical outcomes for CABG over PCI in patients with multivessel coronary artery disease [3, 6]. Despite this, the proportion of patients undergoing revascularization by CABG has decreased

References (24)

P.W. Serruys et al.
Five year clinical outcomes of the ARTS II (Arterial Revascularization Therapies Study II) of the sirolimus-eluting stent in the treatment of patients with multivessel de novo coronary artery lesions
J Am Coll Cardiol
(2010)
L.J. Caldon et al.
What influences clinicians' operative preferences for women with breast cancer?An application of the discrete choice experiment
Eur J Cancer
(2007)
L. Mauri et al.
The EVEREST II trial: design and rationale for a randomized study of the Evalve MitraClip system compared with mitral valve surgery for mitral regurgitation
Am Heart J
(2010)
S. Kaul et al.
Making sense of noninferiority: a clinical and statistical perspective on its application to cardiovascular clinical trials
Prog Cardiovasc Dis
(2007)
L.D. Hillis et al.
2011 ACCF/AHA guideline for coronary artery bypass graft surgery: executive summary
J Am Coll Cardiol
(2011)
V. Regitz-Zagrosek et al.
Gender as a risk factor in young, not in old, women undergoing coronary artery bypass grafting
J Am Coll Cardiol
(2004)
N. Freemantle et al.
Composite outcomes in randomized trials: greater precision but with greater uncertainty?
JAMA
(2003)
KnowledgePanel, 1998-2011
A.P. Kappetein et al.
Comparison of coronary bypass surgery with drug-eluting stenting for the treatment of left main and/or three-vessel disease: 3-year follow-up of the SYNTAX trial
Eur Heart J
(2011)
Sawtooth Software Technical Papers Library, 2012

C.J. DeFrances et al.

2006 National Hospital Discharge Survey

Natl Health Stat Rep

(2008)

A.J. Epstein et al.

Coronary revascularization trends in the United States, 2001-2008

JAMA

(2011)

Cited by (53)

Novel surgical quality metrics in abdominal aortic aneurysm repair
2022, Journal of Vascular Surgery
Existing surgical quality metrics have limited utility, are primarily used for high-mortality procedures, and often fail to account for differences in non-fatal outcomes. Our objective was to develop more comprehensive, novel surgical quality metrics, for patients undergoing abdominal aortic aneurysm (AAA) repair.
Non-ruptured open and endovascular AAA repair (EVAR) from the Vascular Quality Initiative database were studied, 2016-2019. A win was defined as AAA repair without major complication (in-hospital) or mortality (in-hospital or within 30 days). Centers were divided into quality quartiles based on performance in two novel win-based metrics: (1) Wins Above Average (WAA) and (2) weighted Wins Above Average (wWAA). Patient-level and center-level analyses compared demographics and outcomes between “best” and “worst” quartiles, including wins, mortality, and failure to rescue (FTR) rates. Additional patient-level analyses were performed based on center stratification into volume quartiles. Correlation in surgical quality for open repair and EVAR was determined at centers performing both procedures.
Overall, 3683 patients underwent open repair and 21,165 patients underwent EVAR. For open repair, crude rates of win, mortality, and FTR were 62.8%, 4.2%, and 10.3%, respectively. For EVAR, crude rates for win, mortality, and FTR were 94.4%, 1.1%, and 12.3%, respectively. When stratified by wWAA, patients undergoing open repair at “best” quartile centers had a higher win rate (72.0% vs 52.7%; risk ratio [RR], 1.37; 95% confidence interval [CI], 1.28-1.46) and lower mortality (3.1% vs 6.2%; RR, 0.50; 95% CI, 0.33-0.74) compared with “worst” quartile centers. Similarly, for EVAR, “best” quartile centers had a higher win rate (96.2% vs 92.1%; RR, 1.04; 95% CI, 1.03-1.05), lower mortality (0.4% vs 2.2%; RR, 0.19; 95% CI, 0.13-0.29), and a lower FTR rate (5.7% vs 17.9%; RR, 0.32; 95% CI, 0.18-0.56) compared with “worst” quartile centers. Stratification by volume showed that high-volume centers demonstrated improved wWAA for open repair (P = .04) but not for EVAR (P = .3) compared with low-volume centers. For centers that performed both open repair and EVAR, there was no correlation in quality as determined by wWAA (r = −0.056; P = .6).
wWAA is a novel, risk-adjusted, complication-weighted surgical quality metric that also accounts for volume differences. wWAA successfully distinguishes high- and low-quality centers for both open AAA repair and EVAR. This methodology is potentially broadly applicable for measuring surgical quality.
Composite outcomes at OMERACT: Multi-outcome domains and composite outcome domains
2021, Seminars in Arthritis and Rheumatism
The OMERACT Technical Advisory Group recognises that working groups during the process of creating a core outcome set may identify an outcome domain that would be best represented as a composite that encapsulates these component outcome domains by bringing them together into a single outcome. A multi-outcome domain (MOD) is a within-patient combination of component outcomes, and an individual patient's evaluation depends on the observation of all of the components in that patient with a single overall rating determined according to a specified rule; which is often applicable when we consider a disease activity score. A composite outcome domain (COD) consists of a number of component outcomes and is defined as the occurrence in a patient of one, some or all of these specified components; which is often applicable when we consider the risk of adverse events or remission criteria. We review the general benefits, challenges, reporting and interpretation of using MODs and CODs. The development of the MOD or COD instrument for an OMERACT core outcome measurement set is considered through four distinct steps: choosing relevant outcome domains; finding high quality instruments for each of these outcome domains; weighting the outcome domain instruments in the MOD/COD instrument; and putting MOD/COD instrument through the OMERACT Filter. Guidance and training are in preparation for working groups who will be completing the OMERACT Instrument Selection Algorithm (OFISA). As for other initiatives in OMERACT, we will seek feedback from OMERACT working groups who complete the development of their MOD/COD, which will then be incorporated into the refinement of the guidance and training.
A Weighted Scoring System Based on Preoperative and Long-Term Patient-Reported Outcome Measures to Guide Timing of Knee Arthroplasty
2021, Journal of Arthroplasty
There is currently no existing consensus regarding timing of knee arthroplasty. This study aimed to develop a weighted scoring system from patient-reported outcome measures (PROMs) to guide timing of knee arthroplasty based on preoperative severity and long-term effectiveness.
Prospectively collected data of 766 total knee arthroplasties (TKAs) and 382 unicompartmental knee arthroplasties (UKAs) at a single institution were analyzed. PROMs were assessed preoperatively and at 10 years using the Knee Society Score (KSS), Oxford Knee Score (OKS), and Short Form-36 physical component score (SF-36 PCS). Receiver operating characteristic analysis identified thresholds where preoperative PROMs predicted 10-year clinically meaningful improvements (minimal clinically important difference [MCID]). Threshold weights were assigned to PROMs based on their ability to predict MCID in isolation or in combination.
Poorer baseline PROMs predicted 10-year MCID attainments. The threshold of 49.5 points for the KSS, 30.5 points for the OKS, and 40.7 points for the SF-36 PCS and 55.5 points for the KSS, 33.5 points for the OKS, and 40.5 points for the SF-36 PCS was weighted 1 point for predicting MCID in 1 PROM for TKA and UKA, respectively. The threshold of 33.4 and 33.9 points for the SF-36 PCS was weighted 2 points for predicting MCID in 2 PROMs, whereas 29.3 and 31.3 points for the SF-36 PCS were weighted 3 points for predicting MCID in 3 PROMs for TKA and UKA, respectively. The sum of weighted components from 0 (lowest) to 5 (highest) represented likelihood for long-term benefits at 10 years.
This scoring system is a useful clinical adjunct for deciding timing of knee arthroplasty and prioritizing patients in institutions with long waitlists.
II.
Patient preferences to value health outcomes in rheumatology clinical trials: Report from the OMERACT special interest group<sup>✰</sup>
2021, Seminars in Arthritis and Rheumatism
To inform a research plan for future studies by obtaining stakeholder input on the application of preference-based methods to clinical trial design.
We conducted a virtual OMERACT session to encourage stakeholder engagement. We developed materials for the session to facilitate discussion based on identified case examples and feedback sessions.
Participants prioritized incorporating patient preferences in all aspects of trial design with an emphasis on outcome selection. Participants highlighted the need for careful consideration around preference heterogeneity and equity factors.
Including patient preferences in trial design was considered a priority requiring further exploration to develop comprehensive guidance.
Composite endpoints: What were the odds of that?
2020, Medical Hypotheses
Citation Excerpt :
We have concentrated on death and factors associated with death, however the hypothesis presented is adaptable to non-death components such as quality of life, as long as the factors can be quantified and an explanation of their weighting provided in the methodology. Weighting composite endpoints in clinical trials based on patient perception of importance of death, stroke, increased longevity and repeat revascularisation has been proposed previously, however the use of “survivors” is a major confounding factor [14]. We have concentrated on death and survival as the outcome indicator, however patient experience and perception are potentially more important.
Composite endpoints are frequently utilised in clinical studies. The creation of composite endpoints in their current form, a death carries the same weight as for example a stroke, repeat revascularisation or myocardial infarction. We hypothesise that the non-death terms of a composite score should be created such that the factor involved is weighted via odds ratios for its risk of association with death. We explore and demonstrate this using previously published odds ratios for these factors, and apply it to a hypothetical trial similar to Excel, to demonstrate the principle. The same principle could be applied to composite survival endpoints, when the factors are weighted via hazard ratios.
Neurological Complications After Transcatheter Versus Surgical Aortic Valve Replacement in Intermediate-Risk Patients
2018, Journal of the American College of Cardiology
Neurological events after aortic valve interventions are associated with increased mortality and morbidity. Transcatheter aortic valve replacement (TAVR) is increasingly offered for lower-risk patients with severe aortic stenosis, previously considered candidates for surgical aortic valve replacement (SAVR). Differences in post-procedural neurological events have important implications in treatment allocation.
The authors sought to analyze the neurological events in the randomized SURTAVI (Surgical Replacement and Transcatheter Aortic Valve Implantation) trial.
Patients with severe, symptomatic aortic stenosis at intermediate surgical risk were randomized 1:1 to TAVR or SAVR. The rates of neurological events and quality of life were analyzed at 30 days, and 6 and 12 months post-procedure in a modified intention-to-treat population (mean age 79.8 ± 6.2 years; N = 1,660).
The rates of early (30-day) stroke and post-procedural encephalopathy were higher after SAVR versus TAVR (5.4% vs. 3.3%; p = 0.031; and 7.8% vs. 1.6%; p < 0.001, respectively). At 12 months, the rate of stroke was not different between SAVR and TAVR (6.9% vs. 5.2%; p = 0.136). Early stroke and early encephalopathy resulted in an elevated mortality at 12 months in both treatment groups. Quality of life after an early stroke was significantly lower in SAVR versus TAVR patients at 30 days and was similar at 6 and 12 months.
The early stroke rate was lower after TAVR than SAVR. In patients with early strokes, QOL improved earlier after TAVR. At 12-month follow-up, stroke rates and QOL were not different between TAVR and SAVR patients. (Surgical Replacement and Transcatheter Aortic Valve Implantation [SURTAVI]; NCT01586910)

View all citing articles on Scopus

View full text

Original articleAdult cardiacWeighting Composite Endpoints in Clinical Trials: Essential Evidence for the Heart Team

Background

Methods

Results

Conclusions

Section snippets

Patients and Methods

Results

Comment

J Am Coll Cardiol

Eur J Cancer

Am Heart J

Prog Cardiovasc Dis

J Am Coll Cardiol

J Am Coll Cardiol

Composite outcomes in randomized trials: greater precision but with greater uncertainty?

JAMA

KnowledgePanel, 1998-2011

Comparison of coronary bypass surgery with drug-eluting stenting for the treatment of left main and/or three-vessel disease: 3-year follow-up of the SYNTAX trial

Eur Heart J

Sawtooth Software Technical Papers Library, 2012

2006 National Hospital Discharge Survey

Natl Health Stat Rep

Coronary revascularization trends in the United States, 2001-2008

JAMA

Original article
Adult cardiac
Weighting Composite Endpoints in Clinical Trials: Essential Evidence for the Heart Team