Abstract
Objective. To assess the validation status of echocardiography with continuous Doppler (echo-Doppler) as an outcome measure in pulmonary arterial hypertension associated with systemic sclerosis (PAH-SSc).
Methods. Structured literature review on full-text English articles was performed using the PubMed and Cochrane databases. Assessment of validation of echo-Doppler was based on the OMERACT filter criteria with the domains truth (face, content, construct, and criterion validity), discrimination, and feasibility.
Results. Out of 35 studies eligible for analysis, only 5 included well defined PAH-SSc subgroups (World Health Organization criteria). Echo was considered as having face validity based on expert opinion and high number of studies using echo for evaluation of patients with SSc. Echo was considered partially validated with respect to criterion validity based on significant correlations between echo measures and right-heart catheterization in patients with SSc at risk of PAH/PH. However, echo was found to lack specificity (lack of content validity), since measurements of echo pulmonary pressure may be influenced by left-heart disease and interstitial lung disease. Data from general populations of patients with scleroderma indicate that evaluation of pulmonary artery pressure by echo might not be available in all PAH-SSc patients because of technical factors. No studies enabling evaluation of the discriminant capacity over time and treatment of echo in PAH-SSc could be identified.
Conclusion. Further studies are needed to fully validate echo-Doppler as an outcome measure in PAH-SSc. These studies would include cross-sectional analysis of baseline measures and longitudinal data of placebo and verum groups in randomized controlled trials of patients with PAH-SSc.
- SCLERODERMA
- HYPERTENSION
- ECHO-DOPPLER
- EXPERT PANEL ON OUTCOMES MEASURES IN PULMONARY ARTERIAL HYPERTENSION
- SYSTEMIC SCLEROSIS
- OUTCOMES
- CLINICAL TRIALS
Pulmonary arterial hypertension (PAH) associated with systemic sclerosis (scleroderma, SSc) is one of the most frequent causes of death in patients with SSc. PAH develops on the basis of obstructive proliferative vasculopathy of small and medium-size pulmonary arteries. In the setting of SSc, other causes than primary pulmonary vasculopathy may also lead to an increase in pulmonary artery pressure (PAP). Indeed, significant lung disease, which might lead to pulmonary hypertension (PH) due to hypoxemia, was found in up to 30%–75% of SSc patients with elevated PAP1,2. Left-heart dysfunction, which might cause postcapillary/venous PH, was found in up to 13%–19% of SSc patients suspected of having PAH3,4.
To date only intravenous epoprostenol has been proven to be beneficial in a randomized controlled trial (RCT) exclusively in patients with PAH-SSc5. In contrast, the other therapies were investigated in more general PAH populations, where PAH-SSc contributed roughly one-quarter of patients. Although numbers were always small and statistical power calculations may have confounded the SSc-specific results, post-hoc analyses of these studies showed that the PAH-SSc subgroup was usually less responsive than patients with idiopathic PAH6. Thus, there is an urgent need for clinical studies aimed at evaluation of new therapeutics specifically in patients with PAH-SSc.
Since appropriate outcome measures are of key importance for correct evaluation of clinical trials, the OMERACT (Outcome Measures in Rheumatology Clinical Trials) consensus group has developed a set of criteria for the validation of endpoints in rheumatic diseases. These criteria are known as the OMERACT filter and include: truth (face, content, construct, and criterion validity), discrimination (reliability/reproducibility and sensitivity to change), and feasibility7. These OMERACT criteria should be fulfilled before a specific outcome measure is fully validated and recommended for use in clinical trials.
Among 11 measures identified by a recent expert panel on outcome measures in PAH-SSc (EPOSS) utilizing a Delphi process among 74 interdisciplinary experts8, only right-heart catheterization (RHC) has so far been considered validated according to the OMERACT filter criteria and therefore judged ready for use in clinical trials9. However, RHC is often not feasible for repeated measures due to its invasiveness. Echocardiography including assessment of pulmonary arterial pressures by continuous Doppler (echo-Doppler) is another endpoint indicated for consideration by the EPOSS group. However, echo requires full validation before it can be recommended for clinical trials in PAH-SSc. The aim of this study was therefore to assess the current status of validation of echo in PAH-SSc according to the OMERACT criteria using a systematic literature search. The identification of specific aspects of echo that need further validation in PAH-SSc is the basis for the design of further validation studies.
MATERIALS AND METHODS
Systematic literature review
Studies in which echo was used for the evaluation of patients with PAH/PH-SSc were searched in PubMed and Cochrane Controlled Trial Register databases using combinations of predefined key words. The key words used were “systemic sclerosis OR scleroderma OR CREST” AND “pulmonary arterial hypertension OR pulmonary hypertension” AND “echocardiography OR echo.” To identify other relevant articles, references of the retrieved papers and most recent review articles published within the last 2 years were analyzed. In addition, the “related article” tool in PubMed was used. All original studies published between 1966 and January 15, 2008, were selected if they involved ≥ 5 PAH/PH-SSc patients. Abstracts or congress reports were not included. Studies with mixed populations of PAH patients or patients with different connective tissue diseases were eligible if the subset of patients with SSc was separately analyzed, or if > 45% of the patients in the study had SSc. The literature analysis was limited to studies published in English and those pertaining to humans and adults only.
Studies were excluded if they were not an original study, if by definition only patients with other forms of PH than PAH were analyzed, if ≥ 55% of patients had diseases other than SSc, and if the studies did not include a separate analysis of SSc patients. Studies including < 5 PAH/PH-SSc patients and those in which there was no analysis of information on PAH/PH-SSc patients were also excluded. Studies concerning exercise echo were also not considered for analysis, because exercise echo was not part of the core set recommended by the EPOSS group after the Delphi exercise8.
The systematic literature search and the analysis of retrieved documents were performed independently by 2 trained reviewers (OKB, JA). If differences in judgment occurred, they were resolved by discussion.
Quality evaluation of identified manuscripts according to the methodological quality and level of evidence
The quality of studies fulfilling our inclusion criteria was rated by using the impact factor of the journal in which the study was published (ISI Journal Citation Reports 2006) and by the Jadad scale10. The Jadad scale contains 2 questions to determine appropriate randomization and study masking and 1 question evaluating the reporting of withdrawals and dropouts. Each question requires a yes or no response. Five total points can be awarded, with a higher score indicating superior quality.
The level of evidence was assessed according to established criteria based on study design using a hierarchy of evidence in descending order according to qualities11. In brief, metaanalyses of RCT were considered the highest level of evidence (1a), followed by RCT (1b), nonrandomized controlled studies (2a), quasiexperimental studies (2b), descriptive studies (3), and expert committee reports or opinions (4).
Quality evaluation of identified manuscripts according to the definition of pulmonary hypertension
Because this analysis aimed to examine the validation of echo for PAH, and because other forms of PH have different pathophysiologies, clinical courses, and clinical presentations, we also rated the respective studies according to their definition of PAH. The criteria for this quality assessment are summarized in Table 1.
Based on the tests used for the diagnosis of PAH/PH in the respective studies, the following rating was applied: PAH/PH confirmed by RHC [mean PAP > 25 mm Hg at rest and/or > 30 mm Hg with exercise per World Health Organization (WHO) definition] was assigned category A; PAH/PH assessed by echo with pulmonary artery systolic pressure (PASP) ≥ 45 mm Hg, which has 97% specificity versus RHC12, or PASP > 30 mm Hg by RHC was assigned category B; PAH/PH assessed by echo with 45 mm Hg > PASP/tricuspid gradient ≥ 35 mm Hg was assigned category C; and all other definitions were considered category D.
In addition, studies were analyzed to determine whether clinically significant interstitial lung disease (ILD) and postcapillary PH/left-heart disease were excluded. ILD and left-heart disease are considered the most frequent causes of pulmonary hypertension other than PAH in SSc. ILD was considered clinically significant when restrictive ventilatory defects and/or advanced radiological changes were present. Postcapillary PH was judged based on the wedge pressure > 15 mm Hg on RHC. Accordingly, studies in which the definition of PAH included these exclusions were assigned category 1, while all other studies were considered category 2.
Application of the OMERACT filter
To assess the current status of validation of echo, the OMERACT criteria were used. These include: truth (face, content, construct, and criterion validity), discrimination (reliability/reproducibility and sensitivity to change), and feasibility7,9. Definitions of the OMERACT criteria are given in Table 2.
The OMERACT criteria were applied on the manuscripts retrieved from the systematic literature review. For the final assessment of validation, the quality of the manuscript was taken into consideration as follows (see also Table 1):
Echo was considered valid (V) or not valid (NV) only if high quality studies were available with a definition of PAH according to the WHO criteria and if severe ILD and postcapillary PH/left-heart disease were excluded. This corresponds to the “A1” level of the quality assessment defined above.
Echo was considered partially validated (PV) if lower quality studies indicated that echo was valid. Lower quality studies were defined as all studies with a quality assessment below A1. These strict criteria were used because these studies might include patients with forms of PH other than PAH (e.g., associated with left-heart disease, interstitial fibrosis) and a number of false-positives (PAH not confirmed by RHC). This is relevant for the assessment of outcome measures because forms of PH other than PAH have a different pathogenesis, disease presentation, disease symptoms, and prognosis than those in patients with PAH.
Validation status of echo was considered unclear/possibly not valid (U), if “lower quality studies” indicated that echo was not valid. Again, lower quality studies were defined as studies with a quality assessment below A1.
Moreover, validation of echo with respect to the sensitivity to change over time required longitudinal studies for which parallel data on RHC and echo at 2 different timepoints were available. Validation of sensitivity to change over treatment required in addition data from RCT.
The application of the OMERACT criteria was discussed at 3 face to face meetings of the EPOSS steering committee. If there was disagreement on the status of validation, it was resolved by discussion.
RESULTS
Results of the systematic literature search
Out of 124 articles identified, 87 were excluded based on predefined inclusion/exclusion criteria, 2 studies could not be retrieved for full-text review, and 35/124 articles were included for further analysis1–4,13–42. The search strategy of the systematic literature research including reasons for exclusions is presented in Figure 1.
Quality assessment of retrieved articles
The 35 retrieved studies were next evaluated according to their level of quality. The results of the quality assessment are summarized in Table 3. Only 5 (14%) studies included well defined PAH-SSc subgroups according to the WHO criteria (quality level A1). Five other studies involved patients with PAH/PH diagnosed with RHC, but without excluding lung fibrosis or left-heart disease (quality level A2). One study reported sub-group analyses corresponding to A1 or A2 quality levels12. The remaining 25 studies corresponded to lower quality levels: 3 studies were classified as B2, 13 studies as C2, 9 studies as D2, and one study included subgroup analyses of patients classified as C2 or D242.
No RCT fulfilling the inclusion criteria could be identified. Four uncontrolled studies21–23,30 represented level of evidence 2b, while the remaining studies were classified as level of evidence 3. The impact factor of the identified studies varied from 0 to 7.421 (mean 3.39).
Status of validation according to the OMERACT criteria
The current status of validation of echo according to the OMERACT criteria and based on the systematic literature review and its quality assessment is summarized in Table 4.
I. Truth
1. Face validity. Echocardiography was selected by the experts during the recent Delphi study8 as an appropriate measure for the evaluation of PAP, heart structure and function in patients with PAH-SSc. Thus, by definition, it was considered credible (having face validity).
2. Content validity. Several studies indicated that echo does not differentiate between different forms of PH associated with SSc and is thus not specific for PAH-SSc. Accordingly, left-heart disease/postcapillary PH were found in up to 19% of SSc patients with increased tricuspid velocity (> 2.5 m/s) and in up to 13% of SSc patients considered at high risk of PAH/PH by clinical evaluation including echo examination, radiographic studies, and lung function tests3,4.
Several studies including SSc patients with and without PAH/PH revealed significant associations between higher PASP/tricuspid gradient and the presence of ILD, as evaluated by lung function tests13,15,26,27 or high resolution computed tomography (HRCT) of the lungs1,2,23,24. Significant lung disease (defined by total lung capacity < 70%–80%, and/or diffuse interstitial fibrosis/alveolitis by HRCT) was found in up to 30%–75% of SSc patients with elevated PAP1,2.
Thus, it can be anticipated that measurements of PASP by echo-Doppler might reflect the presence of ILD or left-heart disease and not only the pulmonary vasculopathy underlying true PAH.
Another aspect of content validity is whether the outcome measure of interest covers the whole spectrum of disease severity. Denton, et al looked at a broad range of patients with PAH/PH34. The range of PAP by RHC in the 6 patients with PAH/PH and no tricuspid regurgitation was similar (30–80 mm Hg) to that seen in the 15 patients with PAH/PH in whom echo-Doppler measurement revealed increased PAP (34–109 mm Hg), indicating that even in patients with moderate/severe PAH/PH, tricuspid regurgitation might not be present.
Together, these results show that echo is not specific for PAH and thus does not fulfill this aspect of the content validity criterion of the OMERACT filter. The highest quality level of studies evaluated for this criterion was A14. Content validity according to the OMERACT filter was therefore rated as “not valid” by the expert group.
3. Criterion validity. Sensitivity and specificity of echo versus RHC. The sensitivity of echo-Doppler for identification of SSc patients with PAH/PH ranged from 39% to 100%, and its specificity from 42% to 97% in comparison with RHC as the “gold standard” measure. Sensitivity and specificity depended strongly on the definition of PAH/PH by echo and the population of patients with SSc1,12,34,41. The specificity of echo increased with the higher PASP thresholds, reaching 97% for PASP/tricuspid gradient ≥ 45 mm Hg. In contrast, the sensitivity was highest when the lower cutoff values were used, being 90% in SSc patients with PASP > 30 mm Hg12,34. Thus, there is an inverse relationship between specificity and sensitivity of echo in identifying patients with PAH/PH-SSc.
Of note, the majority of studies comparing echo with RHC used in their definition of PAH/PH a combination of tricuspid gradient/PASP and, particularly if PASP was not measurable or suggestive of PAH/PH, evaluation of right-heart dimensions and function and/or clinical assessment including evaluation of dyspnea, pulmonary function tests, and/or chest radiographs.
Correlation of variables measured by echo with PAP measured by RHC
Three studies involving SSc patients considered to be at high risk of PAH/PH, including those with significant ILD, showed significant correlations between PASP (tricuspid gradient) measured by echo-Doppler and PASP/mean PAP measured by RHC. However, r values were only low to moderate (r2 = 0.5 for PASP and r2 = 0.5 for mPAP)1,12,34. Similarly, a study by Murata, et al including 77 patients with connective tissue diseases of whom 55% had SSc showed significant correlation between PASP by echo and by direct measurement during RHC (p < 0.01)40. Depending on the diagnostic criteria, the presence of pulmonary fibrosis by HRCT or pulmonary function test was reported in 38% to 75% of patients evaluated in these studies.
Another study compared the right ventricular myocardial performance index, also known as the Tei index, which is calculated based on echo measurements (the isovolumic contraction time and isovolumic relaxation time divided by the ejection time) and RHC measures in a group of 35 SSc patients with elevated PASP ≥ 35 mm Hg, of whom 28 patients had mPAP by RHC consistent with the WHO definition of PAH. The right ventricular Tei index correlated significantly with the mean PAP by RHC at low r values (r2 = 0.21, p = 0.01), but not with the pulmonary vascular resistance measured during catheterization (r2 = 0.11, p = 0.08). Similarly to the 2 previous studies, patients with ILD were also included17.
In summary, depending on the cutoff value for PASP, echo showed an acceptable sensitivity and specificity and a significant, but rather weak correlation with RHC. The highest quality level of studies evaluated for sensitivity/specificity and correlation of echo with RHC was A2. The criterion validity was therefore rated “partially validated.”
4. Construct validity. Association with survival/mortality. Only one study of SSc patients with PAH defined according to the WHO criteria (A1) was identified; it showed that the presence of pericardial effusion is associated with higher number of deaths in PAH-SSc (hazard ratio 2.35, 95% CI 1.06–5.2, p = 0.04)19. The presence of pericardial effusion of any size (odds ratio 10.7, p = 0.001) or significant pericardial effusion were also associated with higher mortality in the general SSc population15,36,42.
Another retrospective, case-control study involving 206 patients with limited cutaneous SSc showed that the presence of PH (PASP > 30 mm Hg by echo in combination with clinical symptoms) was associated with lower survival (50% at 2 years and 10% at 5 years) in comparison with well matched controls without PH (88% and 80%, respectively)28.
In the study by MacGregor, et al, high PASP (> 60 mm Hg) by echo was associated with higher mortality in the overall population of SSc patients with PH (PASP > 30 mm Hg by echo), in SSc patients with isolated PH (PH without significant lung disease by lung function tests/HRCT), and in those with PH and lung disease31. The risk of death in the whole PH population increased significantly (hazard ratio 3.6, 95% CI 1.42–9.15) for PASP > 60 mm Hg. The 2-year mortality in SSc patients with isolated PH was 8%, 33%, and 67%, and in those with PH and lung disease 0%, 15%, and 67%, for PASP < 30, 30–60, and > 60 mm Hg, respectively. In the overall SSc population (with and without PAH/PH), high initial PASP (> 60 mm Hg) by echo was an independent risk factor for mortality (by multivariate analysis) 31. Similarly, the presence of PH by echo was associated with lower survival in the overall SSc population combining those with and without PH (odds ratio 9.8, p = 0.002, for PH vs those without PH) in the study by Simeon, et al36, and in patients with early diffuse SSc (PASP ≥ 45 mm Hg; p = 0.001 vs those without PH) in the study by Trad, et al22.
Associations with dyspnea/functional capacity, exercise tolerance [6 minute walk test (6MWT), oxyhemoglobin desaturation after exercise (ΔO2Sat), and maximal oxygen consumption during exercise testing)]
In 67 SSc patients with and without PH, in whom ILD had been excluded (by HRCT), greater dyspnea [by New York Heart Association (NYHA) classification] appeared as an independent factor associated with PH defined as PASP > 50 mm Hg by echo (p = 0.0001 by multivariate analysis)14. In an overall population of SSc patients, frequency of dyspnea (NYHA class II, III, or IV) increased with the levels of velocity of tricuspid regurgitation (VTR) measured by echo-Doppler4. Accordingly, dyspnea was present in 29.3% of patients with VTR ≤ 2.5 m/s, in 40.6% with VTR 2.5–3 m/s, and in 72% with VTR > 3 m/s.
One study of SSc patients with and without PH, 42% of whom had restrictive pulmonary function tests, indicated that the presence of PH by echo (PASP > 30 mm Hg) was significantly associated with abnormal 6MWT (< 400 m) and ΔO2Sat in univariate but not multivariate analysis (multiple regression analysis)16.
In another study, baseline PASP by echo was the only variable that was independently correlated with maximal oxygen consumption (r = −0.66) and anaerobic threshold (r = −0.52) in stress testing33.
Associations with heart structure/function
In a population of SSc patients including those with pulmonary fibrosis, right ventricular and right atrial enlargement were more frequent in SSc patients with PH by echo PASP ≥ 36–40 mm Hg than in those without PH (p = 0.03 and p < 0.0001, respectively)13,25. PAP by echo-Doppler was independently correlated with tricuspid E/Aratio, an index of the right ventricular relaxation (r = −0.35, p < 0.003, multiple regression analysis)32. No correlation was found between PASP and right ventricular ejection fraction20.
In summary, only an association between the presence of pericardial effusion on echo and survival/mortality was supported with the highest quality study (A1 level)19. For this aspect of construct validity, echo was rated as validated.
Although 2 studies reporting association between echo measurements and dyspnea/functional class used the WHO definition of PAH (A1 level)4,14, analyses concerning PASP and functional class included heterogeneous populations of SSc patients with and without PAH/PH. For all other aspects of construct validity including associations of PASP with survival/mortality, functional capacity, and right-heart measures, only studies with a less stringent definition of PH were available (highest quality level A2). Thus, while these studies in patients with PH-SSc indicate that echo passes the construct validity criterion of the OMERACT filter, it can only be rated as partially validated as long as A1 studies involving patients with PAH-SSc do not exist.
II. Discrimination
Discriminant capacity over time and treatment. To assess the validity of discriminant capacity over time, longitudinal studies with parallel data on RHC and echo at 2 different timepoints are required, and, in addition, for validation of discriminant capacity over treatment it has to be a RCT. However, such trials were not available according to our inclusion criteria for the literature review.
Reliability. One study (quality level B2), of SSc patients with and without PH, showed low intra-and interobserver variability in evaluation of right ventricular ejection fraction and left ventricular ejection fraction (range 3.5% to 5.3%, depending on the variables)20. However, no study was available analyzing intra- or interobserver variability of variables relevant for the assessment of PAH in patients with SSc.
In view of the lack of appropriate longitudinal studies including RCT it was concluded that there are no data available to assess the discrimination criterion. Reliability was analyzed only in a study of quality level B2 and was therefore judged “partially validated.”
III. Feasibility
The high number of studies using echo as a diagnostic tool indicates that this method is feasible in the clinical assessment of patients with PAH. It is available in all centers with a cardiology department or a cardiology consultant and the cost-effectiveness for clinical studies is reasonable.
However, it is not possible to obtain PASP values in some patients because of technical limitations. Inability to evaluate PASP due to the lack of tricuspid regurgitation and/or due to insufficient quality of the images obtained was reported in several studies (Table 5)4,13,24,25,29,32,34,35,37–40,42. The percentage of patients in whom evaluation of PASP was not possible ranged from 3%37 up to 74%39. It should be noted that all studies reported above included SSc patients without PAH/PH, in whom the presence of tricuspid regurgitation might be less frequent. Of interest, in the study by Denton, et al34, with SSc patients considered at high risk of PAH/PH, PASP could not be evaluated due to the lack of tricuspid regurgitation in 13 out of 33 (39%) including 6 out of 21 (29%) patients with PAH/PH by RHC.
In summary, inability to evaluate PASP by echo-Doppler was shown in several studies of different populations of patients with SSc. All these studies evaluated heterogeneous populations with and without PAH/PH or, if including separate analyses of PAH/PH patients, were of quality levels below A1 (highest quality level A2)34,38. Thus, until studies on the ability of echo-Doppler to evaluate PASP exclusively in PAH-SSc (level A1) are available, this aspect of feasibility was judged “unclear.”
DISCUSSION
This is the first study addressing the validity of echo as an outcome measure in PAH-SSc according to a systematic literature review, while recent assessments were based on expert opinion only. In addition, one of our main tasks was to consider the quality of available studies, which largely depended on the definition of PAH in this patient population. Surprisingly, only 5 out of 35 studies (14%) used the current WHO definition of PAH (diagnosis confirmed by RHC) and excluded other forms of PH such as left-heart disease and ILD. Based on the analysis of these few studies, a final evaluation regarding validity of echo for PAH-SSc could be completed only for the content validity criterion of the OMERACT filter. In addition, face validity could be evaluated, because this criterion does not require studies in patients with PAH-SSc. For some other aspects of the OMERACT filter, indirect information could be obtained from studies with more heterogeneous SSc populations, including SSc patients with and without PAH/PH. Finally, the OMERACT criterion “sensitivity to change over time and treatment” could not be evaluated at all, because of lack of appropriate data.
Most importantly, the structured literature review and the assessment of identified reports according to the OMERACT filter revealed several aspects of echo that need further validation in additional studies. This defines a specific research agenda that needs to be addressed to allow full validation of echo as an outcome measure in PAH-SSc. This research agenda can be summarized as follows (Table 6).
Truth.
Face validity is the only OMERACT criterion that is fully validated based on the consensus of experts who selected echo as an appropriate outcome measure for evaluation of PAH-SSc8 as well as based on its frequent use as an evaluation tool in PAH/PH-SSc studies. No further studies are required.
Echo is not valid with respect to its content validity because measurements of pulmonary pressures using echo might be influenced by comorbidities including left-heart dysfunction and/or lung fibrosis in patients with SSc. No further studies are required, because available data were of sufficient quality (level A1) to allow final evaluation of its validation status.
Does the lack of content validity exclude echo as an outcome measure in PAH-SSc? Not necessarily, because the lack of specificity of echo could be overcome by excluding relevant comorbidities before SSc patients enter clinical trials. For instance, clinically relevant pulmonary fibrosis can be excluded by pulmonary function tests and/or computer tomography. Indeed, this strategy has already been utilized in many of the RCT in PAH, where lung fibrosis identified by decreased forced vital capacity was an exclusion criterion43,44. Left-heart dysfunction such as valvular diseases can be partially excluded by echo itself, although it is controversial whether definite diagnosis of diastolic dysfunction, which is more frequent in SSc45, requires invasive testing46. Other comorbidities such as pulmonary venous occlusive disease (PVOD) are more difficult to exclude, and it is currently unknown if PVOD is of relevant prevalence in PAH-SSc or is limited to certain patients with a very severe clinical course of PH47.
The influence that scleroderma lung disease and other comorbidities may have on the correlations between echo and RHC in SSc is unclear; however, evaluation of right ventricular systolic pressure by echo has been found to be inaccurate in patients with idiopathic pulmonary fibrosis48. Therefore, definite validation of the criterion and construct validity is required through evaluation of echo versus RHC and other outcomes specifically in PAH-SSc. This can be achieved, for example, by cross-sectional studies in patients with PAH-SSc undergoing RHC and other relevant outcome measures in parallel with echo (Table 6). Although these data are unpublished, they are actually available as part of the baseline measurements in randomized controlled treatment trials in PAH. Thus, subanalysis of the baseline measurements in the PAH-SSc population of these trials is likely sufficient to fully validate content and construct validity of echo.
Discrimination
To assess the validity of echo with respect to the discriminant capacity over time, longitudinal studies would be required that include PAH-SSc patients without treatment and parallel echo and RHC evaluations at different timepoints (Table 6). Again, these are already available as unpublished data from the placebo groups of RCT with RHC and echo performed at baseline and end of study. Although it might be argued that placebo treatment is an intervention and thus patients from the placebo group are not true untreated patients, analysis of the PAH-SSc subpopulation of these trials would contribute to the validation of discriminant capacity over time. Similarly, subanalysis of PAH-SSc patients in the verum group of these trials would be appropriate to assess the discriminant capacity over treatment (Table 6). Validation of reliability of echo in PAH-SSc requires comparisons of repeated echo assessments performed within a short time by the same investigator (intraobserver variability) and by 2 independent investigators (interobserver variability) at the same time in patients with well defined PAH-SSc (Table 6).
Feasibility
The validation status of echo with respect to its feasibility was considered unclear, since evaluation of pulmonary pressures was impossible in a significant proportion of SSc patients including PAH/PH-SSc. This was due to the lack of tricuspid regurgitation but also due to insufficient quality of echo imaging caused by skin thickening or concomitant severe lung disease. Further studies including exclusively patients with PAH-SSc are required to fully validate the feasibility of echo in the evaluation of PAH-SSc. Studies for full validation would require cross-sectional analysis of PAH-SSc patients, similar to studies for other aspects such as criterion and construct validity (Table 6). However, it appears unlikely that pulmonary pressures can be measured in all patients, because limitations such as skin thickening can still arise in this population. Thus, it is likely that echo cannot be used as a single outcome measure and needs to be combined with other measures to allow assessment of all patients.
Many of the studies identified by our literature review focused on PASP, while other indicators were rarely considered. However, we emphasize that other echo measures might be more relevant for certain aspects of PAH-SSc than PASP alone. For instance, as shown in patients with severe idiopathic PAH, reduction in PASP may reflect disease progression and right-ventricle failure rather than improvement of PAH49. In addition, although correlations between PASP by echo-Doppler and PASP/mean PAP by RHC were statistically significant, they showed low r values, reflecting the clinical experience that PASP by Doppler is not necessarily accurate compared to pressures measured directly in PAH-SSc. Thus, assessment of PASP alone is unlikely to reach full validation in PAH-SSc for the discriminant capacity of time and treatment. Additional measures such as tricuspid annular plane systolic excursion (TAPSE) are promising50,51 and should be considered for further validation studies in PAH-SSc.
In conclusion, this systematic literature analysis revealed that echo in PAH-SSc fulfills the OMERACT criteria for face validity and partially for criterion and construct validity as well as reliability. Other forms of PH such as that associated with lung fibrosis or left-heart disease need to be excluded if echo is used as an outcome measure, because the content validity criterion is not fulfilled. Validation studies for echo in PAH-SSc should not be limited to PASP alone, but should also include other measures such as TAPSE, in particular for the assessment of discriminant capacity over time and treatment. Studies required for further validation of the OMERACT criteria have largely been performed already as part of randomized controlled treatment trials in patients with PAH. However, subanalyses of PAH-SSc with regard to echo as an outcome measure have not been published. Because echo pulmonary pressures cannot be measured technically in all patients, echo needs to be combined with other outcomes in trials with PAH-SSc patients.
Footnotes
-
This study was partially supported by unrestricted educational grants from Actelion, Encysive, GSK, Pfizer, Bayer Schering, United Therapeutics. Dr. Kowal-Bielecka has received research funding from Actelion and Encysive via the EPOSS project. Dr. Pittrow has received honoraria for consultancy from Actelion, Encysive, Bayer Schering Healthcare, GSK and Pfizer in the area of pulmonary arterial hypertension and associated diseases. Dr. Huscher has received research funding from Actelion and Encysive via the EPOSS project. Dr. Denton has received consultancy fees or honoraria from Actelion, Encysive, Pfizer, GSK, BioVitrum, Aspreva and Dyax and Genzyme. He has received unrestricted research funding from Encysive, Actelion and Genzyme. Dr. Humbert has relationships with drug companies including AB Science, Actelion, Altair, Asmacure, Astrazeneca, Bayer Schering, Chiesi, GSK, MSD, Novartis, Nycomed, Pfizer, and United Therapeutics. Relationships include consultancy service and membership of scientific advisory boards. Dr. Nash has received research grants, clinical trial funding, given advice, or lectured on behalf of Actelion, Pfizer and Bayer. Prof. Rubin has received consulting income from Actelion and Encysive, and he has served as an investigator for Actelion and Encysive. Dr. Seibold has consultancy relationships and research funding from Actelion, Pfizer, Encysive, FibroGen, Centocor, Bristol-Myers Squibb, Genzyme, Lilly, Gilead, and United Therapeutics in the area of potential treatments of scleroderma and its complications. He has received lecture honoraria from Actelion, Pfizer, Encysive and United Therapeutics. His spouse is a full-time employee of Actelion. Dr. Strand has consultancy relationships with Pfizer, FibroGen, and Bristol-Myers Squibb in the area of potential treatments of scleroderma and its complications. Prof. Furst has had a consultancy relationship, has been a member of scientific advisory boards, and/or has received research funding from Abbott, Actelion, Amgen, BMS, BiogenIdec, Centocor, Genentech, Gilead, GSK, Merck, Nitec, Novartis, Roche, UCB, Wyeth, and Xoma. He has received lecture honoraria and/or lectured on behalf of Abbott, Actelion, Amgen, BMS, Biogen, Biogenidec, Centocor, Genentech, Gilead, Merck, Nitec, and UCB. Dr. Distler has consultancy relationships and/or has received research funding from Actelion, Pfizer, Encysive, FibroGen, Ergonex, NicOx, and Biovitrum in the area of potential treatments of scleroderma and its complications. He has received lecture honoraria from Actelion, Pfizer, Encysive and Ergonex.
- Accepted for publication August 20, 2009.