CURRENT MEASURES OF TREATMENT EFFECT

The most common measure used in reporting clinical trials is the relative risk reduction (RRR). In brief, consider a parallel group, randomised trial comparing an active treatment with placebo; the outcome of interest is dichotomous (event/no event) and assumed to be prevented by the active treatment. The probability of the event in the treatment group (X) is supposed to be less than the probability of the event in the placebo group (Y). Relative risk is defined as the ratio of the probability of the event in the treatment group and that of the control group (X/Y). RRR is defined as the reduction of the probability of the event in the treatment group in proportion to that in the placebo group [(Y−X)/Y] and it can be calculated from relative risk as RRR=1−relative risk (1−X/Y). RRR is usually expressed as a percentage [(1−(X/Y))×100%]. The problem of the RRR is that it fails to discriminate between enormous absolute treatment effects and very small effects in absolute numbers. Table 1 shows an example of this. The relative risk and RRR calculated from the trial by Moreland and colleagues² and a hypothetical trial were similar.

In contrast, the absolute difference in rates between the experimental and control groups does discriminate between low and high magnitudes of benefit and harm. Absolute risk reduction is the difference in the event rates between the placebo and treatment groups (Y−X).^1,3–5 The inverse of the absolute risk reduction applies for estimating the increase in absolute risk for a side effect (absolute risk increase). Because the values of relative risk and RRR depend on the probability of the event in the control group, they provide less information on the clinical benefit. Absolute risk reduction is considered a better measure of treatment effect than relative risk or RRR because it expresses the consequences of giving no treatment.^1,3 However, the absolute risk reductions and increases are often <1 and have to be expressed as a decimal fraction, which is difficult to incorporate into clinical practice. Table 1 shows that the difference between treatment groups is much larger when looking at absolute risk reductions than the relative risk or RRR. However, it may still be difficult to understand the clinical benefit of an absolute reduction of 0.45 compared with that of 0.0024.

The risk reduction obtained from either of these measures of treatment effect is regarded as a “point estimate” because the precise value of risk reduction is unknown but lies within a certain extent—that is, confidence intervals (CIs). The point estimates are usually provided by most clinical trials and the 95% CI is frequently used.³

The advantages of the absolute risk reduction can be retained but made much easier to use by “inverting” it and taking its reciprocal—this is called the NNT. This is clinically useful because it is the number of patients who must be treated in order to obtain the benefit of interest in one new patient.³ The point estimates of NNT should be accompanied by the 95% CI, which is simply the reciprocals of the 95% CI of absolute risk reduction.^1,3 The advantage of the NNT over the relative risk and RRR is that it expresses both the risk without treatment and the risk reduction with treatment. In addition, the NNT informs the clinicians and patients how much effort they must spend to prevent one event and allows comparison of the amount of effort needed to prevent the same event with other treatment options.³

An example for calculation of the NNT is shown in table 1.

Mathematical calculation of the number needed to harm (NNH) is similar to that for the NNT but differs in that the experimental treatment increases the probability of a bad outcome compared with placebo or other comparator.^3,5,6 This is useful when an adverse event is caused by the active treatment. The NNH is defined as the number of patients who receive the active treatment that will lead to one additional patient being harmed compared with those receiving placebo.⁵ The 95% CI for the NNH is calculated as for the NNT. The NNH should be considered together with the NNT because an experimental treatment may help decrease the probability of one event, but may increase the probability of another adverse event, which might exceed the beneficial effect of the active treatment. The NNH calculation is also shown in table 1.

Among the measures of treatment effect, the NNT seems to be the most helpful tool for therapeutic decision making and bedside teaching, as it is easier to interpret and can be compared among different treatment alternatives.¹ The NNT describes the difference in the outcome of interest achieved between treatment and control. For therapeutic trials, small NNTs (with the numbers close to 1) indicate a favourable effect of the treatment.⁷

The NNT for a certain intervention in an individual patient depends on both the nature of treatment and the baseline risk of that patient—that is, the event rate in the control group.^3,6–9 If the baseline risk is high, even a small RRR will produce a low NNT. On the other hand, if the event rate in the control group is low, the RRR must be large to produce a low NNT. For example, in a disease with a baseline risk of 90%, an RRR of only 15% will yield an NNT of 7. But if the baseline risk of another disease is 30%, an RRR of 50% is needed to yield the same NNT. Thus, an NNT provided in the literature must be adjusted for a patient's risk at baseline.^7,8 Table 2 shows the effect of the baseline risk and RRR on the NNT. To evaluate and compare the effectiveness of various interventions in rheumatology, we conducted this study to assess the NNTs and NNHs of the clinically important outcomes provided in the systematic reviews in the Cochrane Library.

METHODS

We used the key words musculoskeletal, rheumatology, and systematic reviews to search for eligible reviews. Eligible trials included rheumatology systematic reviews from the Cochrane Database of Systematic Reviews in the Cochrane Library, issue 2, 2000 for rheumatoid arthritis (RA), osteoarthritis (OA), and systemic sclerosis (SSc). These were supplemented by data from reviews in progress. Additional data were obtained from personal communications with the authors. Where sufficient data on the outcomes were not available in the reviews, we searched for the data from the original articles (randomised, double blind, controlled trials (RCTs) only). We then calculated the NNTs from the absolute risk differences of the outcomes of interest. These outcomes must be primary, dichotomous outcomes. For disease modifying antirheumatic drug (DMARD) treatment for RA the outcome was whether or not the patients met the American College of Rheumatology 20% improvement (ACR20)¹⁰ or Paulus criteria¹¹; for OA interventions, whether or not the improvement of the joint pain met the predefined criteria; for SSc, whether or not the improvement of Raynaud's phenomenon met the predefined study criteria. For trials of RA interventions conducted before the ACR20 or Paulus criteria were developed, using the approach of Norman et al,¹² we calculated the NNT from the mean improvement in the tender joint count, which was the most commonly used continuous outcome in these studies. For the calculation of NNH from these interventions, we used the dichotomous outcome of whether or not the patients withdrew from the studies owing to adverse events.

The NNTs and NNHs were estimated to provide the current best estimate of their positive and negative effects—they were not compared with each other, because these trials vary in the population studied, the activity and severity of the disease, and the period of treatment.

RESULTS

From the Cochrane Database of Systematic Reviews, we retrieved 63 complete reviews of musculoskeletal disorders. From these, nine systematic reviews in RA,^13–21 two in OA,^22,23 and four in SSc^24–27 contained the categorical outcomes required for NNT and NNH calculation and were included in this study. Pooled estimates were available for NNT and NNH calculation in 13 systematic reviews. In two reviews, azathioprine for RA¹³ and penicillamine for RA,¹⁵ a continuous outcome was used for calculating the NNT. Eleven original articles contributed the additional data for NNT calculation from their dichotomous outcomes.^2,28–37

Figure 1 shows the point estimates and 95% CI of the NNTs and NNHs calculated from the RA systematic reviews and original articles. For the NNT, the outcome was whether or not the clinical improvement of the patients met the ACR20 or Paulus criteria. If the studies were published before the time of these two criteria, the outcome measures used for calculating the NNTs were the criteria for clinical improvement defined in each study.^36,37 In the study by Townes et al the criteria for clinical improvement used for calculating the NNT were whether or not the patient improved in five of the following measures of disease activity: tender joint count, swollen joint count, grip strength, 50 foot (15 m) walk time, duration of morning stiffness, and erythrocyte sedimentation rate.³⁶ In the study by Williams et al, the NNT was calculated from whether or not the number of tender joints improved by 30% or more.³⁷ If the ACR20 was not available and a dichotomous measure of clinical improvement was not specified in the trial report, the NNTs were calculated using the mean change in the number of tender joints.^13,15 This methodology was derived from a study by Norman and colleagues on the relationship between effect size and proportion benefiting from treatment.¹² From this study, the association between effect size and proportion benefiting from treatment for parallel group studies was a near-linear curve and nearly independent of minimally important difference.¹² These authors showed that the NNTs derived from continuous outcomes using this curve approximated those calculated from dichotomous outcomes. The 95% CIs of the NNTs calculated from the mean change were also obtained by the same method.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Number needed to treat (NNT) and number needed to harm (NNH) in RA clinical trials and systematic reviews. *95% CI not calculated for non-significant results. SSZ, sulfasalazine; MTX, methotrexate; CYC, cyclophosphamide; AZA, azathioprine; d-Pen, d-penicillamine; CsA, cyclosporin A; Pred, prednisolone; TJC, tender joint count.

Figure 1 shows the NNTs and NNHs of different DMARDs and biological agents. To recap, the NNT is defined as the number of patients with RA that need to receive treatment for one additional patient to achieve a treatment response. In most RA studies the efficacy of most active treatments was compared with placebo. The exceptions were the trials of cyclosporin A (CsA)³⁰ and infliximab,³¹ which studied the additional efficacy over methotrexate (MTX), and the COBRA trial,³⁵ which compared the efficacy of triple drugs (MTX+prednisolone+sulfasalazine (SSZ)) with that of SSZ alone.

For the treatment of active RA at 24 weeks with different agents compared with placebo, using the ACR20 response criteria to calculate the NNTs, etanercept treatment for six months² had an NNT of 1.6 (95% CI 1.4 to 2.0). SSZ had an NNT of 3.7 (95%CI 2.5 to 6.7) and the NNT for leflunomide was 3.6 (95% CI 2.9 to 4.8). In the trials comparing the efficacy of MTX with that of placebo, the NNT calculated from the 18 week trial³³ was 2.9 (95% CI 2.1 to 4.9) and that from a more recent trial with a longer treatment duration of 12 months³² was higher (5.0; 95% CI 3.3 to 11.0). These differences were not statistically significant.

The NNT for combined MTX, prednisolone, and SSZ compared with SSZ alone was 4.8 (95% CI 2.1 to 21.3) at 28 weeks.³⁵ The NNT for combined CsA+MTX treatment compared with MTX alone at 24 weeks was 3.2 (95% CI 2.2 to 5.7),³⁰ whereas the NNT for infliximab+MTX compared with MTX alone at 30 weeks of treatment was 3.2 (95% CI 2.3 to 5.9).³¹

Figure 1 shows the NNHs for these treatments of RA. The point estimates demonstrate a wide range and a number of these estimates have infinitely broad CIs—for example, antimalarial drugs and etanercept. The point estimates of the NNTs and NNHs without 95% CI are not of proven benefit or harm, but they may still be clinically important.⁷ Further studies are required to achieve a finite 95% CI to confirm the benefit or harm of these interventions.

Table 3 shows the results for OA, listing the NNTs to improve pain or patient global assessment of disease severity from the systematic reviews in OA of the hip and knee. For the treatment of hip OA, different analgesics and non-steroidal anti-inflammatory drugs (NSAIDs) were compared in a single systematic review.²² A significant improvement in pain was seen when etodolac and tenoxicam were compared with placebo. The NNT for a four week treatment with etodolac was 4.4 (95% CI 2.4 to 24.4) and that for an eight week treatment by tenoxicam was 3.8 (95% CI 2.5 to 7.3). The NNTs of the other NSAIDs and analgesics are shown as point estimates only.

For knee OA, only low level laser treatment, which is one of the modalities used in physiotherapy, was systematically reviewed.²³ The point estimate of the NNT for the low level laser treatment to improve pain in one patient was 11.6. The NNH results of both OA interventions were not available owing to insufficient data in the reviews.

Table 4 shows the NNTs for different drugs in patients with SSc to improve digital ulcers or patient global assessment of disease severity.^24–27 Although the NNTS calculated from certain drugs were small—for example, intravenous iloprost (NNT=1.2)²⁵ and ketanserin (NNT=2.3),²⁶ their efficacy is still inconclusive because the 95% CIs are large and include 1.

Table 4 also shows the NNHs of the agents used in SSc. The calculated NNHs of iloprost and ketanserin were small, approximately similar to their NNTs. Although these two drugs may be efficacious in the treatment of Raynaud's phenomenon in SSc, their small NNHs indicate that half of the treated patients would develop adverse events requiring the treatment to be stopped.

The NNTs and NNHs in these tables may need to be adjusted to match the individual patients' baseline risk; as mentioned earlier, this will depend upon factors such as the severity of the presenting disease and whether in primary or tertiary referred care; most of the trials included here are derived from the tertiary practices of rheumatologists. Health professionals in other situations, such as those in primary care, will need to adjust the NNTs and NNHs using table 2.

DISCUSSION

This study demonstrates the use of NNT and NNH as one approach that should be considered for communicating the clinical importance of results of intervention studies assessing interventions in rheumatological disorders, which can be used as a guideline for decision making in rheumatology clinical practice.

Systematic reviews are increasingly used for evaluating the efficacy and safety of interventions in clinical trials. They also explicitly provide information on the effectiveness and safety of the experimental treatment when applied to patients in routine clinical practice.⁷ However, the numerical and statistical results in systematic reviews are difficult to understand and use in daily practice. A more understandable term is needed to translate these results into a term which can easily be used.⁷

The NNT is a measure of clinical benefit that is useful in clinical practice because it interprets the abstract terms used in clinical trials to a more concrete term for routine practice decision making.³ Although many clinical trials in rheumatology interventions have been conducted and published, the main focus is usually upon statistical significance rather than clinical importance. The objective of this review was to provide the best current estimates of the clinically important benefit and adverse effects using the NNT and NNH metric for rheumatology interventions on the basis of published data from systematic reviews and RCTs. They are intended to demonstrate the NNTs calculated from different interventions in rheumatological disorders and might be used as a guideline for decision making in rheumatology clinical practice.

The NNH should be considered along with the NNT to assess the benefit and harm of an intervention.^3,7 For example, the NNT for a 12 month treatment with leflunomide was 4 for patients with RA to achieve the ACR20 response.³² The NNH was 10 for withdrawal due to adverse events caused by leflunomide.³² Thus, if 20 patients with active RA are treated with leflunomide for a year, five will achieve the ACR20 response while two are expected to withdraw owing to the adverse events induced by this drug.

The NNT can be affected remarkably by the risk at baseline of the patient and risk reduction by the intervention. The higher the event rate in the control group, the smaller the NNT will be.³ An adjustment for treatment duration is also required for comparisons among different interventions. The NNT for the same intervention will be smaller if the treatment duration is longer.

Although the NNTs in fig 1 might be considered to be comparable among different treatments because most of the studies on RA treatment were conducted in white patients with active RA that could not be adequately controlled by conventional treatment (analgesic drugs and/or NSAIDs), the NNT tables and figure presented in this study are not intended to be used as “league tables” for a comparison across different interventions or diseases. Even in the same disease condition, adjustments for baseline risk and time period of treatment may invalidate such comparisons.

Other limitations of the NNT/NNH include⁷: (a) the need to take the 95% CI of the NNT/NNH into consideration. For example, if the 95% CI of an NNT estimate is infinite, it is possible that the experimental treatment has no benefit or causes harm even if the point estimate of the NNT shows a beneficial effect; the inverse applies for the NNH. Inclusion of the 95% CI makes the point estimate of the NNT/NNH more clinically relevant and interpretable. (b)The NNTs can be compared among different interventions for the same condition and severity with the same outcome and period of treatment, but because the concept of the NNT is one of frequency, not of utility or importance, it is inappropriate to compare the NNTs across different disease outcomes or treatment duration.

CONCLUSION

The NNT is a term translated from the less understandable results of clinical trials and systematic reviews to help clinicians in routine practice decision making. It provides the information about treatment benefit by incorporating both the baseline risk without treatment and the risk reduction with treatment. The NNTs demonstrate the number of patients needed to be treated in order to prevent one event and they can be compared among different treatment options for the same disease and outcome. The point estimates of the NNT should be presented with a 95% CI. The NNH can be used to describe the harm caused by the intervention in the same context as the NNT. In this study, the NNTs and NNHs for the interventions of RA, OA, and SSc from clinical trials and systematic reviews are presented. Both the NNTs and NNHs should be adjusted for the baseline risk and treatment duration in individual patients.