| JOURNAL HOME | CME HOME | THIS MONTH | PAST ISSUES | ETOC | COLLECTIONS |
| AUTHORS | REVIEWERS | EDITORIAL BOARD | FEEDBACK | RSS | HELP |
| ||||||||||||||
| ||||||||||||||
|
|
|
|
||||||||||||
|
||||||||||||||
Department of Anaesthesia and Intensive Care, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, New Territories, Hong Kong
Address correspondence to Anna Lee, PhD, Department of Anesthesia and Intensive Care, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, New Territories, Hong Kong. Address e-mail to annalee{at}cuhk.edu.hk
 |
Introduction |
|---|
 Top Introduction ConclusionsReferences |
|---|
This paper outlines a framework for how quantitative systematic reviews (meta-analyses) should be reported and how they may be used to identify those individuals in whom the treatment is likely to do more good than harm. We illustrate the concepts by using data from systematic reviews of ondansetron for the treatment and prevention of postoperative nausea and vomiting (PONV). Throughout this paper, we use the terms "baseline" and "underlying risk" interchangeably. Underlying risk is defined as the risk of event for a patient under the control condition; it indicates the average risk of a patient if not treated (3).
Limitations of Systematic Reviews and Meta-Analysis
In a quantitative systematic review, results of primary studies are combined by using complex statistical methods to estimate a common pooled effect with increased precision (4). The overall results of a meta-analysis represent an "average" treatment effect.
Before applying the results of systematic reviews to individual patients, clinicians must consider some of the limitations and biases (5). One of the weaknesses of meta-analysis is that it magnifies the problems of individual trials. In the case of systematic reviews of ondansetron, for example, the pooling of results from different studies may lead to inconsistencies in separate incidences of nausea, vomiting, and combined PONV because investigators do not often distinguish between nausea and vomiting but combine the symptoms of PONV into a single outcome (6). Second, if the overall conduct of the individual trial is poor, a summary treatment effect will generally be an overestimate of the "true" effect by up to 41% (7). Other specific methodological issues related to systematic reviews of ondansetron include variability of the underlying risk and event rates and small trial sample size, problems related to antiemetic comparisons without a placebo group and covert duplication of primary trials (8,9). Therefore, the interpretation of the relative efficacy and harm of antiemetic treatment can be difficult and a cautious attitude to accepting the results is warranted.
Existing Methods for Applying Results to Individual Patients
Despite the above limitations, systematic reviews and meta-analyses can highlight how effective a treatment is and identify subgroups of patients who would most benefit from an intervention. Systematic reviews and meta-analyses can guide clinical decision-making, but they are integrated with other factors in caring for an individual patient (10).
Common methods of applying the overall results of meta-analysis to individual patients are outlined below but these have limitations. First, the number needed to treat (NNT) to achieve one unit of benefit is a common clinically useful measure. However, pooled NNT may be misleading because NNT are sensitive to factors that change the baseline risk, such as the outcome considered, trends in disease risk, and clinical setting (11). A solution to this is that NNT should be derived by applying the relative risk reductions from treatment estimated by the meta-analysis to relevant baseline risks for different types of patients (Table 1) (11).
|
Economic evaluation seeks to predict net change in benefits and costs arising from alternative approaches to providing a particular form of care (13) and can be useful in deciding the optimal strategy for treatment in a patient. The applicability of the results of an economic evaluation depends on the characteristics of the patients treated, the resources used that are related to outcome, the intrinsic effectiveness of the treatments compared, the viewpoint of the analysis and time scale of the decision to be made (14). However, a review of published economic evaluations showed that two-thirds of articles gave misleading conclusions about the relative costs of alternative treatments without supporting statistical evidence (15). Therefore, the reliability and validity of cost-effectiveness analyses of antiemetics to manage PONV and its subsequent applicability to individual patients may be questionable. Currently, there is no formal methodology for the conduct and reporting of systematic reviews of economic evaluations (13).
Framework for Applying the Evidence from Systematic Reviews to Individual Patients
An approach for assessing the applicability of results from systematic reviews to individual patients has been proposed by O’Connell et al. (2). This requires the following five questions to be addressed:
The first three questions address the issue of transferability of the average treatment effect (2). The last two questions cover aspects of individualizing the treatment decision through estimating the expected absolute risk reduction based on an individual’s baseline risk and then taking into account the patient’s preferences in determining benefits and harms (2).
When considering these five issues, there may be insufficient data (2). However, this will highlight the additional information required to be reported in future systematic reviews. In the next section, we use data from the systematic reviews of ondansetron to illustrate the five issues.
Appraising Current Systematic Reviews of Ondansetron
We identified seven quantitative systematic reviews of ondansetron for the prevention and/or treatment of PONV (16–22) by searching the MEDLINE (1966 to August 2000) and EMBASE (1980 to August 2000) electronic databases. One paper was a meta-analysis of individual patient data from three randomized controlled trials (16). Each systematic review was assessed according to the five criteria outlined above (Table 2). Table 3 shows how each systematic review met these criteria.
|
|
Are there Variations in the Relative Treatment Effect?
To understand how an intervention works, consideration is given to determine whether treatment effect is constant or whether it varies according to patient, disease, intervention, and measure of effect used (2). Variations in relative treatment effect were best examined using individual patient data (16) rather than a series of meta-analyses of subgroups from primary randomized controlled trials. In the individual patient data meta-analysis, significant factors associated with the incidence of PONV were dose of ondansetron, neuromuscular blocker use, country, volatile anesthetic, type of surgery, and age, the last two factors showing a significant interaction (16). Subgroup meta-analyses of patients undergoing eye surgery, ear/nose/throat surgery, and anesthesia with or without propofol were conducted in one paper (22).
Many authors examined the dose-response efficacy of ondansetron (17–21), different routes of administration (18,19), and timing of ondansetron administration in relation to surgery (19). There were variations in the optimal dose of ondansetron for prophylaxis; 4 mg (19), 8 mg in patients with a history of PONV (21), 4 mg and 8 mg both equally effective in patients with a history of motion sickness (20), 8 mg IV (18), 16 mg orally (18), or repeated 8 mg oral doses (16). For the treatment of established PONV, there was no clinically relevant dose response between 1 mg and 8 mg (17).
Heterogeneity can be defined as the variability or differences between studies in the estimates of effect (23). All systematic reviews except one (16) tested for heterogeneity because if present, potential sources of heterogeneity should be identified, as this can affect the overall conclusion as well as the clinical implications of the review. An important source of heterogeneity is the underlying risk (24). As the underlying risk is not a measurable quantity; the best estimate is the observed risk of events in the control group (24).
How Does the Treatment Effect Vary with Baseline Risk Level?
Of the seven systematic reviews, only one (18) assessed how treatment varied with baseline risk. A sensitivity analysis of two predefined ranges of baseline risk—early PONV within 20% to 60% of control event rates and late PONV within 40% to 80% of control event rates was conducted in one systematic review of ondansetron used for prophylaxis of PONV (18). As there was little variation in the control group in the trials included in the systematic review of ondansetron for the treatment of PONV (17), the criteria of how treatment varies with baseline risk may not be relevant. Which patients will obtain a net benefit will depend on the harms associated with the treatment. This tradeoff and how it relates to the underlying risk is discussed in more detail in the next section.
What are the Predicted Absolute Risk Reductions for Individuals?
To judge whether treatment is worthwhile for an individual patient, an appropriate NNT is estimated. This was done in one systematic review (19) using the Cook and Sackett method of applying patient’s baseline risk to the overall NNT (25).
Do the Expected Benefits Outweigh the Harms?
None of the seven systematic reviews discussed patient preferences associated with risk-benefit ratio, as the original trials did not report these data. The tradeoffs between risks and benefits can involve patient-centered outcomes, such as patient satisfaction and quality of recovery after anesthesia (26), measures that have been developed and validated recently.
Which Patients Will Benefit?
We applied the risk-benefit approach (12) to individualizing treatment as an example of how an anesthesiologist may consider a risk-benefit ratio using data from a meta-analysis and a cohort study. We show how this approach can be applied to the prophylactic use of IV ondansetron 4 mg using data from a systematic review (18). For the purposes of this example, benefit was defined as reduction in postoperative vomiting in the first 48 h and harm was defined as the incidence of headache. In clinical practice, consideration is given to all clinically important adverse effects.
Schmid et al. (27) showed that across a wide range of therapies, low-risk patients gain less absolute benefit compared with high-risk patients and as patient’s expected risk increases, the absolute risk reduction increases proportionally. We assessed the net benefit of prophylactic ondansetron by estimating the absolute risk reduction from the primary randomized controlled trials included in a systematic review (18). The overall risk reduction in postoperative vomiting associated with ondansetron was 20% (95% confidence interval [CI], 17%–24%) using a random-effects model that took into account within and between study variability. Therefore, the pooled NNT was 5 (95% CI, 4–6).
The risk of vomiting in the placebo group, an estimate of the underlying risk (x axis) was plotted against reduction in absolute risk of vomiting (y axis, plot 1) and excess absolute risk of headache (y axis, plot 2). Patients should weight the benefit and harm against each other. This can be done by assigning monetary value for the two items. In this example, the monetary value given by patients in decreasing and eliminating emesis compared with experiencing headache was a 1:1 ratio (28). The mean risk of vomiting in the control group was 56% (95% CI, 54%–59%), and the excess risk of headache was 1% (95% CI, -1% to 3%). The point at which the line of benefit and line of harm crossed was the threshold. Net benefit occurred only when the line of benefit was above the threshold of 5% (Fig. 1). It is important to note that this threshold will vary according to the weighting of benefit to harm ratio.
|
Caution is needed in assessing the relationship between treatment effect and underlying risk as it can be misleading as a result of the "regression to the mean" phenomenon as the baseline risk forms part of the definition of treatment difference (24). In another words, even if there is no true relationship between baseline risk and treatment effectiveness, one is likely to be observed because of this statistical artifact. This problem is reduced if studies are large, if variation in true underlying risks is large, and if there are a large number of studies (24,31). A Bayesian procedure for random-effects meta-analysis has been proposed (3,24) to overcome this problem. Another alternative, if individual patient data are available for meta-analysis, is to relate treatment effects to individual patient covariates (predictors of risk). This would be directly useful for the clinician considering treatment for an individual patient (24).
|
Conclusions |
|---|
|
TopIntroductionConclusionsReferences |
|---|
Systematic reviews should address the following two questions:
Depending on the availability of data from primary trials and observational prognostic studies, we believe that if the framework described is applied after considering some of the limitations, the divergence between research evidence and current clinical practice may narrow.
|
Acknowledgments |
|---|
|
References |
|---|
|
TopIntroductionConclusionsReferences |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
