JOURNAL HOME CME HOME THIS MONTH PAST ISSUES ETOC COLLECTIONS
AUTHORS REVIEWERS EDITORIAL BOARD FEEDBACK RSS HELP
A&A International Anesthesia Research Society
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a colleague
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowRequest Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (2)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Chow, T. K. F.
Right arrow Articles by McNeil, J. J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Chow, T. K. F.
Right arrow Articles by McNeil, J. J.
Related Collections
Right arrow Economics and Health Care Research
Right arrow Pain

Anesth Analg 2004;98:1557-1565
© 2004 International Anesthesia Research Society
doi: 10.1213/01.ANE.0000114071.78448.2D

PAIN MEDICINE

A Simple, Fast, Easy Method to Identify the Evidence Base in Pain-Relief Research: Validation of a Computer Search Strategy Used Alone to Identify Quality Randomized Controlled Trials

Tony K. F. Chow, FANZCA*, Elean To, MB, BS{dagger}, Colin S. Goodchild, PhD*, and John J. McNeil, PhD{ddagger}

*Department of Anaesthesia, Monash University, Monash Medical Centre, Clayton, Victoria, Australia; {dagger}Templestowe District Medical Centre, Templestowe, Victoria, Australia; and {ddagger}Department of Epidemiology and Preventive Medicine, Monash University, Alfred Hospital, Prahran, Victoria, Australia

Address correspondence and reprint requests to Tony K. F. Chow, FANZCA, Department of Anesthesia, Monash University, Monash Medical Centre, Clayton, Victoria 3168, Australia. Address e-mail to tchow{at}bbhdoa.org.au


   Abstract
Top
 Abstract
Introduction
 Methods
 Results
 Discussion
 References
 
Clinicians need a simple, fast, reliable, and inexpensive way of identifying the evidence base relevant to their clinical practice. It is often believed that the only way to identify all relevant evidence is to perform hand-searches of the literature to supplement computer searches; this is complex and labor intensive. However, most of quality randomized controlled trials cited in systematic reviews in pain medicine are listed in computer databases. We performed two studies to investigate the efficiency—in terms of sensitivity, specificity, and precision—of three computer search strategies: Optimally Sensitive Search Strategy, which is used by the Cochrane Collaboration; RCT.pt, a standard MEDLINE strategy; and DBRCT.af, which is a new single-line computer algorithm based on the assumption that double-blinded, randomized controlled trials would be indexed with "double-blind," "random," or variations of these terms in MEDLINE and EMBASE. DBRCT.af was found to be highly sensitive (97%) in identifying quality randomized controlled trials in pain medicine. The precision (ratio of randomized controlled trials to the number of nonrandomized trials identified) was 82%, and the specificity in excluding the nonrandomized controlled trials was 98%. We conclude that clinicians can now use DBRCT.af to update and conduct de novo systematic reviews in pain-relief research.

IMPLICATIONS: Quality evidence about what is good clinical practice in pain treatment is buried in the medical literature among large quantities of other information. This article describes how any clinician with access to the Internet can identify those quality studies reliably, quickly, and inexpensively.


   Introduction
Top
 Abstract
 Introduction
Methods
 Results
 Discussion
 References
 
"It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials." (1)

Good medicine is based on scientific evidence. A comprehensive database of randomized controlled trials (RCTs) in medicine would provide the essential data for evaluation of current therapy and pinpoint areas in need of further research. The Oxford University Pain Relief Unit has created a database of RCTs in pain-relief medicine. This database was created by a refined MEDLINE search strategy from 1966 to 1990 and by an exhaustive hand-search of more than 1,000,000 pages from 40 biomedical journals, encompassing all the major anesthetic/analgesic journals published from 1950 to 1990 (2). This database appears in the Cochrane Library and is maintained by the Cochrane Pain, Palliative and Supportive Care Collaborative Group (3). The database is updated with electronic searches in MEDLINE, EMBASE, and the Cochrane Library by using a combination of free text words and subject heading terms. Additional reports are identified from reference lists of retrieved reports, review articles, and specialist textbooks. This database contains a large number of potential articles that constituted the starting point from which many systematic reviews have resulted. However, hand-searching is inefficient and is not feasible on an individual basis. The Oxford Unit concedes that more than 97% of the time spent by hand-searchers was in looking at irrelevant material (2). The large effort, time, and cost of this method restrict the method to large organizations, and even these find it difficult to find the resources to update reviews that have already been performed. This labor-intensive searching method has not changed significantly over the past two decades, in which advances in information technology have allowed unprecedented access to medical information. This, along with the increase in awareness of proper indexing of RCTs, suggests that it is time to explore the possibilities of computer search strategies as the sole method of identification of RCTs in the medical literature.

However, reports on the sensitivities of computer strategies vary from 50% to 80% (4) compared with the methods involving hand-searching. It has also been found that the more sensitive the computer strategy is made, the less precise it becomes; estimated precisions range from 2% to 82% (4). The latter leads to large numbers of irrelevant articles concealing a smaller number of articles that might contribute information to a review. These observations make it difficult to justify preparing systematic reviews with a computer strategy alone.

These reports, however, do not take into account the quality of the RCTs. It has been well demonstrated that compared with double-blinded RCTs (DBRCTs), the treatment effects are exaggerated if the RCTs are unblinded or if the concealment of randomization is unclear or inadequate (5). It is generally accepted that systematic reviews should be conducted with high-quality RCTs. On the basis of this premise, it is not so important to identify all RCTs, but rather to identify all high-quality RCTs.

There is evidence that most quality RCTs are listed in computer databases. Knipschild (6) conducted an exhaustive search for vitamin C–related RCTs and found 61 eligible trials, from which only 22 were computer listed. Interestingly, after grading the trials according to methodological soundness, Knipschild concluded that there were 15 good-quality trials and that all but one were computer listed. There is a similar pattern in systematic reviews pertaining to methods of pain relief. This has led to the investigations reported here on the efficiency of computer strategies in identifying these computer-listed RCTs. Two studies were performed to define the sensitivity, precision, and specificity of three computer search strategies: the Optimally Sensitive Search Strategy (OSSS), which is used by the Cochrane Collaboration; RCT publication type (RCT.pt), which is a standard MEDLINE strategy; and DBRCTs in all fields (DBRCT.af), which is a new single-line computer algorithm based on the assumption that DBRCTs would be indexed with "double-blind," "random," or variations of these terms in MEDLINE and EMBASE.


   Methods
Top
 Abstract
 Introduction
 Methods
Results
 Discussion
 References
 
Three computer strategies were used to identify RCTs in pain-relief research. The OSSS is a comprehensive, 29-line computer strategy (Table 1). It is contained in the Cochrane Collaboration reviewers’ handbook (4,7). It was developed with the intention to cast a wide net to identify as many eligible RCTs as possible.


View this table:
[in this window]
[in a new window]
  		Table 1. Three Computer Strategies
 
The United States National Library of Medicine introduced the MEDLINE field of RCT.pt in 1991 (Table 1) (8). The RCTs are identified by trained searchers and coded appropriately into MEDLINE. RCT.pt is the only computer search strategy that has been studied and described in print. RCT.pt is contained within the OSSS and DBRCT.af (detailed below) and contributes to the sensitivity of these two computer strategies when used in MEDLINE. The RCT.pt field does not exist in EMBASE, an important electronic resource that is often used to conduct systematic reviews.

The advanced OVID search labels (Ovid Technologies© 2000–2001) within MEDLINE and EMBASE were used to develop a single-line computer algorithm (9–11): (double-blind$ or random$).af. This strategy assumes that DBRCTs will be indexed with "double-blind," "random," or variations of these terms. This algorithm (abbreviated to DBRCT.af) identifies specific terms with unlimited truncation (e.g., "double-blinded," "double-blinding," "randomly," "random allocation," "randomized," "randomised," "randomization," "randomisation," and "randomized controlled trial"). Conversely, DBRCT.af excludes study designs other than DBRCTs, RCTs, and double-blinded but non-RCTs and, hence, should not be indexed with the above-mentioned terms. Additional strategies will need to be developed to identify these other studies, which may be better to inform about rare events, diagnosis, or prognosis. DBRCT.af identifies these terms in all fields (.af). The fields of most importance are the title, abstract, subject, and publication type. The simple program shown in Table 1 creates a database of eligible RCTs in MEDLINE and EMBASE. Lines 2 and 3 limit the database to human articles; the same limitation phrases were used in OSSS. These three strategies were evaluated in two studies to determine their relative specificity, precision, and sensitivity in identifying RCTs in the field of pain medicine.

Study 1: Morphine.af Study
This study was performed to determine the specificity and precision of the OSSS and DBRCT.af. All articles published in the medical scientific literature between 1980 and 1996 relating to the use of morphine as an analgesic were extracted from EMBASE and MEDLINE by the general search algorithm morphine.af (Table 2; Lines 1–4). One-thousand of these were chosen by using the random number generation analysis tool in Microsoft Excel 2000. The list of random numbers and references can be found at the following Web address: http://www.med.monash.edu.au/anesthesia./


View this table:
[in this window]
[in a new window]
  		Table 2. Morphine-Related Articles in EMBASE and MEDLINE (1980 to 1996)
 
Hard copies of the 1000 articles were obtained. Two authors (TKFC and ET) read the methods section to classify the articles into RCTs or non-RCTs. Once this was completed, the OSSS and DBRCT.af were applied to the database containing these articles. The specificity and precision of each of the methods was then calculated as shown in the example for DBRCT.af in Table 2 (Lines 5 and 6).

Precision was defined as


Specificity was defined as


Study 2: Comparison with Published Reviews
The sensitivity of the 3 computer strategies in pain medicine was assessed by comparing the RCTs identified by the 3 computer strategies used alone, with actual work published in the field by the Oxford University Pain Relief Unit in a textbook that contains 23 quality systematic reviews (12). The Oxford Strategy used to identify articles suitable for inclusion in a systematic review consisted of computer searches through MEDLINE, EMBASE, and the Cochrane Library, supplemented by large-scale hand-searches of reference lists, reviews, textbooks, and 40 biomedical journals. All 23 systematic reviews published in the Oxford Unit’s textbook were used for this study. Reviews were included for comparison if they fulfilled the following criteria:

  1. They were published in a peer-reviewed journal.
  2. They contained only published clinical RCTs (hence, unpublished and nonclinical materials were excluded).
  3. The randomization and concealment used in the RCTs were adequate.

Hard copies of the original systematic reviews and the referenced RCTs were obtained. The referenced articles were first categorized into those listed in MEDLINE and EMBASE and those that were not. The computer-listed RCTs were then checked to see whether they were contained in the databases generated by the respective computer search strategies: that is, in Line 29 of Table 1 and in Line 3 of Table 1. These three databases were stored as citations in Reference Manager and Microsoft Excel files.

The sensitivity of each of the computer search strategies was calculated in two ways)


Finally, analyses that contained more than three RCTs were extracted from each review. The methods of analysis used by the Oxford Unit to draw their conclusions were repeated with the RCTs identified by the three computer strategies to see whether the same conclusions could be drawn. Regarding the quantitative analyses, the following data were extracted: the clinical setting, treatment groups, patient numbers, duration, mode and dose of treatment, mean and derived pain-relief outcomes (TOPAR, SPID, VASTOTPAR, VASSPID), and dichotomous pain-relief outcomes. By applying verified equations developed by McQuay and Moore (12), derived outcomes were calculated (50%maxTOTPAR) to facilitate calculations of relative risk, numbers needed to treat, and corresponding 95% confidence intervals by using a random effect model. Statistical significance was assumed in the relative risk when the 95% confidence intervals did not include 1 and in the numbers needed to treat when the 95% confidence intervals did not overlap. Regarding the qualitative analyses, the following data were extracted: the clinical setting, treatment groups, patient numbers, duration, mode and the dose of treatment, and pain outcomes. Trials were considered as positive or negative according to the original authors’ statistical estimate, with P < 0.05 considered as statistically significant. The treatment effect was considered to be present when positive trials outnumbered the negative trials. No distinctions were made between listed and nonlisted RCTs in this evaluation. Analyses containing fewer than four RCTs were deemed inconclusive.


   Results
Top
 Abstract
 Introduction
 Methods
 Results
Discussion
 References
 
Study 1: Morphine.af Study
A total of 36,780 morphine-related human articles published between 1980 and 1996 were identified by the morphine.af search of MEDLINE and EMBASE (Line 4; Table 2). The period of 1980–1996 was chosen to correspond with the Oxford morphine-related RCTs, and this set of morphine-related articles contained all of the 32 computer-listed RCTs analyzed in the 2 Oxford reviews that studied morphine RCTs (13,14). Most of the morphine-related articles (4:1) were listed in EMBASE compared with MEDLINE. This proportion was preserved in the experimental sample with 800 articles selected randomly from EMBASE and 200 from MEDLINE. The random numbers were evenly distributed to 2.7% (SE ± 0.1%) of articles published per year.

From the 1000 morphine-related articles, there were 97 RCTs and 903 non-RCTs (approximately 1 RCT per 10 articles identified; Table 3). The OSSS improved the precision to 32% (3 RCTs per 10 articles), and DBRCT.af improved the precision to 83% (8 RCTs per 10 articles). Both the OSSS and DBRCT.af were highly specific in excluding the non-RCTs. The OSSS excluded 705 (specificity 78%), while DBRCT.af excluded 886 (specificity, 98%). Conversely, the OSSS failed to exclude 198 non-RCTs compared with DBRCT.af, which failed to exclude 17 non-RCTs. Thus, this translates into 91% less work if one uses DBRCT.af to identify RCTs for a systematic review instead of OSSS, because it is better able to exclude non-RCTs. The precision and specificity of RCT.pt could not be compared with the OSSS or DBRCT.af because RCT.pt cannot be used in EMBASE and hence has a denominator of only 200 articles. Of the 17 non-RCTs that were incorrectly identified by DBRCT.af, 6 were reviews that discussed RCTs, 5 were double-blinded but nonrandomized trials, 4 were surveys that included a random sample, 1 was a letter that commented on an RCT, and 1 was a case study that was indexed with "double-blind procedure" in the key words section.


View this table:
[in this window]
[in a new window]
  		Table 3. The Specificity and Precision of the Three Computer Strategies
 
Study 2: Comparison with Published Reviews
Fifteen reviews, nine pertaining to acute pain management, (14–22) five to chronic pain management (19,23–26), and one pertaining to both (27), fulfilled the inclusion criteria (Table 4). Eight reviews were excluded: one nonpeer-reviewed publication on the effectiveness of transcutaneous electrical nerve stimulation in chronic pain and two reviews that included nonrandomized and unpublished data (28,29), along with five unpublished reviews relating to epidural corticosteroids for sciatica, spinal cord stimulators for back pain, steroid injections for shoulder disorder, the use of dihydrocodeine in postoperative pain, and the use of topical capsaicin in chronic pain. A total of 288 clinical RCTs were referenced, of which 284 (99%) were listed in MEDLINE or EMBASE and 4 were unlisted (30–33). The sensitivity in identifying all RCTs was 98%, 95%, and 65% for the OSSS, DBRCT.af, and RCT.pt, respectively (Table 4). The sensitivity in identifying MEDLINE- and EMBASE-listed RCTs was 99.6%, 96%, and 65% for the OSSS, DBRCT.af, and RCT.pt respectively (Table 4). The distribution of publication dates of those RCTs in consecutive 5-yr epochs is presented in Figure 1. This analysis shows that most of the RCTs used by the McQuay and Moore study (12) were contained within the more recent strata. The sensitivities of the 3 computer strategies in identifying computer-listed RCTs over the same 5-yr epochs are plotted in Figure 2; all three strategies improved with later epochs. It can be seen that OSSS was 100% sensitive for publications after 1970 and that DBRCT.af was >95% sensitive for publications after 1980, whereas RCT.pt remained <90% sensitive for publications in all epochs.


View this table:
[in this window]
[in a new window]
  		Table 4. Comparison of Randomized Controlled Trials (RCTs) Identified by the Oxford Strategy and the Three Computer Strategies
 


View larger version (27K):
[in this window]
[in a new window]
  		Figure 1. Randomized controlled trials identified by Oxford with respect to publication year. RCT = randomized controlled trial.

 


View larger version (12K):
[in this window]
[in a new window]
  		Figure 2. The sensitivity of the three computer strategies with respect to publication year: sensitivity (%). OSSS = Optimally Sensitive Search Strategy; DBRCT.af = double-blinded, randomized controlled trials in all fields; RCT.pt = randomized controlled trial publication type.

 
From the 15 systematic reviews, there were 30 analyses containing more than 3 RCTs (Table 5): 17 quantitative and 13 qualitative analyses. DBRCT.af was 97% sensitive in identifying 236 of the 244 MEDLINE/EMBASE-listed RCTs and was 96% (236 of 246) sensitive in identifying all the RCTs used by McQuay and Moore (12) in their reviews (range, 75%–100%; mode, 100%; 21 of 30). When the same analyses performed by McQuay and Moore were applied to the RCTs identified by DBRCT.af, the conclusions drawn were not altered significantly. Twenty-one analyses (14 quantitative and 7 qualitative analyses) drew precisely the same conclusions because DBRCT.af identified all the RCTs analyzed by the McQuay and Moore. The remaining three quantitative analyses drew the same conclusions with minor, insignificant differences in the calculated relative risks and numbers needed to treat (17, 17, and 18). All but one of the six qualitative analyses drew the same conclusions with 85%–95% of the original patient numbers (2, 15, 22, 27, and 27). In the remaining qualitative analysis, DBRCT.af identified three of the four original RCTs, and, hence, the analysis was deemed inconclusive because of the small number of trials identified (20). The data of the three quantitative and six qualitative analyses were extensive and can be viewed on our Web site: http://www.med.monash.edu. au/anesthesia./


View this table:
[in this window]
[in a new window]
  		Table 5. Comparison of Analyses with More Than Three Randomized Controlled Trials (RCTs) Between Oxford and Double Blind (DB) RCT.af
 
The OSSS identified all of the computer-listed RCTs included in the 30 analyses. As a result, 28 of the 30 analyses drew precisely the same conclusions. The remaining two analyses drew the same conclusions with 85% of the original patient numbers (Table 4) (23,27). There were insufficient data to reproduce the analyses with the RCTs identified by RCT.pt because of its poor sensitivity.


   Discussion
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
References
 
Study 1: Morphine.af Study
The first step in organizing a critical summary and periodic update of pain-relief research is to identify the relevant RCTs. Ideally, this should be done quickly and with high sensitivity and specificity. Morphine-related articles were used to test the precision and specificity of computer search strategies because morphine is one of the oldest and most commonly used analgesics worldwide. It is the "gold standard" with which all analgesics are compared, and there are many scientific publications pertaining to morphine. This study showed that 90% of the published morphine-related articles are non-RCTs. It would require a large amount of effort, time, and cost to exclude these by hand before a systematic review. To identify all eligible RCTs, it is necessary to read all of the extracted articles to exclude the non-RCTs. Computers are far more efficient in performing such monotonous tasks if the compilation of the databases can be relied on for correct classification.

The efficiency of search strategies can be measured by calculating specificity and precision. Specificity (the ability to exclude non-RCTs) is a better measure than precision (the ratio of RCTs and non-RCTs identified). Although both measures depend on the number of non-RCTs identified, the number of RCTs identified, regardless of their quality, confounds the calculated precision. For example, if one calculates precision with all eligible RCTs as the numerator (as is the usual practice), then a strategy that identifies more poor-quality RCTs will appear more precise. Conversely, if one calculates precision with the high-quality RCTs that are normally included in the systematic reviews as the numerator, then all strategies are imprecise. To the best of our knowledge, we are not aware of the specificity of a search strategy ever being reported, because the number of articles excluded has not been reported. However it is the low specificity that prevents clinicians with limited resources from performing systematic reviews. Even in institutions with large resources, most are reluctant to update reviews already published because so much labor, time, and cost is required in separating the quality trials that can go into the analysis from those that need to be excluded.

The specificity of DBRCT.af (98%) translates into substantial savings. It cost our department approximately $A10,000 to obtain the 903 non-RCTs identified in this study, to read them, and to exclude them by hand. Using the OSSS reduced this to approximately $A2,000 to obtain the 198 non-RCTs identified by that strategy, and this was reduced further to approximately $A200 for the 17 non-RCTs caught by using DBRCT.af. The effort and time spent to obtain, read, and exclude these articles are in the same proportions. Remembering that these estimates pertain to a random sample consisting only 2.7% of the 36,780 morphine-related articles, the actual savings are considerable. Having made these savings, it is important to know whether the results of the search identify enough quality RCTs, such that the subsequent systematic review can come to a meaningful conclusion. This was addressed by comparing the results of the computer searches with established high-quality systematic reviews in pain medicine.

Study 2: Comparison with Published Reviews
From the published works of the Oxford Pain Relief Unit, one may see that most RCTs (99%) used in their systematic reviews are listed in the major computer databases: EMBASE and MEDLINE. The Oxford Strategy used to identify RCTs is exhaustive, is recognized as a gold standard, and is well regarded by the Cochrane Collaboration and the International Association for the Study of Pain. The large number of systematic reviews published by McQuay and Moore provided an opportunity to evaluate the efficiency of computer search strategies used alone to identify relevant RCTs.

The results of this study showed that most computer-listed RCTs were identified solely by computer strategies such as the OSSS (99.6%) and DBRCT.af (96%), but not with RCT.pt (65%). The OSSS and DBRCT.af were sufficiently sensitive, such that when the RCTs identified by them were used to reproduce the Oxford analyses, the conclusions drawn were representative of the conclusions published by Oxford Unit. This is a very important result and leads one to ask whether it is necessary to include all RCTs to conduct a systematic review. It is true that when one conducts a clinical trial, only a sample of the entire patient population is examined. Scientific principles of randomization and blinding reduce bias sufficiently so that the studied sample is representative of the entire population. In conducting systematic reviews, it may be sufficient to accept a representative sample of RCTs rather than to strive to capture and include all of these. The results from this study provide support for this suggestion.

An important trend in the indexing of RCTs over the past 30 years was revealed in Figure 2, when the sensitivities of the three computer search strategies were compared. The sensitivity of RCT.pt from 1966 to 1997 showed the efforts by the US Library of Medicine and the Cochrane Collaboration in retrospectively indexing MEDLINE articles. In essence, the sensitivity of RCT.pt was static from 1971 to 1985 and improved thereafter. In comparison, the sensitivity of DBRCT.af progressively increased from 1966 to 1980 and remained more than 95% thereafter (Fig. 2). Although RCT.pt is contained in DBRCT.af (and OSSS) and contributes to its sensitivity, it is clear that RCT.pt did not contribute to the progressive increase in sensitivity over the 1974 to 1985 period. This period can be thought of as the "pre-Cochrane era," that is, before Cochrane criticized the medical profession for not using RCTs effectively. The progressive increase was more likely because authors who conducted RCTs recognized the need to index their publications with the appropriate terms. The DBRCT.af strategy was developed with the assumption that DBRCTs would be indexed with these terms. Despite the efforts in retrospective indexing, the sensitivity of RCT.pt did not reach 90%. In part, this may be because RCT.pt is limited to searching MEDLINE. It is also likely that retrospective indexing does not identify all RCTs. Other publication-type strategies, such as controlled clinical trial.pt and clinical trial.pt, were not investigated (8). These two strategies are similar to RCT.pt in that they rely on trained searchers to code the articles appropriately. Unlike RCT.pt, however, they are not contained in DBRCT.af. Because DBRCT.af was highly sensitive and specific, it did not seem relevant to investigate these two strategies.

In conclusion, the two studies reported in this article demonstrate that the OSSS and DBRCT.af are highly efficient compared with the Oxford Strategy. Their routine use can lead to considerable savings in effort, time, and cost. In light of these results, we recommend that these computer strategies be used to periodically update the 15 Oxford reviews. The consistent efficiency demonstrated in these quality reviews of wide-ranging topics suggests that this may apply throughout the specialty of pain-relief research. We also believe that a sufficient number of quality RCTs can be identified by the OSSS or DBRCT.af to conduct de novo systematic reviews to give reasonably robust answers in the efficacy of pain-relief treatments. In considering poorly reported data, such as adverse events, one needs to be cautious in drawing conclusions with limited information. A reasonable approach may be to search with DBRCT.af and progress to more extensive strategies if more information is required. Further work is required to confirm that the conclusions drawn by using computer strategies alone are representative of the true clinical picture.

The application of DBRCT.af in medical specialties other than pain relief would require validation. In ophthalmology, for example, the term "masked" is preferred over "blinded." In this regard, DBRCT.af would rely on identifying the term "random" or variations thereof to identify DBRCTs. In contrast to the OSSS, DBRCT.af was designed not to include all possible terms. It was intended that in maintaining the simplicity of DBRCT.af, the message to improve the indexing of articles filters through as researchers read and conduct reviews with DBRCT.af.


   Acknowledgments
 
We acknowledge the Oxford University Pain Relief Unit for conducting the systematic reviews, without which this article would not be possible. We thank OVID Technology for their technical assistance. We also thank Professor Chris Silagy (since deceased) and Steve McDonald of the Institute of Public Health & Human Services Research, Monash University, Monash Medical Centre, for their invaluable support and advice.


   References
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 

  1. Cochrane AL. 1931-1971: a critical review, with particular reference to the medical profession. In: Medicines for the year 2000. London: Office of Health Economics, 1979: 1–11.
  2. Jadad AR, Carroll D, Moore A, McQuay H. Developing a database of published reports of randomised clinical trials in pain research. Pain 1996; 66: 239–46.[Web of Science][Medline]
  3. The Cochrane Collaboration. Cochrane Hand Search Manual. Oxford: The Cochrane Library, 2000.
  4. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ 1994; 309: 1286–91.[Abstract/Free Full Text]
  5. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273: 408–12.[Abstract/Free Full Text]
  6. Knipschild P. Systematic reviews: some examples. BMJ 1994; 309: 719–21.[Free Full Text]
  7. Clarke M, Oxman AD, eds. The Cochrane library issue 2, 2002: Cochrane reviewers’ handbook 4.1.5. Oxford: The Cochrane Library, 2002.
  8. McDonald SJ, Lefebvre C, Clarke MJ. Identifying reports of controlled trials in the BMJ and the Lancet. BMJ 1996; 313: 1116–7.[Free Full Text]
  9. Chow T, To E, Goodchild C, McNeil J. Improved indexing of randomised controlled trials has enabled computer search strategies to identify them with high sensitivity and specificity, in pain relief research. Conference Proceedings of the 8th International Cochrane Colloquium, Cape Town, South Africa. 2000:41–2.
  10. Chow TKF, To E, Goodchild CS, McNeil JJ. Computer search strategies are efficient in identifying the quality randomised controlled trials in pain relief research. Anaesth Intensive Care 2001; 29: 658.
  11. Chow TKF, Goodchild CS, To E. A simple, sensitive and precise strategy (Strategy-DR) to identify randomised controlled trials in medicine. Anaesth Intensive Care 1999; 27: 77.
  12. McQuay H, Moore A. An evidence-based resource for pain relief. Oxford: Oxford University Press, 1998.
  13. Kalso E, Tramer MR, Carroll D, et al. Pain relief from intra-articular morphine after knee surgery: a qualitative systematic review. Pain 1997; 71: 127–34.[Web of Science][Medline]
  14. McQuay HJ, Carroll D, Moore RA. Injected morphine in postoperative pain: a quantitative systematic review. J Pain Symptom Manage 1999; 17: 164–74.[Web of Science][Medline]
  15. Carroll D, Tramer M, McQuay H, et al. Randomization is important in studies with pain outcomes: systematic review of transcutaneous electrical nerve stimulation in acute postoperative pain. Br J Anaesth 1996; 77: 798–803.[Abstract/Free Full Text]
  16. Carroll D, Tramer M, McQuay H, et al. Transcutaneous electrical nerve stimulation in labour pain: a systematic review. Br J Obstet Gynaecol 1997; 104: 169–75.[Web of Science][Medline]
  17. Collins SL, Moore RA, McQuay HJ, Wiffen PJ. Oral ibuprofen and diclofenac in post-operative pain: a quantitative systematic review. Eur J Pain 1998; 2: 285–91.[Medline]
  18. Collins SL, Edwards JE, Moore RA, McQuay HJ. Single-dose dextropropoxyphene in post-operative pain: a quantitative systematic review. Eur J Clin Pharmacol 1998; 54: 107–12.[Web of Science][Medline]
  19. Kalso E, Tramer MR, McQuay HJ, Moore RA. Systemic local-anaesthetic-type drugs in chronic pain: a systematic review. Eur J Pain 1998; 2: 3–14.[Web of Science][Medline]
  20. McQuay HJ. Pre-emptive analgesia: a systematic review of clinical studies. Ann Med 1995; 27: 249–56.[Web of Science][Medline]
  21. Moore A, Collins S, Carroll D, McQuay H. Paracetamol with and without codeine in acute pain: a quantitative systematic review. Pain 1997; 70: 193–201.[Web of Science][Medline]
  22. Picard PR, Tramer MR, McQuay HJ, Moore RA. Analgesic efficacy of peripheral opioids (all except intra-articular): a qualitative systematic review of randomised controlled trials. Pain 1997; 72: 309–18.[Web of Science][Medline]
  23. Jadad AR, Carroll D, Glynn CJ, McQuay HJ. Intravenous regional sympathetic blockade for pain relief in reflex sympathetic dystrophy: a systematic review and a randomized, double-blind crossover study. J Pain Symptom Manage 1995; 10: 13–20.[Web of Science][Medline]
  24. McQuay H, Carroll D, Jadad AR, et al. Anticonvulsant drugs for management of pain: a systematic review. BMJ 1995; 311: 1047–52.[Abstract/Free Full Text]
  25. McQuay HJ, Tramer M, Nye BA, et al. A systematic review of antidepressants in neuropathic pain. Pain 1996; 68: 217–27.[Web of Science][Medline]
  26. McQuay HJ, Collins SL, Carroll D, Moore RA. Radiotherapy for the palliation of painful bone metastases. Cochrane Database Syst Rev 2000; CD001793.
  27. Tramer MR, Williams JE, Carroll D, et al. Comparing analgesic efficacy of non-steroidal anti-inflammatory drugs given by different routes in acute and chronic pain: a qualitative systematic review. Acta Anaesthesiol Scand 1998; 42: 71–9.[Web of Science][Medline]
  28. Moore RA, McQuay HJ. Single-patient data meta-analysis of 3453 postoperative patients: oral tramadol versus placebo, codeine and combination analgesics. Pain 1997; 69: 287–94.[Web of Science][Medline]
  29. Moore RA, Tramer MR, Carroll D, et al. Quantitative systematic review of topically applied non-steroidal anti-inflammatory drugs. BMJ 1998; 316: 333–8.[Abstract/Free Full Text]
  30. Amanor-Boadu SD, Jadad AR, Glynn CJ, et al. Influence of pre and postoperative morphine administration on pain after body surface surgery: a double blind randomised controlled study. Proceedings of the 7th World Congress on Pain. 1993:538–9.
  31. Beaver WT, Feise G. Comparison of the analgesic effects of morphine, hydroxyzine, and their combination in patients with postoperative pain. Adv Pain Res Ther 1976; 1: 553–7.
  32. Dhar S, McGlone F, Dean J, Klenerman L. The temporal significance of the placebo response in reflex sympathetic dystrophy. Proceedings of the British Orthopaedic Research Society Meeting 1992;16:16–7.
  33. Moore AP, Thorpe JA, Mulley BA, Cruise M. Which diclofenac form is best? Post-thoracotomy pain: intramuscular v. rectal administration of diclofenac. Hosp Pharm Pract. 1993; September: 431–5.
Accepted for publication December 5, 2003.




This article has been cited by other articles:


Home page
 Anesth. Analg.Home page
T. K.F. Chow
It Is the 30th Birthday of "Evidence Based Medicine" and Time to Meet Cochrane's Challenge
Anesth. Analg., May 1, 2009; 108(5): 1720 - 1721.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a colleague
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowRequest Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (2)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Chow, T. K. F.
Right arrow Articles by McNeil, J. J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Chow, T. K. F.
Right arrow Articles by McNeil, J. J.
Related Collections
Right arrow Economics and Health Care Research
Right arrow Pain

Lippincott, Williams & Wilkins Anesthesia & Analgesia® is published for the International Anesthesia Research Society® by Lippincott Williams & Wilkins and Stanford University Libraries' HighWire Press®. Copyright 2004 by the International Anesthesia Research Society. Online ISSN: 1526-7598   Print ISSN: 0003-2999 HighWire Press