JOURNAL HOME CME HOME THIS MONTH PAST ISSUES ETOC COLLECTIONS
AUTHORS REVIEWERS EDITORIAL BOARD FEEDBACK RSS HELP
A&A International Anesthesia Research Society
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow En Espanol
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a colleague
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (8)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Hernández-Borges, A. A.
Right arrow Articles by Ormazábal-Ramos, C.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Hernández-Borges, A. A.
Right arrow Articles by Ormazábal-Ramos, C.
Anesth Analg 1999;89:520
© 1999 International Anesthesia Research Society

ECONOMICS AND HEALTH SYSTEMS RESEARCH

Assessing the Relative Quality of Anesthesiology and Critical Care Medicine Internet Mailing Lists

Angel A. Hernández-Borges, MD*, Pablo Macías-Cervi, MD{dagger}, M. Asunción Gaspar-Guardado, MD{dagger}, M. Luisa Torres-Álvarez de Arcaya, MD{ddagger}, Ana Ruiz-Rabaza, MD§, and Carlos Ormazábal-Ramos, MD*

*Pediatric Intensive Care Unit, {dagger}Department of Family and Community Medicine, Hospital Universitario de Canarias; and {ddagger}Primary Care Center of La Orotava, §Research Unit, Hospital Nuestra Señora de La Candelaria, Tenerife, Canary Islands, Spain

Address correspondence and reprint requests to Dr. Hernández-Borges, C/Valencia, 21, 3°, 38005 Santa Cruz de Tenerife, Tenerife (Canary Islands), Spain. Address e-mail to borges{at}redkbs.com


   Abstract
Top
 Abstract
Introduction
 Methods
 Results
 Discussion
 References
 
We studied the relative quality of a subset of anesthesiology and critical care medicine Internet mailing lists regarding the publishing capacity of their members to compare them with the major journals and conferences regarding these specialties. Using systematic searches on MEDLINE and according to the Science Citation Index 1995, we investigated the impact factor of mailing list subscribers, of the first authors of the selected articles, and of the first authors of published abstracts from conferences. We studied six mailing lists, seven journals, and four conferences. Journals and conferences showed a higher percentage of published authors and higher average impact factor among their first authors than the mailing lists did per subscriber. However, when only the subset of publishing authors from the three media was considered, no significant differences were found. We conclude that qualified authors may be found among the subscribers of Internet medical mailing lists on anesthesiology and critical care medicine. These professional discussion groups could complement peer-reviewed publications and conferences in professional information exchange and continuing medical education.

Implications: Internet publishing is not governed by rules that assure certain basic quality standards. Methods for assessing these standards are needed. We compared discussion groups with medical journals and conferences on anesthesiology and critical care medicine by calculating the impact factor of their members and first authors, respectively. Our study shows that qualified authors may be found in all three media.


   Introduction
Top
 Abstract
 Introduction
Methods
 Results
 Discussion
 References
 
Some authors have voiced concern about the validity of certain medical resources on the Internet after finding that some residents and fellows may have been misinformed by using unreviewed documents on the Internet instead of other, more academic sources of information, such as medical journals and textbooks (1,2). There is also a concern that unreviewed health information could bring potential risks to lay people (3,4).

Internet publishing is not governed by uniform rules. In fact, any information with certain multimedia capabilities could attract many people, whatever its accuracy. Therefore, it would be useful to have methods of evaluating medical resources on the Internet. Many review articles about medical resources on the Internet have been published over the last years, but only a few have tried to assess quality (5,6). Some journals have introduced World Wide Web review sections in which selected medical resources on the Internet are presented to users (7). Finally, there are some recognized approaches of ranking medical resources on the Internet (8), but their feasibility has not been demonstrated.

Professional medical mailing lists are an important subset of the medical resources on the Internet (9–14). These are nonacademic forums in which members or subscribers focus their discussions on a variety of issues via electronic mail. In contrast to classic congresses, mailing lists allow the exchange of professional information immediately and inexpensively. Only an electronic mail account is needed to contact other members of an unmoderated mailing list. The democracy of these groups allows the subscribers unlimited participation, except for certain agreed-upon rules, or "netiquette." Moreover, barriers such as having to have an academic degree and paying high costs for attending classic conferences may be avoided. Therefore, these groups could be a valuable tool for many physicians from developing countries to obtain professional advice and could complement other academic resources in continuing medical education. However, there is no peer-review process that guarantees validity to on-line commentaries. Finally, for some authors, adopting the opinions and practices generated in Internet discussion groups in the clinical setting might imply legal responsibilities for the subscribers who post such opinions (15).

One way to determine the validity of the commentaries of mailing list subscribers could be to evaluate their scientific production in peer-reviewed journals. The aim of the present study was to compare the scientific production of the subscribers who posted to a subset of mailing lists on anesthesiology and critical care medicine with that of authors in major journals. For some authors, the discussions on these electronic forums would be the same as those among colleagues in a daily conference (16,17). Therefore, we also compared these mailing lists with a subset of international conferences on these medical specialties.


   Methods
Top
 Abstract
 Introduction
 Methods
Results
 Discussion
 References
 
We analyzed a subset of Internet mailing lists dealing with anesthesiology and critical care medicine from December 1996 to February 1997. Those with significant activity among physicians were selected: AIC-L, ANESTHESIOLOGY, CCM-L, LARA-L, PAC, and Pediatric Pain (Table 1). The discussions generated on-line were registered. The impact factor of each subscriber who posted after searching in the 1995 CD of MEDLINE SilverPlatter database (SilverPlatter, Boston, MA) was calculated using the 1995 Science Citation Index (Institute for Scientific Information, Philadelphia, PA). In addition, some other indexes were calculated, including:


View this table:
[in this window]
[in a new window]
  		Table 1. Mailing Lists on Anesthesiology and Critical Care Medicine Included in This Studya
 
1. The activity of the lists.
2. The percentage of subscribers who posted or percentage of participants.
3. The percentage of participants who published in 1995.
4. The average impact factor per subscriber who posted (IFp). That is, the cumulative impact factor of the participants on discussions divided by the total number of people posting messages.
5. The average impact factor per posting (IFe). That is, the sum of the individual impact factor of participants on the discussions multiplied by the number of postings of each individual. This sum was then divided by the total number of postings during the period of observation.
6. IFe multiplied by the activity of each list provided a yield index (YI) for each mailing list.

We calculated the average impact factor of the total number of subscribers on each mailing list (IPQ), as a measure of their potential quality, by searching a randomized representative sample of each mailing list subscriber on MEDLINE. Their cumulative impact factor divided by the number of searched individuals gave us the IPQ.

Major anesthesiology and critical care medicine journals with an impact factor >=1.9 were selected to compare them with the mailing lists. We analyzed the December 1996 issues of Anesthesiology, Anesthesia & Analgesia, The British Journal of Anaesthesia, Anaesthesia, Pain, The American Journal of Respiratory and Critical Care Medicine (Critical Care Section), and Critical Care Medicine. We then calculated the average impact factor per first author of the articles and short communications (IF-a) by searching MEDLINE as described above.

We selected four 1996 international anesthesiology and critical care medicine conferences: the 1996 annual meeting of the ASA, the 70th Clinical and Scientific Congress of the International Anesthesia Research Society, the European Society of Anaesthesiologists Annual Congress, and the 26th Educational and Scientific Symposium of the Society of Critical Care Medicine. A randomized representative sample of the first authors of the published abstracts of the conferences (18–21) were searched on MEDLINE as described above. We calculated their average impact factor (IF-au) to compare them with the journals and the mailing lists.

The size of the representative sample for each mailing list subscriber and each conference first author was calculated using a previously described statistical method (6).

Correlation studies (Spearman’s coefficient, r2) and comparisons of means among groups were made. All results are expressed as mean ± SD. P values <=0.05 were considered significant. Computations were made by using SPSS 7.0 for Windows (SPSS Inc., Chicago, IL).


   Results
Top
 Abstract
 Introduction
 Methods
 Results
Discussion
 References
 
The results of the study of the discussions generated on-line are summarized in Table 2. The average observation period was 30 ± 17 days. The most active list was ANESTHESIOLOGY (39 messages per day). The average participation was 11% (5% for AIC-L to 22% for LARA-L). On average, 21% of subscribers who posted had published at least one article in medical peer-reviewed journals in 1995 (14% for Pediatric Pain to 25% for AIC-L), and their IF-al was 3.8. The IF-p was 0.9. LARA-L had the highest IFe (of 2.2). Finally, ANESTHESIOLOGY had the highest YI (of 50).


View this table:
[in this window]
[in a new window]
  		Table 2. Results of the Study of Postings to On-Line Discussions
 
When the subscribers who posted were classified by the number of postings they sent on-line, no significant differences were found among their average impact factors (data not shown).

The study of the journals (Table 3) showed that a high percentage of first authors of the selected articles had published medical articles in 1995 (58% ± 19%). Their IF-aj was 5.6. The average IF-a was 3.3.


View this table:
[in this window]
[in a new window]
  		Table 3. Study of the Major Journals on Anesthesiology and Critical Care Medicine
 
The results of the study of the conferences are summarized in Table 4. Of the first authors of abstracts, 45% had published in 1995, and their average impact factor (IF-ac) was 4.7. The average IF-a (of 2.2) significantly correlated with the impact factor of the journals in which the abstracts were published (P < 0.001).


View this table:
[in this window]
[in a new window]
  		Table 4. Study of Some 1996 International Conferences on Anesthesiology and Critical Care Medicine
 
The percentage of publishing authors among the subscribers who posted on the mailing lists was significantly lower than that among the first authors of the selected articles (P < 0.001) and among the first authors of the conference abstracts (P = 0.005).

The average IF-p was significantly lower (P = 0.004) than the average IF-a. Similarly, IF-p was lower than the average IF-au, but there was no statistically significant difference.

When only the subset of publishing authors from 1995 of each media was considered, their average impact factors did not show significant differences among the mailing lists (IF-al), the journals (IF-aj), and the conferences (IF-ac) (Fig. 1).



View larger version (47K):
[in this window]
[in a new window]
  		Figure 1. Average impact factor per mailing list participant (IF-al) versus average impact factor per first authors from the articles (IF-aj) and conferences abstracts (IF-ac), when only authors who published in 1995 were considered (expressed as mean ± SEM). Mailing lists showed lower average impact factors than the other media, but there was no statistically significant difference.

 
The results of the study of the mailing list subscribers are summarized in Table 5. Of the total of subscribers, 15% were publishing authors (13% for Pediatric Pain to 18% for LARA-L), significantly less than among the subscribers who posted on-line (P = 0.02). The average impact factor of the subscribers was 0.6 (0.4 for Pediatric Pain to 0.8 for CCM-L), which is lower than the average IF-p, but there was no statistically significant difference.


View this table:
[in this window]
[in a new window]
  		Table 5. Results of the Study of Mailing List Subscribers
 

   Discussion
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
References
 
In a previous article, we compared the quality of a subset of pediatric mailing lists with that of some pediatric journals (6). In the present study, we compared six mailing lists on anesthesiology and critical care medicine with not only the major journals of these specialties, but also four international conferences. The journals and the conferences showed higher percentages of authors who published in 1995 and a higher IF-aj and IF-au than IF-al. However, our results demonstrate that authors with a similar IF-a may be found among the three media, although in a lower percentage on the Internet mailing lists. Finally, IPQ values (Table 5) were always lower than IF-p (Table 2); that is, the discussions on every mailing list were carried by a subset of subscribers with a high publishing capacity.

When we compared the pediatric mailing lists from the previous work (6) with the mailing lists on anesthesiology and critical care medicine, we did not find significant differences in the percentage of authors among the subscribers who posted in the IPQ or IF-p.

In this article, we tried to assess the relative quality of the most popular anesthesia and critical care medicine resources on the Internet by using the calculation of the average impact factor of the mailing list subscribers. The impact factor, although criticized (22), is a classic tool for ranking medical journals (23). Its calculation is based on the citation analysis of those peer-reviewed journals, and it is obtained from dividing citations received in 1 yr by articles published in the 2 previous yr (24). We believe that our method could help to inform new subscribers about participants’ prestige as publishing authors on a given medical mailing list. In any case, a critical reading of participants’ commentaries is probably the best tool to ascertain the value of the group. However, this is not an efficient method for assessing other Internet peculiarities, such as immediacy, multimedia capabilities, and interactivity. The quality of Internet medical resources must be assessed by using specific methods.

The reliability of Internet medical resources is a current question. Actual concerns about the hazards of the Internet might be clarified by using methods that allowed the lay or professional public to rank its resources according to relative quality. LaPorte et al. (25) proposed the quantification of the retrieval of medical documents on the Internet as a possible method for assessing their quality. Just as not every article will be used as a reference, not every retrieved document on the Internet must be accepted as accurate. Therefore, the second method proposed by LaPorte et al. (25) seems a more reliable method; that is, the analysis of the citation of Internet medical resources both on the Internet and in other media.

For years, medical journals have been included in a ranking system based on the analysis of their citation. A similar approach could be attempted for electronically published articles, so that a ranking of Internet medical resources based on their citation analysis could serve as a guide for new users. An important point is the acceptance of documents from the electronic media in the printed journals. In fact, the most recent requirements for manuscripts submitted to biomedical journals establish the citation style for articles from reviewed electronic journals (26).

In summary, Internet mailing lists on anesthesiology and critical care medicine are a valuable tool in information exchange among physicians. These resources may be a complement to medical journals and medical conferences as continuing education tools. However, Internet publishing is a unique publishing medium, and more work is needed to achieve specific methods for evaluating quality.


   Acknowledgments
 
We acknowledge support from the staff of the Research Unit of the Hospital Universitario de Canarias. We especially thank Dr. Reina García Closas for her help in the statistical analysis and Dr. Armando Torres for his help and advice. We thank Ms. Chari Hernández-Borges for her help in the translation of this manuscript. We thank mailing list editors for providing us with data about the lists: Dr. A. J. Wright (ACCRI), Dr. P. Cumpston (AIC-L), Dr. K. Ruskin (ANESTHESIOLOGY), Dr. D. Crippen (CCM-L), Dr. J. Oyston (LARA-L), Dr. J. Tibbals (PAC), and Dr. G. A. Finley (Pediatric Pain).


   References
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 

  1. Shapira Y, Talmor D, Artru AA, Lam AM. Resident education and unreviewed material. Anesth Analg 1996;83:886–91.
  2. Lief PD. The Internet and the Journal [letter]. N Engl J Med 1995;333:1077.[Free Full Text]
  3. Sonnenberg FA. Health information on the Internet: opportunities and pitfalls. Arch Intern Med 1997;157:151–2.[ISI][Medline]
  4. Wiviott LD. The Internet and the Journal [letter]. N Engl J Med 1995;333:1078.
  5. Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home. BMJ [serial online] 1997;314:1875–9. Available at: http://www.bmj.com/archive/7098ip1.htm. Accessed November 17, 1998.
  6. Hernández-Borges AA, Pareras LG, Jiménez A. Comparative analysis of pediatric mailing lists on the Internet. Pediatrics [serial online] 1997;100:E8. Available at: http://www.pediatrics.org/cgi/content/full/100/2/e8. Accessed November 17, 1998.
  7. Eisenach JC. Web site reviews: a new feature. Anesthesiology 1997;86:510.
  8. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: caveant lector et viewor—let the reader and viewer beware. JAMA 1997;277:1244–5.[ISI][Medline]
  9. Kuppersmith RB, Holsinger C, Jenkins HA. The use of e-mail by otolaryngologists. Surg 1996;122:921–2.
  10. Meachan RB, Niederberger CS. Use of a moderated international Internet information exchange in the study of male reproduction. Urology 1996;48:3–6.[ISI][Medline]
  11. Tarczy-Hornoch P. NICU-Net: an electronic forum for neonatology. Pediatrics 1996;97:398–9.[Abstract/Free Full Text]
  12. Pallen M. Electronic mail. BMJ 1995;311:1487–90.[Free Full Text]
  13. Ruskin KJ, Tissot M. A new method of communication between anesthesiologists. Anesthesiology 1993;79:867.
  14. Ruskin KJ, Kofke A, Turndorf H. The anesthesiology discussion group: development of a method of communication between anesthesiologists. Anesth Analg 1995;81:163–6.[ISI][Medline]
  15. Elliott SJ, Elliott RG. Internet listservers and pediatrics: newly emerging legal and clinical practice issues. Pediatrics 1996;97:399–400.[Abstract/Free Full Text]
  16. Pareras LG, ed. Internet y medicina. Barcelona: Masson S.A., 1996.
  17. De Ville KA. Internet listservers and pediatrics: newly emerging legal and clinical practice issues II. Pediatrics 1996;98:453–4.[Abstract/Free Full Text]
  18. Scientific Papers. 1996 annual meeting of the American Society of Anesthesiologists. Anesthesiology 1996;85:A1–1344.[ISI][Medline]
  19. Abstracts to the 70th Clinical and Scientific Congress of the International Anesthesia Research Society. Anesth Analg 1996;82:S1–567.[ISI][Medline]
  20. Abstracts to the European Society of Anaesthesiologists Annual Congress. Br J Anaesth 1996;76:A1–418.
  21. Abstracts to the 26th Educational and Scientific Symposium of the Society of Critical Care Medicine. Crit Care Med 1997;25:A1–153.
  22. Hansson S. Impact factor as a misleading tool in evaluation of medical journals [letter]. Lancet 1995;346:906.
  23. Garfield E. Citation analysis as a tool in journal evaluation. Science 1972;178:471–9.[Free Full Text]
  24. Garfield E. How can impact factors be improved? BMJ 1996;313:411–3.[Free Full Text]
  25. LaPorte RE, Marler E, Akazawa S, et al. The death of the biomedical journals. BMJ 1995;310:1387–90.[Free Full Text]
  26. International Committee of Medical Journals Editors.Uniform requirements for manuscripts submitted to biomedical journals. JAMA 1997;277:927–34.[Abstract]
Accepted for publication March 9, 1999.




This article has been cited by other articles:


Home page
 Anesth. Analg.Home page
K. Ruskin, J. Oysten, and A. A. Hernandez-Borges
Quality of Medical Information on the Internet Response
Anesth. Analg., May 1, 2000; 90(5): 1246 - 1246.
[Full Text] [PDF]

Home page
 Anesth. Analg.Home page
M. R. Tramer
How Can We Cope with the Internet?
Anesth. Analg., August 1, 1999; 89(2): 271 - 271.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow En Espanol
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a colleague
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (8)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Hernández-Borges, A. A.
Right arrow Articles by Ormazábal-Ramos, C.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Hernández-Borges, A. A.
Right arrow Articles by Ormazábal-Ramos, C.

Lippincott, Williams & Wilkins Anesthesia & Analgesia® is published for the International Anesthesia Research Society® by Lippincott Williams & Wilkins with the assistance of Stanford University Libraries' HighWire Press®. Copyright 2006 by the International Anesthesia Research Society. Online ISSN: 1526-7598   Print ISSN: 0003-2999 HighWire Press