| JOURNAL HOME | CME HOME | THIS MONTH | PAST ISSUES | ETOC | COLLECTIONS |
| AUTHORS | REVIEWERS | EDITORIAL BOARD | FEEDBACK | RSS | HELP |
| ||||||||||||||
| ||||||||||||||
|
|
|
|
||||||||||||
|
||||||||||||||
Virginia Commonwealth University Richmond, VA 23221
To the Editor:
In Greek mythology, Procrustes was a robber, and as the story goes, would kidnap his victims and place them on his bed. Obsessed in his effort to make them fit, he would stretch or alternatively amputate their legs at the desired level. Dr. Laurent Glance creates a Procrustean bed for himself (and the reader) in his attempt to evaluate the economics of provider mixture in his recent paper examining the economics of provider mix (1). Here, he seems intent on deciding on the hypothesis to be proved and making the data fit that hypothesis. I wish to address a few points in questioning the validity of the model that Dr. Glance used, as well as his analysis.
First, the notion that nurse anesthetists working alone result in worse patient outcomes is not only advanced but also immutably embraced by the model. In doing so, Dr. Glance builds a house of cards for the scientific framework of his model. His model is based on the assumption that certified registered nurse anesthetists (CRNAs) have twice the mortality of nonboard-certified anesthesiologists and a 5 times higher mortality rate than that of solo, board-certified anesthesiologists. Despite an intensive review of the literature, I could not find a single, prospective, systematically performed study to support this. My review seems supported by Dr. Glance who writes, "it may not be possible to devise adequately powered studies to examine the effect of provider mix on anesthesia outcome" (p. 588) and, "there are no randomized, controlled trials evaluating patient outcome as a function of provider mix" (p. 587). How could the peer reviewers of Anesthesia & Analgesia permit Dr. Glance to write, "In the absence of outcome studies comparing mortality as a function of anesthesia provider, baseline assumptions were developed using the available literature" (p. 585)? This appears to violate a number of the foundations of good science, including the failure to minimize researcher bias.
The reader is asked to accept that models that call for greater use of CRNAs may achieve overall cost reduction but that "the use of unsupervised or minimally supervised CRNAs may also result in worse patient outcomes" (p. 585). At best, this statement is speculative, at worst it corrupts the scientific literature. The few studies urged on us are retrospective, suffer major design limitations, and use outcome measures (such as failure to rescue) that may be insensitive or held hostage to modifiers that have researchers (Dr. Jeffrey Silber included) looking elsewhere for the Holy Grail.
The second point that I would like to make has to do with the staffing models used in the analysis. Without a clear demarcation as to what exactly each provider contributed to both the outcome and expense of each case, it seems difficult to draw meaningful conclusions. The assignment of the cases to the model mixes was determined a priori by factors other than objective clinical or cost criteria. This is quite acceptable if the sole purpose of the study was to describe average costs of the various mixtures. However, the model does not allow one to predict cost effectiveness, much less outcome. Obvious to enlightened readers is that the outcome measure used—mortality—simply does not meet the requirements of a reasonable measure (2), nor does it possess construct validity in this setting. Why would Dr. Glance select five models in which only physicians anesthetize the "high-risk" patients? Why would the editors permit modeling so at odds with real-world practice?
Cost effectiveness analysis is useful in quasi-simulation approaches in which controlled trials are difficult; however, the attendant sensitivity analysis must be based on reasonable, real-world assumptions to explore the implications of actually having data that might be very different from what is assumed. The assumptions on which Dr. Glance bases his model and on which his sensitivity analysis is predicated is akin to wishing that cows could fly in a study of the migratory patterns of farm animals.
There are other issues that degrade the scientific value of this paper. For example, the use of "primary" and "secondary" mortality rates are of concern. Implicit in the care of patients is the idea that surgical mortality is often the result of multiple factors. Separating primary (anesthesia) factors from secondary (patient driven) factors is fret with pitfalls and has been a Beelzebub in virtually all of the outcomes-based literature. Well known to anesthesia-related outcomes researchers, mortality alone is an inadequate measure. Furthermore Dr. Glance’s use of adjusted life-years seems entirely arbitrary. Dr. Glance acknowledges this himself noting "there is no data on anesthesia related morbidity as a function of provider mix to derive this measure of effect" (p. 586).
Dr. Glance’s paper seems to be "proof by proclamation" rather than "proof by experimentation." In the parlance of the academic research community, this work is less an example of a type I (false rejection of the null hypothesis) or type II (false acceptance of the null hypothesis) error; rather, it is a classic type III error (the wrong experiment was performed). In creating a Procrustean bed for himself and the reader, Dr. Glance fails in his attempt to illuminate a previously shadowed area of understanding, achieving only to cloud the view in a shroud of politic.
References
Department of Anesthesiology University of Rochester Rochester, NY 14642
In Response:
Both authors criticized this article primarily on the basis of the outcomes data used to analyze the cost-effectiveness of anesthesia workforce models using different ratios of physicians and certified registered nurse anesthetists (CRNAs). One of the purposes of this simulation was to frame the discussion evaluating anesthesia staffing scenarios along economic lines to "help guide the debate regarding the optimal staffing mix of physicians and CRNAs delivery anesthesia care," (1) without resorting to rhetoric. Mr. Biddle’s references to a Procrustean bed, house of cards, the Holy Grail, flying cows etc. are entertaining, but needlessly inflammatory.
To respond to the specifics of these letters, it is helpful to give further background on decision analysis: First, "decision analysis is useful when the clinical or policy decision is complex and information is uncertain" (2). The issues addressed by this study are clearly complex and the policy conclusions uncertain.
Second, the information used in a decision analysis can be based on "literature review, including meta-analysis, primary data collection, and consultation with experts" (2). The article clearly states that "there are no randomized, controlled trials evaluating patient outcome as a function of provider mix" (1). As described in the article, the difference in outcomes between CRNAs, either working alone or as part of an anesthesia care team were extrapolated from a study examining the incidence of "failure-to-rescue" among board-certified anesthesiologists versus nonboard-certified anesthesiologists. The authors of the letters are entitled to challenge these baseline assumptions. The Discussion section further emphasizes "the absence of reliable data on anesthesia outcomes as a function of skill mix" as a "significant limitation of this model."
Third, sensitivity analysis is used to "compare the stability of the conclusions of the analysis to the assumptions made in the analysis" (2). Sensitivity analysis was included in this simulation to take into account the uncertainty inherent in the baseline assumptions.
In conclusion, policy analysis is extremely difficult without some rational basis for exploring the consequences of alternative approaches. Both letters are correct in that there are no prospective data assessing the effect of skill mix on anesthesia outcomes. As described in my article, the number of patients necessary to construct an adequately powered trial is prohibitively large. Alternatively, logistic regression could be used to examine the relative risk of mortality as a function of provider mix after adjusting for severity of disease and procedure risk. However, it may not be possible to adequately perform the necessary risk-adjustment given the extremely low incidence of adverse outcomes. Therefore, given what we know, and the need to critically evaluate current and future manpower utilization in anesthesiology, it seems reasonable to use decision analysis as a tool for exploring this complex issue.
References
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
