| JOURNAL HOME | CME HOME | THIS MONTH | PAST ISSUES | ETOC | COLLECTIONS |
| AUTHORS | REVIEWERS | EDITORIAL BOARD | FEEDBACK | RSS | HELP |
| ||||||||||||||
| ||||||||||||||
|
|
|
|
||||||||||||
|
||||||||||||||
Department of Anesthesiology, The Mount Sinai School of Medicine, New York, New York
Address correspondence and reprint requests to Meg A. Rosenblatt, MD, Clinical Associate Professor of Anesthesiology, The Mount Sinai School of Medicine, Department of Anesthesiology, Box 1010, One Gustave L. Levy Place, New York, NY 10029-6574. Address e-mail to meg.rosenblatt{at}mountsinai.org
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
In response to and in a joint venture with the OPMC, the New York State Society of Anesthesiologists, Inc. (NYSSA) has developed a remediation program in anesthesiology for individuals ordered into retraining by the BPMC1. The NYSSA program is coordinated though the Committee on Continuing Medical Education and Remediation (CME&R). To engage in Phase II retraining the NYSSA program requires that the candidate undergo an anesthesiology-specific evaluation (ASE) to identify areas of deficiency both to determine their suitability for retraining in anesthesiology and to facilitate the generation of an appropriate prescription for retraining.
We describe the first use of a human patient simulator, under the auspices of this NYSSA program, to aid in determining the suitability of an anesthesiologist candidate to enter Phase II of PRP.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The members of the RSC were charged with developing an ASE tool to determine the suitability of a candidate for remedial training. A 3-day, four-part evaluation protocol was designed. The ASE consisted of an interview, a written multiple-choice evaluation, oral case discussions, and simulated cases conducted in a human patient simulation laboratory. All interactive sessions were videotaped.
The interview was conducted to ascertain background information with respect to past medical education, residency training, clinical practice, practice environment, and continuing medical education. The interview also provided an opportunity for the RSC to understand the candidate’s thoughts and feelings concerning the evaluation and remediation program and to provide insight into the candidate’s acceptance of and commitment to retraining.
The written evaluation was necessary to determine the candidate’s basic anesthesiology knowledge. The Anesthesia Knowledge Test 6 (Metrics Associates, Inc., Chelmsford MA), an examination administered to anesthesiology residents at the completion of the first 6 mo of training, was used to assess basic science knowledge. It was administered only after the RSC had examined each question for relevance, importance, and applicability to anesthesia practice of the experienced practitioner.
A series of four case studies relevant to the individual’s practice was created and administered in an oral examination format. This required the candidate to develop comprehensive pre-, intra- and postoperative management plans and allowed the evaluators the opportunity to assess the candidate’s depth of anesthesia and general medical knowledge, the clinical rationale for treatment choices and responses to a wide variety of clinical situations.
The candidate was given sufficient time on the first day of evaluation to become familiar and comfortable with the METI® (Medical Education Technologies Inc., Sarasota FL) simulation technology. Four simulated cases were scripted and conducted, each lasting 30–60 min. These cases allowed the RSC to explore the candidate’s preparedness, approach, technical abilities, and judgmental responses to clinical problems and to problem solving. Medical record keeping was evaluated through the use of records from the individual’s own institution.
After the completion of these four sections of the evaluation process, the RCS reviewed their individual findings, including the videotapes, to determine the suitability of the candidate for retraining in anesthesiology, and if deemed acceptable, to develop a prescription for remediation.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The use of the human patient simulator offered an added dimension to the evaluation process—one only recently made available to the anesthesiology community. This experience enabled the RSC to observe the candidate within an operating room environment and to evaluate technical proficiency as well as knowledge. Clinical situations explored included difficult airway management, hypotension, congestive heart failure, fluid resuscitation, and complications of mechanical ventilation. It was possible to observe how the candidate responded to critical events, formulated differential diagnoses, and implemented treatment strategies.
The RSC began with the imperatives that the purpose of retraining was not to totally recreate residency training and that any prescription would have to return a candidate to a level of basic proficiency within a limited amount of time. The results of this evaluation process produced a recommendation for a 6-mo program, at an academic medical center, of clinical and academic experiences that address the specific deficiencies that had been identified. Planning for actual Phase II retraining is currently underway.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
In New York State, physicians ordered into remediation by the BPMC because their medical practices have come into question are referred to the PRP. During Phase I extensive physical, psychological, and general medical evaluations (chart review, literature reviews, OSCEs) are conducted to exclude any physical or mental impairment that might inhibit them from completing an educational program. After successful completion of Phase I, candidates are referred to their respective medical specialties for Phase II and any evaluation process that may be required to determine Phase II suitability. In New York State, retraining candidates in anesthesiology are referred to the NYSSA CME&R for evaluation. They are directed, specifically, to the RSC, whose responsibility is to determine the exact nature of the anesthesiology-related deficits and, if possible, to develop an educational prescription that addresses those issues. The RSC developed a 3-day, four-part ASE consisting of an interview, oral examination, written examination, and human patient simulation, to collect anesthesiology-specific information about the candidate and to supplement the information gathered by the initial PRP assessment.
Human patient simulation is a recreation of the anesthesiologist’s task environment, requiring actual performance of interventions, using authentic equipment (1), and adding a new dimension to performance assessment. Our candidate was given sufficient time on the first day of the evaluation to become familiar and comfortable with the technology, including the anesthesia machine, monitoring devices and anesthesia cart. Simulations were enacted with the candidate, a "surgeon," and available "assistants," who could aid the subject by providing equipment or performing tasks, but who were not permitted to offer suggestions regarding interventions. Scenarios were designed by the RSC to evaluate the deficiencies cited by the initial OPMC investigation, within the context of the candidate’s current practice, (i.e., knowledge and implementation of the ASA Difficult Airway Algorithm in a patient with asthma undergoing pelvic laparoscopy). Cases were designed to explore specific potential deficiencies identified by the candidate’s initial evaluation, not to determine overall competence. Thus, we did not feel it necessary to compare the candidate’s performance with those of "competent" physicians.
The candidate discussed his actions, diagnoses, and therapies as the case progressed. Questions, by the examiner, were inserted at appropriate times. Rather than assigning a pass/fail or numeric grade to a scenario, evaluators who were either present or reviewed the videotapes rated behaviors and interventions as excellent, acceptable, or deficient in comparison with those expected of a graduating CA-3 resident. Rating systems are imperfect, with better interrater agreement for technical rather than behavioral performances (2). Interrater reliability between observers of videotapes of simulated critical events has been found to be excellent in a study of structured scenarios acted by the investigator and rated by two physicians with longer than 5 yr active clinical practice. Interventions were scored as no response to clinical situation = 0, compensating intervention defined as physiological correction = 1, and definitive therapy = 2, with agreement on the scoring occurring for 29 of the 30 items (3). The members of the RSC achieved a consensus about the appropriateness of interventions and the level of proficiency with which the candidate functioned.
Human patient simulation not only helped to confirm gaps in general medical knowledge that had been identified in the other parts of the ASE but also revealed deficits in interpretation of data, synthesis of information, and development of treatment strategies—all that would need to be addressed during retraining. Simulation also allowed the candidate to demonstrate technical proficiency and certain aspects of applied knowledge and judgment, which were unable to be elicited by other evaluative tools. The candidate’s performance on the simulator positively contributed to the RSC’s decision to recommend the candidate for Phase II retraining. After successful achievement of the prescribed goals of retraining, the candidate’s practice would continue to be monitored for quality performance for a specified period. This is considered Phase III retraining, on uneventful completion of which he or she could return to fully independent practice.
The issue of dyscompetent physicians has not been extensively addressed in the literature, and no information specific for anesthesiologists with lapsed skills currently exists. The College of Physicians and Surgeons of Ontario sponsors the Physician Review Program (PREP). The PREP experience with five physicians who underwent a 3-yr program of individualized and group CME activities shows that severely incompetent physicians can improve with intensive remedial continuing medical education, but that older physicians were less likely to benefit from their intensive interventions (4). They found that a significant number of elderly physicians who performed poorly on tests of competency had neuropsychological impairments that would explain their suboptimal performances (5). These physicians, though, would not likely be candidates for PRP retraining.
The clinical competence of physicians has been questioned in the media, and is of concern to both the medical profession and general population. The American Board of Anesthesiologists (ABA) began its certification process in the 1930s to assure the public that the diplomats had completed an approved training program, had demonstrated an acceptable level of knowledge, and had both the experience and the skills to provide high-quality care (6). This process, which involves both a written and oral examination, has been validated by Slogoff et al. (7), who showed that it identified many of the same qualities that were used in global assessments of competence developed over an extended period of personal observation by residency directors.
Since January 1, 2000, ABA has offered "Continued Demonstration of Qualification" certificates, which are valid for 10-yr periods, after which candidates will be required to take a written examination to obtain certificate renewals (8).
Board certification attempts to meet societal goals and governmental regulations; it neither ensures clinical competence, nor is it requisite to the practice of anesthesiology.
The screening processes of local medical boards of conduct will undoubtedly, and sadly, continue to identify anesthesiologists with lapsed medical skills. Programs like PRP can provide retraining, which should allow the resolution of quality of care issues and, ideally, offer the opportunity for qualified physicians to safely reenter the workplace. We strongly believe that the ASE developed by the NYSSA CME&R positively influenced this individual’s opportunity to continue in the practice of medicine. The incorporation of an untested modality, the human patient simulator, contributed to the understanding of the candidate’s performance characteristics by reproducing anesthetic patient encounters in a controlled environment. Specifically, the use of simulation provided information about breadth of thinking during critical events, technical proficiency, and medical record keeping, which could not be elicited during the written, oral, or interview sections of the ASE.
The process of remediation of anesthesiologists in New York State is, admittedly, in its infancy, evaluating only two candidates and offering the ASE to one. Hard data are lacking, but it is our hope that reporting our experience will serve to open dialogue among others who are also wrestling with the issue of dyscompetent, yet remediable, physicians. Sharing information will minimize duplicative efforts and validate techniques, with the ultimate goal of providing safer patient care. We feel strongly that human patient simulation should be considered an integral tool in the evaluation of, and development of a remediation prescription for, dyscompetent physicians.
Appendix
.
|
.
|
|
Footnotes |
|---|
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  H. Berkenstadt, G. S. Kantor, Y. Yusim, N. Gafni, A. Perel, T. Ezri, and A. Ziv The Feasibility of Sharing Simulation-Based Evaluation Scenarios in Anesthesiology Anesth. Analg., October 1, 2005; 101(4): 1068 - 1074. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. L. Garden, B. J. Robinson, C. U. Arancibia, T. J. Carron, S. Monk, J. Vollmer, W. Heinrichs, C. Grube, B. M. Graf, and E. B. Johnson Unrecognized malfunction in computerized patient simulators Br. J. Anaesth., December 1, 2004; 93(6): 873 - 875. [Full Text] [PDF]
|
|
|
|
|
|
M. A. Seropian General Concepts in Full Scale Simulation: Getting Started Anesth. Analg., December 1, 2003; 97(6): 1695 - 1705. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. J. Lechner, M. G. van Wijk, A. J. Maas, F. R. van Dorsten, R. A. Drost, C. J. Langenberg, L. J. Teunissen, P. H. Cornelissen, and J. van Niekerk Clinical Results with the Acoustic Puncture Assist Device, a New Acoustic Device to Identify the Epidural Space Anesth. Analg., April 1, 2003; 96(4): 1183 - 1187. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
W. D. Grant Addition of Anesthesia Patient Simulator Is an Improvement to Evaluation Process Anesth. Analg., September 1, 2002; 95(3): 786 - 787. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
