JOURNAL HOME CME HOME THIS MONTH PAST ISSUES ETOC COLLECTIONS AUTHORS REVIEWERS EDITORIAL BOARD FEEDBACK RSS HELP
		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (12) Citing Articles via Google Scholar Google Scholar Articles by Rosenblatt, M. A. Articles by Abrams, K. J. Search for Related Content PubMed PubMed Citation Articles by Rosenblatt, M. A. Articles by Abrams, K. J. Related Collections Economics and Health Care Research

---

ECONOMICS AND HEALTH SYSTEMS RESEARCH

The Use of a Human Patient Simulator in the Evaluation of and Development of a Remedial Prescription for an Anesthesiologist with Lapsed Medical Skills

Department of Anesthesiology, The Mount Sinai School of Medicine, New York, New York

Address correspondence and reprint requests to Meg A. Rosenblatt, MD, Clinical Associate Professor of Anesthesiology, The Mount Sinai School of Medicine, Department of Anesthesiology, Box 1010, One Gustave L. Levy Place, New York, NY 10029-6574. Address e-mail to meg.rosenblatt{at}mountsinai.org

	Abstract

Top Abstract Introduction Methods Results Discussion References

 
The New York State Society of Anesthesiologists’ Committee on Continuing Medical Education and Remediation has been charged by the Office of Professional Medical Conduct of the New York State Department of Health to develop a remediation program for individuals ordered into retraining. We describe the development of an anesthesiology-specific evaluation to identify areas of deficiency to both determine a candidate’s suitability, as well as to facilitate the creation of an appropriate prescription for retraining. A human patient simulator was used to aid in the gathering of information during the evaluation process. Specifically, the use of simulation allowed the exploration of a candidate’s preparation, approach to clinical situations, technical abilities, response to clinical problems, ability to problem solve, and accuracy of medical record keeping. Human patient simulation should be considered a valuable tool in the process of evaluating physicians with lapsed medical skills.

	Introduction

Top Abstract Introduction Methods Results Discussion References

 
The New York State Department of Health, in conjunction with the Office of Professional Medical Conduct’s Physician Monitoring Program, is committed to offering retraining programs in numerous medical specialties. The Physician Retraining Program (PRP) of New York State is a three-phase program that began in 1992. It identifies and evaluates physicians with lapsed medical skills who may be candidates for specialty-specific individualized remediation programs that might allow their return to practice. Entrance into the PRP is by an order from the Board for Professional Medical Conduct (BPMC) after careful collection of evidence, investigation, expert reviews, and individual hearings coordinated by the Office of Professional Medical Conduct (OPMC) of the New York State Department of Health. Once referred for evaluation, a candidate undergoes Phase I, which consists of an extensive assessment aimed at identifying any physical or mental impairment and a comprehensive assessment of general medical knowledge that might preclude him or her from successfully completing a retraining program. If the candidate is deemed suitable for remediation, a goal-directed, structured, individualized educational program, or prescription, is formulated for retraining (Phase II).

In response to and in a joint venture with the OPMC, the New York State Society of Anesthesiologists, Inc. (NYSSA) has developed a remediation program in anesthesiology for individuals ordered into retraining by the BPMC¹. The NYSSA program is coordinated though the Committee on Continuing Medical Education and Remediation (CME&R). To engage in Phase II retraining the NYSSA program requires that the candidate undergo an anesthesiology-specific evaluation (ASE) to identify areas of deficiency both to determine their suitability for retraining in anesthesiology and to facilitate the generation of an appropriate prescription for retraining.

We describe the first use of a human patient simulator, under the auspices of this NYSSA program, to aid in determining the suitability of an anesthesiologist candidate to enter Phase II of PRP.

	Methods

Top Abstract Introduction Methods Results Discussion References

 
A candidate for ASE was directed to the members of the Remediation Sub-Committee (RSC) of the NYSSA CME&R. This six-member group of senior anesthesiologists who practice in academic medical centers reviewed the results of the Phase I evaluation. This included a personal history and an extensive psychological assessment with cognitive evaluation and personality inventories and an assessment of general medical knowledge, all of which had been conducted at the Upstate Clinical Competency Center at Albany Medical College. The general medical evaluation included videotapes of the observed structured clinical evaluations (OSCEs), with trained personnel as simulated patients. The medical records of each specific incident in which the candidate’s delivery of medical care was considered to be substandard were reviewed. Additionally, the RSC members reviewed all investigative reports of the OPMC and BPMC, as well as all of the expert reports solicited by the OPMC, as part of their initial investigation.

The members of the RSC were charged with developing an ASE tool to determine the suitability of a candidate for remedial training. A 3-day, four-part evaluation protocol was designed. The ASE consisted of an interview, a written multiple-choice evaluation, oral case discussions, and simulated cases conducted in a human patient simulation laboratory. All interactive sessions were videotaped.

The interview was conducted to ascertain background information with respect to past medical education, residency training, clinical practice, practice environment, and continuing medical education. The interview also provided an opportunity for the RSC to understand the candidate’s thoughts and feelings concerning the evaluation and remediation program and to provide insight into the candidate’s acceptance of and commitment to retraining.

The written evaluation was necessary to determine the candidate’s basic anesthesiology knowledge. The Anesthesia Knowledge Test 6 (Metrics Associates, Inc., Chelmsford MA), an examination administered to anesthesiology residents at the completion of the first 6 mo of training, was used to assess basic science knowledge. It was administered only after the RSC had examined each question for relevance, importance, and applicability to anesthesia practice of the experienced practitioner.

A series of four case studies relevant to the individual’s practice was created and administered in an oral examination format. This required the candidate to develop comprehensive pre-, intra- and postoperative management plans and allowed the evaluators the opportunity to assess the candidate’s depth of anesthesia and general medical knowledge, the clinical rationale for treatment choices and responses to a wide variety of clinical situations.

The candidate was given sufficient time on the first day of evaluation to become familiar and comfortable with the METI^® (Medical Education Technologies Inc., Sarasota FL) simulation technology. Four simulated cases were scripted and conducted, each lasting 30–60 min. These cases allowed the RSC to explore the candidate’s preparedness, approach, technical abilities, and judgmental responses to clinical problems and to problem solving. Medical record keeping was evaluated through the use of records from the individual’s own institution.

After the completion of these four sections of the evaluation process, the RCS reviewed their individual findings, including the videotapes, to determine the suitability of the candidate for retraining in anesthesiology, and if deemed acceptable, to develop a prescription for remediation.

	Results

Top Abstract Introduction Methods Results Discussion References

 
The schedule for the ASE appears in Appendix A. The ASE afforded the members of the RSC the opportunity to create an in-depth evaluation of the candidate’s ability to perform as an anesthesiologist. The interview provided insight into the education activities, professional plans, and motivation of the candidate, whereas the written examination elucidated significant and specific deficits in the candidate’s fund of basic anesthesiology knowledge. The oral cases were limited to ones that would be encountered in the candidate’s present institution. They allowed an assessment of current practice patterns and sought evidence of the understanding of practice guidelines, techniques, and pharmacologic drugs that had been introduced since the candidate’s initial residency training.

The use of the human patient simulator offered an added dimension to the evaluation process—one only recently made available to the anesthesiology community. This experience enabled the RSC to observe the candidate within an operating room environment and to evaluate technical proficiency as well as knowledge. Clinical situations explored included difficult airway management, hypotension, congestive heart failure, fluid resuscitation, and complications of mechanical ventilation. It was possible to observe how the candidate responded to critical events, formulated differential diagnoses, and implemented treatment strategies.

The RSC began with the imperatives that the purpose of retraining was not to totally recreate residency training and that any prescription would have to return a candidate to a level of basic proficiency within a limited amount of time. The results of this evaluation process produced a recommendation for a 6-mo program, at an academic medical center, of clinical and academic experiences that address the specific deficiencies that had been identified. Planning for actual Phase II retraining is currently underway.

	Discussion

Top Abstract Introduction Methods Results Discussion References

 
Dyscompetent physicians, unfortunately, exist in all medical subspecialties. Once identified, it is necessary to be fair but cautiously conservative in the evaluation and determination of whether physicians with lapsed medical skills are suitable for remediation. Because these evaluations serve a critical public health service, we have a fiduciary responsibility to ensure that candidates returning to practice after remedial training do not pose a risk to patients. It is incumbent on the evaluation team to use a comprehensive approach to aid in the identification and extent of a candidate’s deficits. To this end, we have found the human patient simulator to be a valuable tool.

In New York State, physicians ordered into remediation by the BPMC because their medical practices have come into question are referred to the PRP. During Phase I extensive physical, psychological, and general medical evaluations (chart review, literature reviews, OSCEs) are conducted to exclude any physical or mental impairment that might inhibit them from completing an educational program. After successful completion of Phase I, candidates are referred to their respective medical specialties for Phase II and any evaluation process that may be required to determine Phase II suitability. In New York State, retraining candidates in anesthesiology are referred to the NYSSA CME&R for evaluation. They are directed, specifically, to the RSC, whose responsibility is to determine the exact nature of the anesthesiology-related deficits and, if possible, to develop an educational prescription that addresses those issues. The RSC developed a 3-day, four-part ASE consisting of an interview, oral examination, written examination, and human patient simulation, to collect anesthesiology-specific information about the candidate and to supplement the information gathered by the initial PRP assessment.

Human patient simulation is a recreation of the anesthesiologist’s task environment, requiring actual performance of interventions, using authentic equipment (1), and adding a new dimension to performance assessment. Our candidate was given sufficient time on the first day of the evaluation to become familiar and comfortable with the technology, including the anesthesia machine, monitoring devices and anesthesia cart. Simulations were enacted with the candidate, a "surgeon," and available "assistants," who could aid the subject by providing equipment or performing tasks, but who were not permitted to offer suggestions regarding interventions. Scenarios were designed by the RSC to evaluate the deficiencies cited by the initial OPMC investigation, within the context of the candidate’s current practice, (i.e., knowledge and implementation of the ASA Difficult Airway Algorithm in a patient with asthma undergoing pelvic laparoscopy). Cases were designed to explore specific potential deficiencies identified by the candidate’s initial evaluation, not to determine overall competence. Thus, we did not feel it necessary to compare the candidate’s performance with those of "competent" physicians.

The candidate discussed his actions, diagnoses, and therapies as the case progressed. Questions, by the examiner, were inserted at appropriate times. Rather than assigning a pass/fail or numeric grade to a scenario, evaluators who were either present or reviewed the videotapes rated behaviors and interventions as excellent, acceptable, or deficient in comparison with those expected of a graduating CA-3 resident. Rating systems are imperfect, with better interrater agreement for technical rather than behavioral performances (2). Interrater reliability between observers of videotapes of simulated critical events has been found to be excellent in a study of structured scenarios acted by the investigator and rated by two physicians with longer than 5 yr active clinical practice. Interventions were scored as no response to clinical situation = 0, compensating intervention defined as physiological correction = 1, and definitive therapy = 2, with agreement on the scoring occurring for 29 of the 30 items (3). The members of the RSC achieved a consensus about the appropriateness of interventions and the level of proficiency with which the candidate functioned.

Human patient simulation not only helped to confirm gaps in general medical knowledge that had been identified in the other parts of the ASE but also revealed deficits in interpretation of data, synthesis of information, and development of treatment strategies—all that would need to be addressed during retraining. Simulation also allowed the candidate to demonstrate technical proficiency and certain aspects of applied knowledge and judgment, which were unable to be elicited by other evaluative tools. The candidate’s performance on the simulator positively contributed to the RSC’s decision to recommend the candidate for Phase II retraining. After successful achievement of the prescribed goals of retraining, the candidate’s practice would continue to be monitored for quality performance for a specified period. This is considered Phase III retraining, on uneventful completion of which he or she could return to fully independent practice.

The issue of dyscompetent physicians has not been extensively addressed in the literature, and no information specific for anesthesiologists with lapsed skills currently exists. The College of Physicians and Surgeons of Ontario sponsors the Physician Review Program (PREP). The PREP experience with five physicians who underwent a 3-yr program of individualized and group CME activities shows that severely incompetent physicians can improve with intensive remedial continuing medical education, but that older physicians were less likely to benefit from their intensive interventions (4). They found that a significant number of elderly physicians who performed poorly on tests of competency had neuropsychological impairments that would explain their suboptimal performances (5). These physicians, though, would not likely be candidates for PRP retraining.

The clinical competence of physicians has been questioned in the media, and is of concern to both the medical profession and general population. The American Board of Anesthesiologists (ABA) began its certification process in the 1930s to assure the public that the diplomats had completed an approved training program, had demonstrated an acceptable level of knowledge, and had both the experience and the skills to provide high-quality care (6). This process, which involves both a written and oral examination, has been validated by Slogoff et al. (7), who showed that it identified many of the same qualities that were used in global assessments of competence developed over an extended period of personal observation by residency directors.

Since January 1, 2000, ABA has offered "Continued Demonstration of Qualification" certificates, which are valid for 10-yr periods, after which candidates will be required to take a written examination to obtain certificate renewals (8).

Board certification attempts to meet societal goals and governmental regulations; it neither ensures clinical competence, nor is it requisite to the practice of anesthesiology.

The screening processes of local medical boards of conduct will undoubtedly, and sadly, continue to identify anesthesiologists with lapsed medical skills. Programs like PRP can provide retraining, which should allow the resolution of quality of care issues and, ideally, offer the opportunity for qualified physicians to safely reenter the workplace. We strongly believe that the ASE developed by the NYSSA CME&R positively influenced this individual’s opportunity to continue in the practice of medicine. The incorporation of an untested modality, the human patient simulator, contributed to the understanding of the candidate’s performance characteristics by reproducing anesthetic patient encounters in a controlled environment. Specifically, the use of simulation provided information about breadth of thinking during critical events, technical proficiency, and medical record keeping, which could not be elicited during the written, oral, or interview sections of the ASE.

The process of remediation of anesthesiologists in New York State is, admittedly, in its infancy, evaluating only two candidates and offering the ASE to one. Hard data are lacking, but it is our hope that reporting our experience will serve to open dialogue among others who are also wrestling with the issue of dyscompetent, yet remediable, physicians. Sharing information will minimize duplicative efforts and validate techniques, with the ultimate goal of providing safer patient care. We feel strongly that human patient simulation should be considered an integral tool in the evaluation of, and development of a remediation prescription for, dyscompetent physicians.

Appendix
.

	Footnotes

 
¹ Abrams KJ. The New York State Society of Anesthesiologists, Inc. remediation program: a model for retraining [abstract]. Anesthesiology 1995;83:A1042.

	References

Top Abstract Introduction Methods Results Discussion References

 

1. Gaba DM, DeAnda A. A comprehensive anesthesia simulation environment: re-creating the operating room for research and training. Anesthesiology 1988:387–94.
2. Gaba DM, Howard SK, Flanagan B, et al. Assessment of clinical performance during simulated crises using both technical and behavioral ratings. Anesthesiology 1998; 89: 8–18.[ISI][Medline]
3. Devutt HH, Kurrek MM, Cohen MM, et al. Testing the raters: inter-rater reliability of standardized anaesthesia simulator performance. Can J Anaesth 1997; 44: 924–8.[Abstract/Free Full Text]
4. Hanna E, Premi J, Turnbull J. Results of remedial continuing medical education in dyscompetent physicians. Acad Med 2000; 75: 174–76.[ISI][Medline]
5. Turnbull J, Carbotte R, Hanna E, et al. Cognitive difficulty in physicians. Acad Med 2000; 75: 177–81.[ISI][Medline]
6. American Board of Anesthesiology. Continued demonstration of qualifications for board-certified anesthesiologists. Anesthesiology 1990; 73: 770–1.[ISI][Medline]
7. Slogoff S, Hughes FP, Hug CC, et al. A demonstration of validity for certification by the American Board of Anesthesiology. Acad Med 1994; 69: 740–6.[ISI][Medline]
8. Glass DD. Time-limited certification. Anesthesiology 1995; 83: 413–5.[ISI][Medline]

This article has been cited by other articles:

				H. Berkenstadt, G. S. Kantor, Y. Yusim, N. Gafni, A. Perel, T. Ezri, and A. Ziv The Feasibility of Sharing Simulation-Based Evaluation Scenarios in Anesthesiology Anesth. Analg., October 1, 2005; 101(4): 1068 - 1074. [Abstract] [Full Text] [PDF]

				A. L. Garden, B. J. Robinson, C. U. Arancibia, T. J. Carron, S. Monk, J. Vollmer, W. Heinrichs, C. Grube, B. M. Graf, and E. B. Johnson Unrecognized malfunction in computerized patient simulators Br. J. Anaesth., December 1, 2004; 93(6): 873 - 875. [Full Text] [PDF]

				M. A. Seropian General Concepts in Full Scale Simulation: Getting Started Anesth. Analg., December 1, 2003; 97(6): 1695 - 1705. [Abstract] [Full Text] [PDF]

				T. J. Lechner, M. G. van Wijk, A. J. Maas, F. R. van Dorsten, R. A. Drost, C. J. Langenberg, L. J. Teunissen, P. H. Cornelissen, and J. van Niekerk Clinical Results with the Acoustic Puncture Assist Device, a New Acoustic Device to Identify the Epidural Space Anesth. Analg., April 1, 2003; 96(4): 1183 - 1187. [Abstract] [Full Text] [PDF]

				W. D. Grant Addition of Anesthesia Patient Simulator Is an Improvement to Evaluation Process Anesth. Analg., September 1, 2002; 95(3): 786 - 787. [Full Text] [PDF]

				M. Girard and P. Drolet Anesthesiology simulators: networking is the key/Les simulateurs d'anesthesiologie : le travail en reseau est la cle Can J Anesth, August 1, 2002; 49(7): 647 - 649. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (12) Citing Articles via Google Scholar Google Scholar Articles by Rosenblatt, M. A. Articles by Abrams, K. J. Search for Related Content PubMed PubMed Citation Articles by Rosenblatt, M. A. Articles by Abrams, K. J. Related Collections Economics and Health Care Research

Anesthesia & Analgesia® is published for the International Anesthesia Research Society® by Lippincott Williams & Wilkins with the assistance of Stanford University Libraries' HighWire Press®. Copyright 2006 by the International Anesthesia Research Society. Online ISSN: 1526-7598   Print ISSN: 0003-2999