Anesth Analg 2004;98:437-442
© 2004 International Anesthesia Research Society
doi: 10.1213/01.ANE.0000097173.20740.06
ECONOMICS, EDUCATION, AND HEALTH SYSTEMS RESEARCH
Resident Training Level and Quality of Anesthesia Care in a University Hospital
Karen L. Posner, PhD*,
, and
Peter R. Freund, MD*,
*Departments of Anesthesiology and
Anthropology, University of Washington; and
Anesthesia Clinical Services, University of Washington Medical Center, Seattle
Address correspondence to Karen L. Posner, PhD, Department of Anesthesiology, Box 356540, University of Washington, Seattle, WA 98195-6540. Address e-mail to posner{at}u.washington.edu
 |
Abstract
|
|---|
In this study, we analyzed the relationship between resident training and patient safety in anesthesia. A retrospective quality improvement database review was used to calculate the relative risk of any quality problem and specific types of quality problems (injury, escalation of care, or operational inefficiency) between anesthesia teams with CA1, CA2, and CA3 residents. It was expected that teams with less experienced residents (CA1) would have more frequent quality problems than teams with more experienced residents (CA2 and CA3 teams). Data showed that risk of injury did not differ between CA1, CA2, and CA3 teams. CA2 teams had higher rates of critical incidents and escalation of care than CA1 and CA3 teams and higher rates of operational inefficiency than CA3 teams. The CA2 yr is when residents move into specialty training, requiring more advanced skills and a larger knowledge base. Their higher relative risk for critical incidents, escalation of care, and operational inefficiencies may reflect lack of experience, uncertainty, and less skill mastery compared with CA3 residents. The higher inefficiency and escalation of care rates associated with CA2 teams may translate into larger costs for the institution.
IMPLICATIONS: Appropriate supervision of anesthesia residents helps to ensure patient safety. Anesthesia management problems are most common during the CA2 yr and result in higher costs for the institution.
|
Introduction
|
|---|
Anesthesia residency training in the United States combines apprenticeship and didactic learning over 3 yr. Appropriate direction of care provided by residents during this apprenticeship period is essential to patient safety. Too little direction by attending anesthesiologists may result in adverse events or outcomes resulting from lack of knowledge and experience on the part of trainees. Traditionally, anesthesia residents are given increasing independence in recognition of their increased knowledge and experience as they progress through their training program. Too much independence too soon could threaten patient safety. Few studies have provided evidence regarding these patient safety implications of residency training.
In our university hospital, the system to achieve balance between anesthesia resident training levels, level of supervision, and case complexity is not completely formalized. In assigning residents to cases each day, the first criterion is resident specialty rotation. After this is taken into consideration, remaining resident assignments for the day are based on individual educational needs, daily anesthesia service needs, and staff availability. Individual educational needs are primarily related to types of surgical procedures for which individual residents require additional experience.
In our team practice, attending anesthesiologists concurrently direct patient care provided by multiple residents or certified registered nurse anesthetists (CRNAs). Within a 1:2 staffing level (1 attending anesthesiologist concurrently directing 2 rooms), our anesthesia service attempts to balance supervision by experience of residents to optimize teaching effectiveness and ensure patient safety. For example, after the first 2 mo of the academic year, an attending anesthesiologist directing a first-year resident may be concurrently assigned a more experienced resident or CRNA rather than concurrent direction of two first-year residents. This allows for more time in the room with the less experienced resident while concurrently directing care by the more experienced resident or CRNA. This attempt to balance resident or CRNA experience is done by the clinical coordinators after generally acceptable principals in our practice without a formal case-assignment algorithm.
We have previously shown that quality of anesthesia care was not adversely affected by increasing productivity or by approaching a 1:2 concurrency level (1,2). This study explores the results of our approach to balancing training level in case assignments to avoid any decrease in quality of anesthesia care from an inexperienced resident. If case assignments are not well balanced for experience level of trainees, we might expect higher rates of continuous quality improvement (CQI) reports from teams with less experienced members compared with those with more experienced residents.
|
Methods
|
|---|
After IRB approval, 1992–2000 case data were retrospectively extracted from the departmental database of 129,930 anesthetics to test the null hypothesis that anesthesia teams with fewer years of resident experience had no difference in CQI report rates from teams with more years of experience. Team composition (CA1, CA2, and CA3) and location of anesthesia care were obtained from billing data. Only teams definitively identified as attending plus resident (n = 80,796) were included in the analysis. Teams with anesthesia fellows, CRNAs, unspecified residents, and solo attending anesthesiologist cases were excluded. Location of anesthesia care was used to differentiate operating room from obstetrical (OB) assignment.
Outcome data were abstracted from the departmental CQI database and matched to departmental case data. The CQI database consists of self-reported adverse outcomes and anesthesia management problems, as previously described (3). Adverse events and outcomes were included in the CQI database if they met the following criteria: the problem was unexpected (not an expected complication of the procedure or side effect of a drug), the problem was not managed appropriately by the anesthesia team, or there was an injury or increased level of care for diagnosis or treatment of the problem. Adverse outcomes were classified into three categories: patient injuries, escalation of care, and operational inefficiencies resulting from anesthesia management problems. Patient injuries are physical injuries resulting from adverse events. Examples include death, brain damage, nerve injury, stroke, myocardial infarction, airway injury, dental damage, soft tissue injury, or any other nontransient physical insult to the patient. Escalation of care includes any significant unplanned increase in the level of care provided to the patient. It includes such outcomes as unplanned intensive care unit admission, unplanned admission of an outpatient, prolonged recovery room stay, delay in extubation, reintubation, administration of extra drugs or tests to diagnose or treat an anesthesia complication, consultation by the medicine or cardiology service, or change from regional to general anesthesia. Operational inefficiencies are outcomes that represent increased costs (in dollars or resources), such as delays or cancellations of surgery. Anesthesia management problems (adverse events) that did not result in patient injury, escalation of care, or operational inefficiency were classified as critical incidents. Resident experience level is not a factor in determining if an adverse event is a reportable anesthesia management problem. All problems should be reported, even if they result from provider inexperience. Only critical incidents and adverse outcomes determined by CQI peer review to be related to anesthesia management were included in the analysis. Surgical problems and problems stemming from patient underlying condition were excluded.
For each type of anesthesia team (CA1, CA2, and CA3), CQI report rates were calculated by dividing the number of anesthesia-related CQI cases by the total number of cases conducted. Rates of specific outcomes (injury, escalation of care, operational inefficiency, and critical incident) were calculated for each type of anesthesia team by dividing the number of cases with each specific outcome by the total number of cases conducted. CQI report rates were analyzed by team composition across all years in the study period (January 1, 1992–December 31, 2000). Statistical analysis of CQI report rates between pairs of teams with differing composition (CA1 versus CA2 teams, CA2 versus CA3 teams, and CA1 versus CA3 teams) was conducted using the relative-risk (RR) ratio, with the less experienced team considered the "exposure" group and CQI reports considered "ill" to test the hypothesis that teams with less experienced residents would have higher CQI report rates than teams with more experienced residents. With Bonferroni correction of P 
0.05 for three tests (CA1 versus CA2, CA2 versus CA3, and CA1 versus CA3), P 0.017 was required for statistical significance. Calculations of risk ratios, P values (with Yates correction for continuity), and power were made using the Center for Disease Control and Prevention’s EpiInfo 2002 (Atlanta, GA), which uses the formulas of Fleiss (4). With the available sample size within each resident training group (CA1, CA2, and CA3) and a mean CQI report rate of 0.047 (4.7% of anesthetics had a CQI report), our study had 
80% power to detect a RR of 1.25 for each comparison in this cross-sectional study design.
To assess the possible confounding of patient acuity with balance of supervision, the analysis was repeated within subsets of patients with identical ASA physical status. RR ratios were calculated within the subset of ASA 1, ASA 2, ASA 3, and ASA 4–5 patients, with the less experienced team considered the "exposure" group and CQI reports considered "ill," as described above. ASA 4 and ASA 5 patients were combined in this subset analysis because of the small numbers of patients in these groups for CA1 and CA2 teams.
|
Results
|
|---|
The overall CQI report rates for teams with CA1 members were lower than rates for CA2 teams and the same as rates for CA3 teams (Table 1). CQI report rates for CA2 teams were higher than rates for CA3 teams (RR ratio, 1.60; P 0.001; Table 1).
When data were analyzed by specific type of CQI report (patient injury, escalation of care, operational inefficiency, or critical incident), similar trends were noted. Teams with CA1 residents had lower rates of escalation of care than CA2 teams (RR, 0.51; P 0.01), and teams with CA2 residents had higher rates of escalation of care than CA3 teams (RR, 1.76; P 0.001). A similar trend was seen in critical incident rates, with CA1 teams having lower rates than CA2 teams, and CA2 teams having higher rates than CA3 teams (Fig. 1). Operational inefficiency rates were higher among CA2 teams than CA3 teams (RR, 1.29; P 0.012). There were no statistically significant differences in patient injury rates between teams with less experienced residents and teams with more experienced residents (Fig. 1).
View larger version (21K):
[in this window]
[in a new window]
|
Figure 1. Rates of adverse outcomes and critical incidents per 1000 anesthetics by teams with different levels of resident training. Each continuous quality improvement (CQI) report was classified as either a critical incident or an adverse outcome. Multiple adverse outcomes (injury, escalation of care, or operational inefficiencies) could be recorded for an individual case. n = total number of anesthetics administered by that type of team during the study period; **indicates significantly higher relative-risk (RR) ratio than CA1 and CA3 teams; *indicates significantly higher RR ratio than CA3 teams; P < 0.017 is required for statistically significant differences in RR ratio with Bonferroni adjustment for multiple comparisons. There was no difference in RR for injury for teams with less experienced residents compared with teams with more experienced residents.
|
|
Analysis within ASA physical status subsets yielded similar results to the overall analysis. Within the subsets of cases with ASA 1, ASA 2, and ASA 3 patients, overall CQI report rates for teams with CA1 members were lower than rates for CA2 teams and did not differ from CA3 teams (Table 2). CQI report rates for CA2 teams were higher than rates for CA3 teams (Table 2). There were no statistically significant differences in CQI report rates between different teams for ASA 4–5 patients.
View this table:
[in this window]
[in a new window]
|
Table 2. Continuous Quality Improvement (CQI) Report Rates Within ASA Physical Status Groups by Resident Training Level
|
|
|
Discussion
|
|---|
Nine years of quality improvement program data suggest that our hospital has achieved a fairly successful balance between resident direction and independence. Patient injuries related to anesthesia management problems were relatively rare and did not differ among teams during the nine-year study period. Post hoc power analysis indicated that our sample size was sufficient to detect a RR ratio of 1.25 with 70% power between teams at the injury rate observed in the study. In other words, our study had sufficient statistical power to detect a 25% difference in injury rates between CA1 and CA2 teams. We did detect a numerically higher injury rate for CA3 compared with CA1 teams, a result opposite the study hypothesis. Had we not statistically adjusted for multiple comparisons, this higher injury rate for CA3 compared with CA1 teams would have been statistically significant (P 0.043), but just barely so. Appropriate statistical testing with adjustment indicates that this numerically higher injury rate is not statistically significant and may simply reflect chance variation.
Other differences in CQI report rates between teams with differing levels of resident experience were not in the predicted direction. If case assignments were perfectly balanced, we expected to find no difference in CQI report rates between CA1, CA2, and CA3 teams. This was the result for the most critical measure of patient safety, injuries related to anesthesia management. Results that were not predicted were the more frequent noninjury adverse outcomes and critical incidents among teams with CA2 residents compared with CA1 and CA3 teams (Fig. 1).
Why do teams with CA2 members have higher CQI report rates than other teams? Clearly our system of balancing case assignment and direction is not perfect. However, there may be factors in the educational and CQI programs that influence the report rates. Our residents receive most of their specialty training through subspecialty service rotations during the CA2 year. These rotations include OB, ambulatory anesthesia, cardiac and transplantation, regional anesthesia, and neuroanesthesia. During the cardiac and OB rotations in particular, residents are closely supervised. The implications for the self-report CQI program are that anesthesia management problems may be more likely to be reported because of the increased presence of 1:1 faculty:resident coverage. The continuous presence of multiple providers makes it more probable that one or both will remember to report any problems to the CQI program. In contrast, CA2 residents have more independence than CA1 residents during their other rotations that year. An appropriate balance of independence versus hands-on case direction during the CA2 year is a challenge to any teaching program. The increased independence during the CA2 year may be leading to more anesthesia management problems in our practice than teams with closely supervised CA1 residents or more experienced CA3 residents. Our CQI program is not sufficiently sensitive to distinguish between these possible factors.
Another factor that may influence the more frequent CQI reports by CA2 teams is the normal acquisition of knowledge and skills as residents advance through the training program. The CA2 year challenges residents to use a higher knowledge and skill base during specialty rotations than the CA1 year. Even under direct hands-on supervision by an attending anesthesiologist, anesthesia management difficulties in applying newly acquired knowledge and skills may occur. Whereas the data suggest that such problems do not result in patient injury, their resolution would be reported to the CQI program if escalation of care or operational inefficiencies developed. The CA3 year may benefit from the experience and practice using these skills and knowledge during the CA2 year. The lower CQI report rates in the CA3 year may reflect the higher level of skill mastery among residents approaching the completion of their training. Skill acquisition over years of training and practice has been demonstrated in radiology (5–7) and internal medicine (8).
Another factor that could influence study results is the self-report basis of our CQI program. We know that all problems are not reported to the program. We do not know if different types of CQI problems have different under-report rates. If different types of anesthesia management problems are differentially reported to the CQI program, and if these problems are differentially experienced in different specialty rotations, this could affect observed differences in the study results. A current audit of the CQI program is attempting to establish and quantify these factors.
ASA physical status is not known when case assignments are made, so it is not incorporated into the mechanism of balancing resident experience with case assignment. However, ASA physical status might be inferred from other factors such as age, surgical procedure, or emergent versus nonemergent surgery, factors which are usually correlated with patient acuity. ASA physical status would only be known when patients were evaluated and would reasonably be expected to affect the attending anesthesiologist’s approach to balancing hands-on case direction. Complex anesthesia management problems stemming from sick patients or complex procedures should not be reflected in our outcome measures (anesthesia-related CQI reports) if managed appropriately by the anesthesia team. The analysis within subsets of patients with similar ASA physical status (Table 2) suggests that our results are not confounded by patient acuity.
Despite limitations, we believe that these data provide an accurate representation of the relationship between resident experience and quality of anesthesia care. Our risk management experience confirms our CQI data, with no exposure of unreported injuries by risk management. The implications of these data for patient safety are encouraging. The implications for the economics of residency training are not unexpected. Other specialties (9–11) have shown that care by trainees is associated with increased patient tests and procedure times and associated costs. Our data show that anesthesia CA2 resident care is correlated with higher rates of operational inefficiencies and escalation of patient care (such as extra drugs and tests and prolonged recovery room stays). These data translate directly into costs, most of which must be absorbed by the institution. These data may also translate into inconvenience and reduced patient satisfaction. Whereas some of these increased costs might be prevented by closer case direction, one-on-one teaching with hands-on supervision throughout medical residency is not an accepted educational practice and would represent greatly increased costs (in both dollars and personnel) to anesthesia departments. Other approaches to reduction of adverse events and outcomes in the CA2 year are required. Changes in the CA2 lectures and seminars or the addition of subspecialty simulations to the CA2 curriculum might target specific problem areas. Practice guidelines and protocols in subspecialty areas might reduce variation and improve quality of care. Attention to preanesthesia planning discussions between CA2 residents and attending anesthesiologists might better prepare CA2 residents to anticipate and manage problems that they otherwise might not anticipate. We have recently added schedule codes to tag complex cases to draw the anesthesia team’s attention to a need for special planning or preparation. These are just a few simple recommendations that might reduce adverse events and outcomes and their associated costs without compromising the required educational experience of increasing independence in clinical practice. We conclude that appropriate supervision of anesthesia residents helps ensure patient safety. Anesthesia management problems are most common during the CA2 year and result in higher costs for the institution.
|
Acknowledgments
|
|---|
The authors wish to thank John Campos for his excellent assistance in data management for this project.
|
Footnotes
|
|---|
Presented at the Annual Meeting of the American Society of Anesthesiologist, October 17, 2000, San Francisco, CA.
|
References
|
|---|
- Posner KL, Freund PR. Trends in quality of anesthesia care associated with changing staffing patterns, productivity, and concurrency of case supervision in a teaching hospital. Anesthesiology 1999; 91: 839–47.[ISI][Medline]
- Freund PR, Posner KL. Sustained increases in productivity with maintenance of quality in an academic anesthesia practice. Anesth Analg 2003; 96: 1104–8.[Abstract/Free Full Text]
- Posner KL, Kendall Gallagher D, et al. Linking process and outcome of care in a continuous quality improvement program for anesthesia services. Am J Med Qual 1994; 9: 129–37.
- Fleiss J. Statistical methods for rates and proportions. 2nd ed. New York: Wiley, 1981: 38–45.
- Velmahos GC, Fili C, Vassiliu P, et al. Around-the-clock attending radiology coverage is essential to avoid mistakes in the care of trauma patients. Am Surg 2001; 67: 1175–7.[ISI][Medline]
- Rosen MP, Sands DZ, Morris J, et al. Does a physician’s ability to accurately assess the likelihood of pulmonary embolism increase with training? Acad Med 2000; 75: 1199–205.[ISI][Medline]
- Engl J, Mysko WK, Weller GE, et al. Interpretation of emergency department radiographs: a comparison of emergency medicine physicians with radiologists, residents with faculty, and film with digital display. AJR Am J Roentgenol 2000; 175: 1233–8.[Abstract/Free Full Text]
- Hicks CM, Gonzalez R, Morton MT, et al. Procedural experience and comfort level in internal medicine trainees. J Gen Intern Med 2000; 15: 716–22.[ISI][Medline]
- Simmer TL, Nerenz DR, Rutt WM, et al. A randomized, controlled trial of an attending staff service in general internal medicine. Med Care 1991; 29: JS31–40.[ISI][Medline]
- Edelman A, Nichols MD, Jensen J. Comparison of pain and time of procedures with two first-trimester abortion techniques performed by residents and faculty. Am J Obstet Gynecol 2001; 184: 1564–7.[ISI][Medline]
- Salazar A, Corbella X, Onaga H, et al. Impact of a resident strike on emergency department quality indicators at an urban teaching hospital. Acad Emerg Med 2001; 8: 804–8.[ISI][Medline]
Accepted for publication September 9, 2003.
This article has been cited by other articles:
|
|
|
|
 
M. Schuster, T. Kotjan, M. Fiege, and A. E. Goetz
Influence of resident training on anaesthesia induction times
Br. J. Anaesth.,
November 1, 2008;
101(5):
640 - 647.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
C. K. Haan, S. Milford-Beland, S. O'Brien, D. Mark, M. Dullum, T. B. Ferguson, and E. D. Peterson
Impact of Residency Status on Perfusion Times and Outcomes for Coronary Artery Bypass Graft Surgery
Ann. Thorac. Surg.,
June 1, 2007;
83(6):
2103 - 2110.
[Abstract]
[Full Text]
[PDF]
|
|
|
Top
Abstract
Introduction
