This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Sandberg, W. S. Articles by Walsh, J. L. Search for Related Content PubMed PubMed Citation Articles by Sandberg, W. S. Articles by Walsh, J. L. Related Collections Economics and Health Care Research Technology

---

ECONOMICS, EDUCATION, AND POLICY

Real-Time Checking of Electronic Anesthesia Records for Documentation Errors and Automatically Text Messaging Clinicians Improves Quality of Documentation

Warren S. Sandberg, MD, PhD^*, Elisabeth H. Sandberg, PhD, Andreas R. Seim, MS, Shaji Anupama, BTech^*, Jesse M. Ehrenfeld, MD^*, Stephen F. Spring, BA^*, and John L. Walsh, MD^*

From the *Department of Anesthesia and Critical Care, Massachusetts General Hospital; Department of Psychology, Suffolk University, Boston, Massachusetts; Department of Production and Quality Engineering, Norwegian University of Science and Technology, Trondheim, Norway; and Harvard Medical School, Boston, Massachusetts.

Address correspondence and reprint requests to Warren S. Sandberg, MD, PhD, Department of Anesthesia and Critical Care, Massachusetts General Hospital, 55 Fruit St., Jackson 4, Boston, MA 02114. Address e-mail to wsandberg{at}partners.org.

Abstract

INTRODUCTION: The quality of electronic anesthesia documentation is important for downstream communication and to demonstrate appropriate diligence to care. Documentation quality will also impact the success of reimbursement contracts that require timely and complete documentation of specific interventions. We implemented a system to improve completeness of clinical documentation and evaluated the results over time.

METHODS: We used custom software to continuously scan for missing clinical documentation during anesthesia. We used patient allergies as a test case, taking advantage of a unique requirement in our system that allergies be manually entered into the electronic record. If no allergy information was entered within 15 min of the "start of anesthesia care" event, a one-time prompt was sent via pager to the person performing the anesthetic. We tabulated the daily fraction of cases missing allergy data for the 6 mo before activating the alert system. We then obtained the same data for the subsequent 9 mo. We tested for systematic performance changes using statistical process control methodologies.

RESULTS: Before initiating the alert system, the fraction of charts without an allergy comment was slightly more than 30%. This decreased to about 8% after initiating the alerts, and was significantly different from baseline within 5 days. Improvement lasted for the duration of the trial. Paging was suspended on nights, weekends, and holidays, yet weekend documentation performance also improved, indicating that weekday reminders had far-reaching effects.

DISCUSSION: Electronic anesthesia documentation performance can be rapidly managed and improved by using an automatic process monitoring and alerting system.

Anesthesia information management systems (AIMS) record data into a database during anesthesia. These databases can be searched synchronously with the creation of the anesthesia record for "errors" that are defined by the AIMS users. This allows the generation of timely alerts to clinicians about such errors so that their impact may be reduced and the quality of documentation can be improved.

High quality documentation is important to facilitate communication with clinicians in units that are downstream in the medical process. Complete and timely documentation in the operating room (OR) is also an important mechanism by which providers can demonstrate appropriate intraoperative diligence to care. An AIMS can assist with quality of documentation, but the AIMS can also record inadvertent lapses in documentation that may be costly, due to either lost revenue or medicolegal liability.1–3

We recently demonstrated that custom software operating independently of the AIMS, but using the accumulating AIMS data tables, can identify missing data and logical inconsistencies in documentation that would prevent billing if left uncorrected.3 In the same report, we demonstrated that by connecting this record-checking system to an automated alert function via the hospital paging system, the time to correct documentation errors and the residual number of errors that were never corrected could both be reduced by an order of magnitude.

Using the paging system offers a potential advantage in that users need not be in the vicinity of the AIMS workstation to receive alerts. Hence, a medically directing clinician may receive alerts about his/her case although in another OR. On the other hand, our approach to improving documentation by using pager alerts must be applied with care so as not to be a nuisance to clinicians but rather provide timely reminders in their daily work.

In 2006, our institution began participating in "pay-for-performance" contracts wherein reimbursement would eventually be tied to achieving set performance goals on clinical metrics. Performance on potential pay-for-performance metrics, such as temperature management, perioperative antibiotic administration, perioperative oxygen therapy, and deep vein thrombus prophylaxis, may be gauged by reference to clinical documentation. Hence, successful participation depends in part on documentation that is complete and timely. Recognizing this, and anticipating the pay-for-performance contracts, we set out to adapt our AIMS-based error alerting system to encompass clinical documentation. This gave us an opportunity to address questions that could not be answered by our previous work.

The first question we sought to address in this report was whether a one-time alert is sufficient to change documentation performance. The system in our previous work relied in part on repeated reminders that continued, in some instances, for days after the case had ended, to achieve the results previously reported. However, clinical documentation must be timely (i.e., it should be completed before any possible transfer of care) to be most useful to the next clinician on the care path. Therefore, it is important to test whether clinical documentation performance can be improved with reminders that are sent only near the time when the documentation must be completed to be most useful.

A second question we sought to address was: "How quickly does performance on documentation tasks change once the reminder system is activated?" Our previous work included a period of limited implementation and refinement of the reminder system, so we were unable to determine how quickly the system worked to change documentation performance.

The final question that we were able to address in this new work was: "How does the alerting system act to improve performance?" We wanted to determine whether the alerting system served merely as a reminder tool, or whether recipients of the reminders were actually learning from the system, an important distinction that we were unable to detect in our previous work. The pairing of behaviors with particular consequences affects the likelihood of subsequent repetition of those behaviors.4,5 The presentation or removal of desirable or aversive stimuli differentially affect behavior. In our study, the alert page can be regarded as an aversive stimulus, which suggests that the system should cause learning to occur.

Persistent compliance with completion of clinical documentation, even during times when no alerts are administered, would be evidence of a mode of learning known as active avoidance learning.5 In active avoidance learning, a cue, referred to as the "preaversive stimulus",6 always precedes the aversive stimulus. The preaversive stimulus soon elicits a response that allows the aversive stimulus to be avoided. Clinicians might learn that they must engage in a particular response (spontaneously completing the documentation task) to avoid an aversive stimulus (an alert page). The preaversive stimulus, cueing clinicians about the impending aversive stimulus, would be the initial sequence of keystrokes to initiate the electronic recording process in the AIMS. Once acquired, avoidance responses are extremely robust.7 Even when an aversive stimulus is suspended, behavior continues in the manner previously required to avoid the stimulus.5,7

To answer the questions outlined above, we developed a test case for monitoring and alerting about clinical documentation during anesthesia. Specifically, we tested whether our documentation checking system, coupled with a one-time pager alert, could reduce the incidence of a clinical documentation error that was present in many of our records; namely, the absence of any comment about whether the patient has medication allergies.8 We used patient allergies as a test case, taking advantage of a unique requirement in our AIMS implementation that allergy data be manually entered into the electronic record¹. In contrast to our previous work, we specifically sought to determine whether any improvement in documentation could be achieved with a single alert during the case, without reminders.

METHODS

We developed software independent of the AIMS to monitor the accuracy and completeness of documentation. The software (Anesthesia Billing Alert System [ABAS]) was originally designed to scan the anesthesia record for documentation errors that would prevent billing.3 We modified the software to allow scanning for all types of documentation errors (as defined by the user) and renamed it as the Anesthesia Documentation Alert System (ADAS). Scanning occurs synchronously with the creation of the anesthesia record during the case. When errors are detected, the ADAS automatically generates alphanumeric text pages through the hospital paging system.

The ABAS, the predecessor to the ADAS, was designed to find and alert clinicians about documentation errors that would prevent billing. The ABAS3 had been fully implemented for one full year before the experiments described in this current article. Thus, clinicians were accustomed to receiving pager alerts about documentation errors as part of their routine practice.

The ADAS was designed to apply logical rules pertaining to completeness of clinical documentation; for the purpose of this pilot, documentation about patient allergies. In our AIMS workflow, the start of anesthesia care event is one of a series of events entered when a new case is opened and data recording is begun. This takes place after the anesthesiologist’s initial encounter with the patient. Allergy documentation was considered complete if any comment, including "None," "NKA," "NKDA" and any other alphanumeric character string was placed in the allergy field. If no allergy information was entered within 15 min of the start of anesthesia care event, a one-time prompt was sent via pager to the clinician signed into the AIMS as the person performing the case. In approximately 80% of cases, this would be a resident, with the remaining 20% split between attending physicians and nurse anesthetists.

We chose the 15-min interval reasoning that a clinician should complete documentation of preanesthetic events including allergy comments before induction, and that induction would infrequently occur within 15 min of the start of anesthesia care. We also reasoned: 1) that documentation need not be completed to safely anesthetize a patient because the anesthesiologist would surely be familiar with the allergies, thus no "hard stop" was needed; 2) that this documentation should be complete before any logical opportunity for a transfer of care (i.e., "handoff") could occur; and 3) that handoffs would be unlikely between the start of anesthesia care and the induction of anesthesia. Thus, the 15-min interval was chosen to ensure that the reminder for this particular clinical data element was close to the start of anesthesia care, before induction and in time to get the data into the record before any handoffs.

Pages were only sent on weekdays between 7 am and 6 pm; alerts were withheld on weekends and holidays. This is because the alert system was also in use for managing documentation pertaining to billing (via the ABAS). Because the ABAS was primarily an administrative application, we had chosen to send pages only during weekday work hours as a courtesy to the staff, and we did not change this for the clinical alerts in this pilot experiment. Thus, we knew from the outset that a fraction of the records missing allergy data would not generate pages, and that the system’s effectiveness would be decreased by this choice.

To measure the effectiveness of a one-time alert, we searched the AIMS database (with approval of the Massachusetts General Hospital IRB, Boston, MA) to determine the number of cases that were missing allergy data during the 6-mo period before activation of the pager alerting system. We then obtained the same data for 9 mo subsequent to the initiation of the page alert function. We were interested in the number of cases performed on any given day that lacked an allergy comment, relative to the total number of cases performed that day and documented with the AIMS. The average number of cases performed per day varies widely by the day of the week at our institution. Thus, we calculated the error rate for each day using the total number of cases documented with the AIMS for that given day as the denominator. This manipulation converted the "number of errors for each day" data to a relative error rate for each day, which is a continuous variable.

We used statistical process control (SPC) methodology (X-bar charts, a common form of charts for variables) to test for systematic performance changes. The general approach to SPC and the creation of X-bar charts can be found in textbooks as well as in our prior work and the work of others.9–14

SPC is frequently used to monitor performance over time. In our analysis, we plotted the fraction of charts lacking allergy comments as a function of calendar date. SPC methods were developed for use with sampling techniques, where only a fraction of the work output (in this case, anesthesia records) is tested. However, we have a complete data set, with results for all records, and SPC can be readily applied to complete data sets. In either scenario, the data must be grouped sequentially into samples of a reasonable size to perform the analysis. The appropriate sample size is found empirically, chosen to balance between the required sensitivity and acceptable noise in the data. For the following analyses, we created uniform samples of five sequential days.

To systematize the visual identification of systematic performance changes in SPC charts, formal rules have been developed. We used the Western Electric Rules for analyzing the SPC charts (Table 1), seeking to separate distinct performance changes from random variation. The Western Electric rules are a series of tests that can be visually applied to a SPC chart without requirements for calculations or performing statistical tests. However, the rules are based on sound statistical reasoning. Thus, they provide a known probability (typically P < 0.005) that processes are experiencing systematic performance changes rather than random variation.9,15

All data were extracted in mid-August of 2006. Data were tabulated in spreadsheets and SPC analyses were performed using standard statistical software.

RESULTS

Our main results are illustrated in Figure 1. This figure tracks documentation performance for weekday cases as a function of calendar date showing performance during a baseline period before activating the alerts, during the provision of alerts, and after the alerts had ended. During the baseline period, between 25% and 35% of all cases had no comments about patient allergies. The average error rate (i.e., the fraction of charts without an allergy comment in each 5-day sample) was slightly more than 30%, indicated by the solid centerline. The upper and lower control limits, which are three standard deviations above and below the centerline, are indicated for the control period by long dashed lines.

View larger version (8K): [in this window] [in a new window]   Figure 1. Fraction of records without allergy documentation, as a function of calendar date for data excluding weekends and holidays. Each point is the average fraction of charts lacking any allergy data for successive 5-week day periods. Baseline data before activating the pager alert system are shown on the left of the figure. The black line and diamonds show the final error rate, i.e., the fraction of charts lacking allergy data at the time of data extraction. The gray line and diamonds during the period when pager alerts were provided shows the initial error rate, i.e., the fraction of cases lacking allergy data 15 min after the "start of anesthesia care," before a reminder specific to the case would be sent. The centerline and control limits are described in the text.

On November 14, 2005, we began sending the one-time alert pages reminding clinicians whose charts lacked allergy documentation to complete this data entry. The first sample of days to contain data from the paging period is below the three standard deviation control limit for the baseline period (activating Western Electric Rule 1, Table 1), indicating that a significant improvement in performance for entering allergy data had occurred within days of activating the paging alerts.

Within 10 days of activating the pager alert system, only 8.2% of charts lacked a comment in the allergy data field. The new three standard deviation control limits for the postpaging intervention period ranged from 11.7% to 4.7%. This is a nearly fourfold reduction of the documentation error rate, achieved within days of the intervention.

We continued sending the alert pages until May 1, 2006, after which they were discontinued so that we could install a new AIMS from a different vendor. Performance was stable at the new, more acceptable level until late June of 2006. Then allergy documentation performance declined from the best result achieved, with multiple 5-day performance samples having error rates above the three standard deviation upper control limit. However, performance remained better than the baseline measured before the trial.

Figure 1 shows the global performance for allergy comment documentation using all weekday data. During weekdays, there were an average of 121 ± 14.5 (mean ± sd) cases per day monitored by the ADAS. An average of 12.4 ± 4.8 pages per day was sent to remind clinicians to complete allergy documentation, resulting in an average of 6.0 ± 3.8 corrections per day. By these criteria, the paging alert does not seem to be all that effective. However, in addition to reminding people during the case to correct records missing allergy data, the system seems to have taught people to add allergy data before the "error condition" would be detected, thus avoiding a page. This can be seen in Figure 1.

In Figure 1, the gray line with small diamonds shows the initial error rate, i.e., the fraction of cases lacking allergy data 15 min after the start of anesthesia care, before a reminder specific to the case would be sent. The error rate decreased from 31% before activating the pages to an initial error rate of 13% during the period when alerts were being sent. Thus, the need for reminders decreased concomitant with the advent of the pager alerts.

The data in Figure 1 include weekday evenings, when no page alerts were sent. Approximately 40% of all weekday cases that lacked any comment in the allergy field had occurred between 6 pm and 7 am, when the paging function was turned off. Restated in affirmative terms, after omitting the cases that did not generate alerts, the automatic process monitoring/paging alert system facilitated a 95% success rate at completing clinical documentation through a combination of reductions in the initial error rate and single reminders sent on an as-needed basis during clinical care.

Figure 2 shows allergy comment errors for weekends and holidays only. Again, the data are grouped as samples of five sequential weekend days and holidays. Baseline data before activating the pager alert system are shown on the left of the figure. The average error rate (i.e., the fraction of charts without an allergy comment in each 5-day sample) was approximately 27%, indicated by the solid centerline. The upper and lower control limits are indicated for the control period by long dashed lines. Again, paging began on November 14, 2005, and a new average performance level for allergy documentation was established. The new centerline (average fraction of charts lacking allergy data) stabilized at 12%. The new upper and lower three standard deviation control limits are at 25% and 0%, respectively. Western Electric rules 1, 2, and 3 (Table 1) were all triggered on January 14, 2006 (i.e., four samples of five sequential weekend and holiday days after beginning the alerts).

View larger version (7K): [in this window] [in a new window]   Figure 2. Fraction of records without allergy documentation, as a function of calendar date, for weekends and holidays. Each point is the average fraction of charts lacking any allergy data for successive 5-day blocks that included only weekends and holidays. Western Electric rules 1, 2, and 3 (Table 1) were all triggered on 1/14/06.

Although there is more variation in the weekend/ holiday data set, and fewer points, the performance for completing allergy comments mirrors that of the weekday data set. This is interesting, because alerts were suspended as a courtesy to staff on nights, weekends, and holidays, so clinicians’ documentation performance was better even though they received no reminders.

Most of the attending anesthesiologists, certified registered nurse anesthetists, and anesthesia residents working in the ORs during the period when alerts were provided received alert pages about allergy documentation. We performed a provider-level analysis of the number of clinicians who performed cases in the OR during the period when alerts were provided, the number of clinicians who received pages notifying them of absent allergy documentation, and the number who subsequently entered allergy data into their charts. These results are summarized in Table 2, and are subdivided by provider type, i.e., certified registered nurse anesthetist, resident, or attending-as-sole-provider signed into the AIMS in the role of "anesthetist." Nurse anesthetists corrected a significantly larger fraction of their records than did the anesthesia residents. Anesthesia residents, in turn, corrected more records than attending physicians who received pages when acting as sole providers.

DISCUSSION

Our results indicate that one-time messaging of clinicians about particular documentation errors detected during anesthesia improves performance of this documentation task. Improvement occurred within days of beginning the intervention and persisted for almost 2 mo after suspension of the alert system. Our system was deployed into 50 OR suites without fanfare and without specific education provided to clinicians. Instead, we simply prompted the anesthesiologist to complete documentation by sending a text page alert using the hospital paging system. Thus, our system is highly effective with minimal training requirements in our environment. Clinicians were accustomed to receiving alert pages from the AIMS as part of our previous work,3 so it is unlikely that our observation is the result of a Hawthorne effect. Examination of Table 2 indicates that most anesthesia providers responded to the alerts by completing the missing documentation, but that the completion rate differed among groups. Attending anesthesiologists responded least frequently to the alert pages, and thus would be logical candidates for introductory education about new alert systems.

How do our results relate to other recent reports of process improvements involving reminders for anesthesia clinicians? Table 3 presents the results of reports in which reminders to clinicians were implemented as part of schemes to change patterns of behavior in the perioperative period. The first five reports (including the present report) provided automatically generated reminders either during or immediately after the case, thus allowing remediation of the targeted error almost as soon as it occurred. All five used either a pop-up window in the AIMS application, or alphanumeric pages to send the alerts, allowing the reminder to be "pushed" to the clinician, rather than relying on them to seek out the information (as would be the case for e-mail reports). The remaining studies provided reminders delayed by at least overnight from the date of service. The table also summarizes what repeat reminders (if any) were reported, and the consequences (if any) of continued deficiency. Finally, the time course to achieve the desired change in performance, and the maximum effect observed are tabulated. With the exception of our results reported here, the more immediate feedback mechanisms were not necessarily associated with rapid onset or completeness of achieving the desired performance change when compared with later reminders. Indeed, our current result is unique, in that a significant performance change could be demonstrated within days of intervention, and peak effect was achieved in about 2 wk. Also, none of the reminder systems were completely effective (or nearly so) unless they included either an overt financial penalty or multiple reminders along with in-person follow-up from an authority figure. Thus, it is conceivable that one-time alerts provided in near real time may not be suitable for situations in which absolute compliance is required.

A potential limitation of our results is that the use of custom software could impede wider implementation, limiting the general impact of the error detection/ alerting system. However, our software was written using customary approaches in a standard programming language, and is meant to be independent of any one vendor’s product. Hence, our system is built to work with server-based data tables rather than as a part of the proprietary AIMS software. Because the system is independent of the AIMS, it can, in principle, be used with any vendor’s system. This philosophy also drove the decision to send alerts via short messaging devices such as the hospital paging system (a system that can be extended to mobile telephones capable of short messaging), rather than implementing alerts through the AIMS itself.

In theory, our data could over-estimate the observed performance improvement. We extracted all data in mid-August of 2006, and clinicians could potentially enter allergy data long after the fact, up until the time of data extraction. However, each time an error was detected, only one alert was sent to the individual clinician whose chart lacked allergy documentation, 15 min after the start of anesthesia care event. Our experience with the ABAS applied to documentation errors that prevent billing indicates that errors are rarely corrected without the reinforcement of repeated alert messages (S. Spring, unpublished observations). Additionally, there is no incentive for clinicians to complete allergy data fields after the fact. Accordingly, it is unlikely that clinicians entered any allergy comments after the immediate time of receiving the page alert, and we believe that our results capture the impact of a one-time alert to correct lapses in clinical documentation.

One could argue that it would be much simpler to make completion of the allergy data field mandatory by preventing further data entry until it was entered. However, we judged this approach to be too restrictive, as the beginning of the case is a busy period, and mandatory documentation tasks might distract anesthesiologists from patient care or inhibit the smooth start of automatic data gathering. Instead, we used the allergy documentation example as a test case to assess whether providing a one-time alert via a pathway distinct from the AIMS could be useful for improving documentation performance. Clinicians receive the alert in an appropriate time frame (i.e., after the start of anesthesia care but before most opportunities for a transfer of care), but the function of the AIMS is not blocked by an untimely demand for mandatory data completion.

A recent study of intraoperative interference with the function of the OR team caused by distractions and interruptions may clarify the relative intrusions caused by pager alerts versus pop-up windows or forced-completion fields emanating from the AIMS.22 Sources of interference were rated on a nine-point ordinal scale. Relevant to the current discussion, the severity of interference ranged from a potential distraction at one end of the scale (graded as 1); to a single team member being momentarily distracted (graded as 4); to a single team member changing tasks to attend to the distraction (graded as 6); to interruption of the flow of the operation (graded as 9) at the most severe extreme.22 Pagers were frequent potential distractions, but the severity of the distraction they created was low, tending not to momentarily distract even a single team member. Applying this scale to modes of documentation alerts in the OR, pages sent to anesthesia personnel and pop-up windows appearing on their AIMS workstations would both seem to be momentary distractions for a single team member, whereas a forced-completion field from the AIMS would require the anesthesia clinician to change tasks to be able to use the AIMS again.

Our proof-of-concept experiment demonstrates that it is possible to monitor and improve performance on clinical tasks such as documentation more quickly than the days-to-weeks time scale provided by a data collection, report generation and feedback system. Instead, we have demonstrated that documentation performance can be managed and improved during the case itself. Furthermore, the alerts do not need to be supplied continually. These observations may have important implications for the execution of interventions that are thought to be most beneficial when applied consistently and in a timely fashion (for which documentation may serve as a reminder). Our results may also point the way to important financial advantages for anesthesia practices that have invested in an AIMS, as it could enhance successful participation in pay-for-performance reimbursement contracts.

Footnotes

¹This is because we had not implemented any of the preoperative modules in the AIMS, and allergy data are not uniformly available in an electronic form that could be imported. However, we wish to emphasize that we used allergy documentation performance as a test of whether the documentation monitoring/alerting system could be applied to clinical documentation as a general notion. Thus, we invite the reader to focus on the general application, rather than the institution-specific problem with documentation that we used as a test case.

²If this is the case, then intermittent application of the paging alerts may prove empirically to be sufficient to maintain performance. In such a scheme, one would monitor performance prospectively, and only provide alerts when performance declined to unacceptable levels. Once performance had improved the alerts could be discontinued, to be provided again when performance declined again.

Accepted for publication August 27, 2007.

This project was sponsored by a Center for Integration of Medicine and Innovative Technology (CIMIT, Cambridge, MA) Career Development Award to W.S. Support for A.S. was provided by The Research Council of Norway’s Program for Research for Innovation and Renewal in the Public Sector (FIFOS, Oslo, Norway), St Olavs Hospital: The Operating Room of the Future (Trondheim, Norway), and Norwegian University of Science and Technology: Strategic Area Medical Technology (Trondheim, Norway). Additional departmental support was provided by the Massachusetts General Hospital Department of Anesthesia and Critical Care (Boston, MA).

REFERENCES

1. Vigoda MM, Lubarsky DA. Failure to recognize loss of incoming data in an anesthesia record-keeping system may have increased medical liability. Anesth Analg 2006;102:1798–802[Abstract/Free Full Text]
2. Vigoda MM, Lubarsky DA. The medicolegal importance of enhancing timeliness of documentation when using an anesthesia information system and the response to automated feedback in an academic practice. Anesth Analg 2006;103:131–6[Abstract/Free Full Text]
3. Spring SF, Sandberg WS, Anupama S, Walsh JL, Driscoll WD, Raines DE. Automated documentation error detection and notification improves anesthesia billing performance. Anesthesiology 2007;106:157–63[Web of Science][Medline]
4. Skinner BF. The behavior of organisms: an experimental analysis. Englewood cliffs, NJ: Prentice Hall, 1938
5. Higgins ST, Morris EK. Generality of free-operant avoidance conditioning to human behavior. Psychol Bull 1984;96:247–72[Web of Science][Medline]
6. Mowrer OH. Preparatory set (expectancy): a determinant in motivation and learning. Psychol Rev 1938;45:62–91[Web of Science]
7. Solomon RL, Wynne LC. Traumatic avoidance learning: acquisition in normal dogs. Psychol Monogr 1954;67
8. Driscoll WD, Columbia MA, Peterfreund RA. An observational study of anesthesia record completeness using an anesthesia information management system. Anesth Analg 2007;104:1454–61[Abstract/Free Full Text]
9. Montgomery DC. Introduction to statistical quality control. 5th ed. New York: Wiley, 2005:147–264
10. Seim AR, Andersen B, Sandberg WS. Statistical process control as a tool for monitoring non-operative time. Anesthesiology 2006;105:370–80[Web of Science][Medline]
11. Benneyan JC, Lloyd RC, Plsek PE. Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003;12:458–64[Abstract/Free Full Text]
12. Fasting S, Gisvold SE. Statistical process control methods allow the analysis and improvement of anesthesia care. Can J Anaesth 2003;50:767–74[Web of Science][Medline]
13. Lagasse RS, Steinberg ES, Katz RI, Saubermann AJ. Defining quality of perioperative care by statistical process control of adverse outcomes. Anesthesiology 1995;82:1181–8[Web of Science][Medline]
14. Parker BM, Henderson JM, Vitagliano S, Nair BG, Petre J, Maurer WG, Roizen MF, Weber M, DeWitt L, Beedlow J, Fahey B, Calvert A, Ribar K, Gordon S. Six sigma methodology can be used to improve adherence for antibiotic prophylaxis in patients undergoing noncardiac surgery. Anesth Analg 2007;104:140–6[Abstract/Free Full Text]
15. AT&T. Statistical quality control handbook. Indianapolis: Western Electric Corporation, 1956
16. St Jacques P, Sanders N, Patel N, Talbot TR, Deshpande JK, Higgins M. Improving timely surgical antibiotic prophylaxis redosing administration using computerized record prompts. Surg Infect (Larchmt) 2005;6:215–21[Medline]
17. Kheterpal S, Gupta R, Blum JM, Tremper KK, O’Reilly M, Kazanjian PE. Electronic reminders improve procedure documentation compliance and professional fee reimbursement. Anesth Analg 2007;104:592–7[Abstract/Free Full Text]
18. Wax DB, Beilin Y, Levin M, Chadha N, Krol M, Reich DL. The effect of an interactive visual reminder in an anesthesia information management system on timeliness of prophylactic antibiotic administration. Anesth Analg 2007;104:1462–6[Abstract/Free Full Text]
19. O’Reilly M, Talsma A, VanRiper S, Kheterpal S, Burney R. An anesthesia information system designed to provide physician-specific feedback improves timely administration of prophylactic antibiotics. Anesth Analg 2006;103:908–12[Abstract/Free Full Text]
20. Lubarsky DA, Sanderson IC, Gilbert WC, King KP, Ginsberg B, Dear GL, Coleman RL, Pafford TD, Reves JG. Using an anesthesia information management system as a cost containment tool. Description and validation. Anesthesiology 1997;86:1161–9[Web of Science][Medline]
21. Vigoda MM, Gencorelli F, Lubarsky DA. Changing medical group behaviors: increasing the rate of documentation of quality assurance events using an anesthesia information system. Anesth Analg 2006;103:390–5[Abstract/Free Full Text]
22. Healey AN, Sevdalis N, Vincent CA. Measuring intra-operative interference from distraction and interruption observed in the operating theatre. Ergonomics 2006;49:589–604[Medline]

This article has been cited by other articles:

				F. Dexter, R. H. Epstein, J. D. Lee, and J. Ledolter Automatic Updating of Times Remaining in Surgical Cases Using Bayesian Analysis of Historical Case Duration Data and "Instant Messaging" Updates from Anesthesia Providers Anesth. Analg., March 1, 2009; 108(3): 929 - 940. [Abstract] [Full Text] [PDF]

				R. H. Epstein, F. Dexter, J. M. Ehrenfeld, and W. S. Sandberg Implications of Event Entry Latency on Anesthesia Information Management Decision Support Systems Anesth. Analg., March 1, 2009; 108(3): 941 - 947. [Abstract] [Full Text] [PDF]

				W. S. Sandberg Anesthesia Information Management Systems: Almost There Anesth. Analg., October 1, 2008; 107(4): 1100 - 1102. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Sandberg, W. S. Articles by Walsh, J. L. Search for Related Content PubMed PubMed Citation Articles by Sandberg, W. S. Articles by Walsh, J. L. Related Collections Economics and Health Care Research Technology