| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
BACKGROUND: We tested melodic auditory alarms recommended in the IEC 60601-1-8 standard for medical electrical equipment for ease of learning and discrimination, and for effectiveness during a timeshared task. METHODS: Twenty-two critical care nurses learned the IEC 60601-1-8 melodic alarms over two training sessions more than a week apart, with or without mnemonics suggested in the standard. Subsequently, the nurses identified alarms arriving at quasi random intervals while performing a timeshared arithmetic task. RESULTS: Only one nurse (4.5%) identified the alarms with 100% accuracy after two training sessions. Mnemonics did not improve overall alarm identification accuracy (mnemonic = 56%, nonmnemonic = 55%) but led to a narrower range of confusions between alarms. Nurses responded faster (P < 0.0001) and more accurately (P = 0.032) to medium priority than high priority alarms, despite rating high priority alarms as sounding more urgent (P < 0.0001). Nurses with at least 1 yr of formal musical training identified the alarms much more accurately (musical training = 73%, no musical training = 38%, P < 0.0001), perceived a greater distinction between high and medium priority alarms (P = 0.002), and found identifying the alarms easier overall (P = 0.023). During the timeshared task, nurses responses were slower (P = 0.002) and became less accurate (P = 0.02). CONCLUSIONS: The slow rate of learning and persistent confusions suggest that the IEC 60601-1-8 melodic alarms should be redesigned before they are adopted for clinical practice.
The design and use of auditory alarms in medical electrical equipment has recently received attention from many organizations. The Joint Commission on Accreditation of Healthcare Organizations included alarm safety on a list of national patient safety goals in 2004 after a Sentinel Event report motivated by concerns with auditory alarms.1 The American Society of Anesthesiologists2 and American Association of Nurse Anesthetists3 have adopted the Anesthesia Patient Safety Foundations 2004 alarm summit recommendations that the variable-pitch pulse oximeter tone and capnography auditory alarm should always be on and audible.4,5 In 2005, an international standard, IEC 60601-1-86 also appeared that guides the design of visual and auditory alarm systems in medical electrical equipment and that will appear as a collateral standard in many countries. IEC 60601-1-8 reflects the input of many anesthesiologists. It includes many good recommendations for alarm design and is far more detailed than its predecessor, ISO 9703-2.7 Included in IEC 60601-1-8, however, is a well-intentioned recommendation for a set of melodic alarms, motivated by a controversial earlier design8, that distinguishes the physical or physiological system each alarm represents9,10 rather than having one alarm for all systems (Table 1). A mnemonic rationale, which explains the mapping of the melody, accompanies each alarm (e.g., "drops of an infusion falling and splashing back up" for the up-and-down infusion alarm melody). Elsewhere, lyrics have been proposed to aid learning (e.g., "IN-FU-SION A-LARM").9
The melodies in IEC 60601-1-8 were designed under the challenging constraint of conforming to fixed rhythmic patterns already established for medium and high priority alarms in ISO 9703-27 (IEC 60601-1-8, p. 87). Unfortunately, there was no formal test of the effectiveness of the melodic alarms with representative users before they were included in IEC 60601-1-8. Manufacturer experience at implementing the alarms is mixed. Independent research suggests that peoples ability to identify and distinguish the melodic alarms is low, with persistent confusions between similar-sounding alarms.11–13 There are further concerns which are rooted in auditory design theory.14,15 Each empirical study has shortcomings in either training, target population, or the range of alarms tested. Williams and Beatty12 trained 21 nonclinicians to identify the melodic alarms using the mnemonic lyrics supplied by Block et al.9 but did not include the IEC 60601-1-8 mnemonic notes giving the rationale for the mapping. After two sessions of practice, alarm identification accuracy was only between 10% and 61% and participants often confused alarms. Sanderson et al.11 tested 33 nonclinicians who learned either with or without the mnemonic lyrics and rationale, and found no difference between conditions. Participants often confused perceptually similar alarms. Finally, Lacherez et al.13 examined 14 critical care nurses ability to learn the high priority alarms unaided by mnemonic lyrics or rationale, as might occur in busy clinical contexts. Despite nurses domain expertise, alarm identification was still poor and confusions between alarms were common. The mnemonic lyrics and rationale should improve learning when participants have clinical monitoring experience. To determine this, first we tested whether critical care nurses can learn the IEC 60601-1-8 melodic alarms to 100% accuracy when supported by mnemonic lyrics and notes. Second, we hypothesized that learning will be better with mnemonics than without. Third, we tested whether learning is different for high priority versus medium priority alarms. Fourth, we hypothesized that alarm identification will be worse when participants are distracted, particularly when the alarms have been learned without mnemonics.
Participants The study was approved by Human Research Ethics Committees of The University of Queensland and of the participating hospitals. Participation was voluntary and participants provided written informed consent. They received AUD$40 and a small gift for their participation. The participants were 22 registered critical care nurses between 21 and 58 yr-of-age (average 35) working in major hospitals in Brisbane. They were allocated randomly to learning condition. Six were male and 16 female with between 1 and 36 yr of nursing experience (average 13 yr). Participants were considered musically trained if they had at least 1 yr of formal training on a musical instrument. No participants had previous experience with the IEC 60601-1-8 melodic alarm sounds. A power analysis based on previous data16 indicated that with 22 participants a 25% improvement in performance with mnemonics would be detected with statistical power of 0.71 and a 25% improvement in performance with musical training would be detected with statistical power of more than 0.9.
Apparatus and Stimuli
The total duration of medium priority alarms was 1044 ms. Pulse rise times, duration, and spacing were twice as long as for high priority alarms and consistent with ranges specified in the standard. The lowest alarm note was middle C (C4: 278.4 Hz) and the highest was an octave higher (C5: 556.8 Hz) (Table 1). The alarms were created in Csound and processed on a Pentium 4 1.9 GHz Acer Laptop with integrated soundcard. They were presented via AKG K 240DF Studio Monitor earphones with WAV on maximum and main volume control at level three from the lowest. The experimenter checked that the sound level was clear and comfortable for the participant. The alarm labels and mnemonics were displayed on a 17 in. flat screen display at 1280 x 1024 screen resolution by a Java program, which ran the experiment and recorded participants responses.
Procedure
Day 1 After familiarization, participants learned the alarms in several learn-test cycles. The alarms were presented as 16 large labeled buttons on a computer screen. In each learning phase, participants could listen to each alarm as often as they needed and in any order until they were ready for the test phase. In the Mn condition, when each alarm button was pressed, the alarms mnemonics (both the lyric and the rationale) appeared at the bottom of the screen. In each test phase, participants clicked a button to hear an unidentified alarm and then identified the alarm by clicking one of 16 possible alarm buttons. After all alarms were tested, the screen displayed the actual alarms played, in the order in which they had been presented, along with the participants response to the alarm and whether the response was correct. Participants then returned to the learning phase. The learn-test cycles continued until either the participant reached the learning criterion of two consecutive sets of perfect test scores or 45 min had elapsed.
Day 2 In Part 2 of Day 2, participants performed learn-test cycles, which continued until the participants achieved two consecutive perfect scores, 45 min had elapsed, or the learn-test cycle had been completed eight times. In Part 3 of Day 2 participants identified the alarms while performing a timeshared task. Participants saw a simple equation such as "4 – 2 = 12" and responded with the "P" key if the equation were true and the "Q" if false. The keys and their mappings were clearly indicated on the screen. At quasi-random moments, between 21 and 55 s apart, a melodic alarm would sound. The participant clicked the button on the screen representing the alarm they heard. The alarms were presented in random order with each alarm occurring once in each of three trials. At the end of Day 1, after Part 2 of Day 2 (Relearning), and after Part 3 of Day 2 (Transfer), participants answered questionnaires that asked them to rate on 7-point scales the ease of learning the alarms, the ease of associating alarm meanings with their sounds, and the relative urgency of the medium and high priority alarms.
Statistical Analysis
Figure 1 shows average accuracy and latency under Mn and NMn learning conditions for the first two test trials on Day 1 (termed "Day 1 start" below), the last two test trials on Day 1 ("Day 1 end"), the Day 2 LTM test ("Day 2 LTM"), the last two trials in Part 2 of Day 2 ("Day 2 end"), and for Part 3 of Day 2 during the timeshared task ("Transfer").
Response Accuracy During Learning There was no significant effect of learning condition on accuracy (Mn = 56%, NMn = 55%, P = 0.88) but participants with musical training identified alarms far more accurately than those without (musical = 73%, non musical = 38%, P < 0.0001). Accuracy changed over test phases (Day 1 start = 42%, Day 1 end = 59%, Day 2 LTM = 52%, Day 2 end = 68%, P < 0.0001) (Table 3). Improvements in accuracy from start to end of Day 1, from LTM test to Day 2 end and from Day 1 start to Day 2 end were significant at P < 0.05. Musical participants gained accuracy over phases faster than the nonmusical participants, leading to an interaction of musical training with phase (P = 0.008). Participants identified medium priority alarms more accurately than high priority alarms (medium = 58%, high = 53%, P = 0.032).
In Figure 2, participants accuracy at identifying each of the eight alarms is shown for the Mn and NMn learning conditions and for high and medium priority alarms for all test trials in Part 2 of Day 2. Large-font percentages adjacent to alarm names indicate the average identification accuracy for each alarm. Smaller-font percentages on arrows indicate the percentage of participants who misidentified the sound at the origin of the arrow as the alarm at the end on more than 25% of their test trials ("persistent confusions"). There were further persistent confusions unique to a single participant (not shown on figures). Although there was no difference in overall accuracy between the Mn and NMn conditions, Mn participants showed a relatively narrow range of persistent confusions (high = 17, medium = 20) whereas NMn participants showed a broader range of persistent confusions (high = 31, medium = 29). The interaction of learning condition with the presence or absence of persistent confusions absorbed the log-linear model with a significant partial association (
Response Latency During Learning
Performance of Timeshared Task
Questionnaires Perceived ease of associating alarms with their meanings was moderately low to moderate and did not change significantly during relearning (Day 1 = 3.4, Day 2 Relearning = 4.0, Transfer = 3.3, P = 0.07) but participants with musical training rated it easier to make the associations than those without musical training (musical = 4.2, nonmusical = 3.0, P = 0.009). Participants rated the high priority alarms as sounding significantly more urgent than the medium priority alarms (high = 5.1, medium = 3.3, P < 0.0001). There were no main effects of phase, learning condition, or musical training. However, musically trained participants differentiated the high and medium alarm more markedly than did participants without musical training leading to an interaction (for musical, high = 5.6 and medium = 3.1; for nonmusical, high = 4.5 and medium = 3.4; P = 0.002).
This is the first study to evaluate the effects of mnemonics and timeshared tasks on clinicians performance in identifying the IEC 60601-1-8 melodic alarms. First, after two learning sessions, only 1 of the 22 nurses could identify the alarms with 100% accuracy in two successive tests. The average percentage correct on Day 2 was only 66% (including the general alarm). Second, there was no significant effect of mnemonics on speed and accuracy of identifying the alarms or on retention. This is surprising because the mnemonics were intended to help clinicians remember the mappings of melodies to systems. Memory research indicates, however, that mnemonics are most useful when people generate their own associations15 whereas IEC 60601-1-8 suggests a specific set of mnemonics. Third, performance was always faster and more accurate for the medium priority alarms than for the high priority alarms, even though the high priority alarms sounded more urgent to participants. It would be better for clinicians to respond faster and more accurately to the high priority alarms, which may indicate more urgent situations, than to the low priority alarms. Fourth, the timeshared task slowed alarm identification from 5.2 s to 6.1 s and led to a small but significant decrease in accuracy from 66% to 62%, indicating that identification was resource-demanding. With the timeshared task, performance worsened more markedly for the more difficult high priority alarms and the mnemonics did not preserve participants ability to identify the alarms. This study reinforces the conclusions of previous investigations11–13 into the IEC 60601-1-8 melodic alarms. The mapping of melodies to alarms through clinical associations may be no more effective than if melodies had been assigned at random to the alarms. Acoustically differentiated alarms may be confusing in clinical practice if misidentified alarms cause inappropriate initial visual checks. Confusion between sounds occurs if sounds have too many common characteristics.17 The IEC 60601-1-8 melodic alarms share a consistent rhythmic structure that was established in the previous ISO 9703-2 standard and retained for IEC 60601-1-8. Moreover, the notes are all between C4 and C5 in the key of C major and it is implied, even if not stated, that they should share the same timbre, making them harder to distinguish. The IEC 60601-1-8 melodic alarms share many similarities with earcons, which are short musical motifs used to represent members of a set of objects or events such as types of software applications in a computer operating system.18,19 Earcons that have similar rhythm, timbre, and key are difficult to identify when heard singly and are exceptionally difficult to discriminate when overlapping.11,13,19 Given the above, it is no surprise that musically trained participants identify the alarms much more accurately than participants without musical training. However, a successful design should be one in which all participants do well, with far less (and possibly no) effect of musical training, as demonstrated by Brewster et al.18 There are several limitations to our study. First, the melodic alarms were tested for auditory discriminability only in the absence of redundant visual alarms. Second, the alarms were not tested under clinical conditions in the context of an episode of care. Third, the standard does not require all alarms to be differentiated, so the result might apply to extreme conditions only. Some confusions between alarms were so persistent for some participants, however, that they might have been experienced even if the set of melodic alarms had been smaller. In summary, although the IEC 60601-1-8 recommendation for melodic alarms is well-intentioned, there is cause for concern that the ability of clinicians to learn the recommended alarms and to discriminate between them may not be adequate.11–13,17,18 Overall, the design of the IEC 60601-1-8 melodic alarms appears to have been over-constrained. Interdisciplinary research teams should explore simple design changes to the melodic alarms such as modifications in rhythm, timbre, and pitch, and should test the results formally. Indeed, any design changes to existing medical alarms should meet the conditions proposed by the standard. In the meantime, medical electrical equipment manufacturers and the anesthesia and critical care communities should be aware of potential problems with the recommended IEC 60601-1-8 melodic alarms before introducing them into clinical settings. An important lesson learned is the need to test design innovations formally with representative users before proposing such designs in equipment standards.
The authors thank Philippe Lacherez and Phil Cole for constructing the alarm stimuli and for programming aspects of the experimental software, Richard Morey for statistical advice, and Associate Professor Marcus Watson for help in earlier phases of the research. The authors thank Dr. Chris Thompson, Dr. Daryle Gardner-Bonneau, and Dr. Edward Israelski for insights into the development and/or implementation of the alarms, and also Professor Judy Edworthy plus two anonymous reviewers for valuable comments on the manuscript.
Accepted for Publication August 3, 2007. Supported by a UQ postgraduate research scholarship to Alexandra Wee. Reprints will not be available from the author.
This article has been cited by other articles:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|