| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
BACKGROUND: Manufacturers recommend maintaining Bispectral (BIS) or Spectral Entropy (State Entropy, SE) indexes between 40 and 60 during the maintenance of anesthesia. We compared these indexes during this period. METHODS: Data were obtained from 58 patients receiving sufentanil-sevoflurane-nitrous oxide anesthesia. The anesthesiologist was blinded to BIS and SE. Artifact-free concurrent BIS and SE values (7792 pairs), automatically recorded at 1-min intervals, were compared using Bland-Altman analysis, Kappa coefficient for agreement and crude proportion of agreement. The occurrence of errors of judgment (Type 1 defined as one parameter <40 and the other >60, or Type 2 defined as BIS and SE values on different sides of a threshold [40 or 60]) was also counted. RESULTS: Bias was –2 with limits of agreement of –18 and 9. Kappa BIS/SE obtained from all patients was 0.537 ± 0.147; crude agreement >0.80 was observed in 45% of patients. Type 1 number of errors of judgment corresponded to two instances. Median and interquartile values of Type 2 number of errors of judgment were 4.5 [3.0–6.0] when considering a difference between BIS and SE more than 5. CONCLUSION: Although limits of agreement between BIS and SE were large, Kappa value moderate, and crude agreement <0.80 in more than half of the patients, making completely contradictory decisions (e.g., deepening the anesthetic based on one parameter and lightening it based upon the other) would have been exceptional. More common would have been a risk of error between no change versus increasing or decreasing anesthetic depth.
Several monitors designed to help the clinician evaluate the depth of hypnosis are currently available. Among them, Bispectral Index (BIS) is a well-known and widely used parameter derived from the patients electroencephalograph (EEG) (1). BIS correlates well with the level of responsiveness and provides an excellent prediction of the loss of consciousness, as reported by Glass et al (2). Among the other available monitors of hypnosis, Spectral Entropy (SE) is a reasonable modality to compare with BIS, because it relies on the extent of disorder in both EEG and electromyography signals and returns two values: SE and Response Entropy (3), SE being an index of the depth of hypnosis. BIS and SE are dimensionless numbers scaled from 100 to 0 for BIS and from 91 to 0 for SE. These parameters can also be treated as categorical variables according to the manufacturers recommendations, whereby values of <40 are consistent with deep anesthesia, values between 40 and 60 are the target range and values more than 60 are consistent with light anesthesia. Aspect Medical claims that values below 60 are typically considered to correlate with adequate anesthetic level ("At a BIS value of <60, a patient has an extremely low probability of consciousness") and recommends maintaining BIS between 40 and 60, a range that "ensures adequate hypnotic effect during general anesthesia while improving the recovery process" (4). An upper limit of 60 has been validated by studies in which maintaining BIS below 60 decreased the incidence of intraoperative awareness (5,6). The advisability of a lower limit of 40 has been disputed, because it has been reported that cumulative deep hypnotic time with a BIS lower than 45 may be an independent predictor of mortality in the first year after major noncardiac surgery (7). However, the range of 40–60 for BIS remains widely used and is reported in many publications, despite large variability and overlap in BIS scores at distinct depths of anesthesia (8–10). On the other hand, the SE manufacturer specifies in the "Entropy Range Guidelines" that SE must be between 40 and 60, an interval of "clinically meaningful anesthesia" with "a low probability of consciousness" (11). As BIS and SE were designed to measure the level of the hypnotic component of anesthesia, they should be expected to provide clinicians with similar information. However, Soto et al. reported a patient monitored simultaneously with BIS and SE, for whom the monitors provided different information (12). We have also seen differences between BIS and SE in some patients, which led us to perform this prospective observational study, undertaken during the maintenance period of sufentanil-sevoflurane-nitrous oxide anesthesia, which aimed to verify whether BIS and SE are similar in value and provide concordant information regarding "too deep" (index <40) and "too light" (index >60) anesthetic states. Assessing the comparability between BIS and SE using usual statistics, such as Bland-Altman analysis (13) appeared to us insufficient to make this study clinically relevant. Consequently, we assessed interrater reliability by using (a) the unweighted Kappa statistic, which measures, for the whole data set, the concordance beyond chance between measurements of categorical data (i.e., "too deep," "adequate," and "too light" in our case) (14), (b) the proportion of crude agreement between the indices for each patient, and (c) the count of number of errors of judgment (NEJ) corresponding to instances where the BIS and SE monitors displayed conflicting information that could result in different, or even opposite, anesthetic titration.
With ethical committee approval and written informed consent, 58 adult patients, aged 18–80 yr, ASA physical status I, II, or III, scheduled for elective abdominal, gynecologic, urologic, or orthopedic surgery expected to last at least 1 h, were evaluated. Noninclusion criteria included a history of any disabling central nervous or cerebrovascular disease, hypersensitivity to opioids, substance abuse, treatment with opioids or any psychoactive medication, and a body weight of <70% or more than 130% of ideal body weight. No patient received local anesthesia or regional block combined with general anesthesia.
All patients received hydroxyzine 100 mg orally 1 h before surgery for premedication. In the operating room, an IV catheter was inserted into a large forearm vein and standard monitors were applied (S/5TM monitor; Datex-OhmedaTM, Helsinki, Finland). After the skin of the forehead had been carefully wiped with an alcohol swab and then allowed to dry, the BIS® and EntropyTM self-adhesive EEG electrode strips (ZipPrep; Aspect Medical Systems, Newton, MA) were positioned on the forehead. The SE sensor was positioned just below the BIS sensor. The side of the forehead for electrode placement of the two monitors was randomly assigned. The BIS and SE plug-in modules were connected to the same S/5 monitor. The sampling rate for the raw EEG was 400 Hz for SE and 256 Hz for BIS. The BIS (BIS version 4.0, XP) was calculated with a smoothing rate of 30 s; the moving average window used to calculate SE was 15–60 s. Electrode impedances were considered as acceptable if <10 k Patients received a standardized anesthetic according to the following procedure. After administration of 100% oxygen, anesthesia was induced with propofol 2–3 mg/kg IV and sufentanil 0.2–0.3 µg/kg IV. After loss of consciousness, oxygen was given by facemask ventilation, and patients received atracurium 0.5 mg/kg IV. After tracheal intubation, the lungs were mechanically ventilated with a tidal volume of 8–10 mL/kg, with the ventilatory rate adjusted to maintain an end-tidal carbon dioxide concentration (partial pressure) of 30–35 mm Hg. Anesthesia was continued with sevoflurane in 60% nitrous oxide with oxygen, sufentanil 0.15–0.20 µg · kg–1 · h–1 IV with a 5 µg bolus administered 5 minutes before surgical incision, atracurium 0.3 mg · kg–1 · h–1 IV initially and thereafter adjusted according to train-of-four monitoring. The fresh gas flow rate was set to 6 L/min until the difference between inspiratory and end-expiratory sevoflurane concentrations was equal to or <0.2%; fresh gas flow rate was then reduced to 1 L/min. The beginning of maintenance of anesthesia was defined as this time. Anesthesiologists were instructed to guide the titration of general anesthesia using routine clinical signs. The sevoflurane concentration was increased or intermittent bolus doses of sufentanil 5–10 µg IV were given in case of hypertension or tachycardia. Nicardipine 1–2 mg or esmolol (dose chosen by the anesthesiologist) IV was given if necessary. Hypotension was treated with IV fluid replacement or by a decrease in sevoflurane concentration and, finally, by ephedrine 3–6 mg IV or phenylephrine 20–100 µg IV if it was judged necessary. The sevoflurane concentration was decreased in case of bradycardia or IV atropine 0.5–1 mg IV was administered. Sevoflurane administration was discontinued at the beginning of skin closure. The fresh gas flow was increased to 6 L/min of pure oxygen at the end of skin closure, which was defined as the beginning of the recovery period. Residual neuromuscular blockade was reversed with atropine 15 µg/kg IV and neostigmine 0.05 mg/kg IV if necessary. BIS, BIS signal quality index, and SE values recorded at 1-min intervals were transferred to a computer hard disk using the software program Datex-Ohmeda S/5 Collect (version 4.0) for off-line analysis. Finally, all patients were visited on the first and third postoperative days and interviewed about any memories or recall of intraoperative awareness using a standardized interview (15).
Statistical Analysis BIS and SE values were compared using Bland-Altman analysis performed by plotting the differences between simultaneous values of BIS and SE versus their average (13). Bias was represented by the median value of the differences between BIS and SE, because these differences did not exhibit a Gaussian distribution (16). Upper and lower limits of agreement were defined as 2.5 and 97.5 percentiles of the distribution of the differences. Precision, the ability to reproduce the same measurement, was represented by the interval [bias – limit1; bias + limit2], limit1 and limit2 corresponding respectively to the first and third quartiles of the differences distribution.
Kappa coefficient of agreement was calculated between BIS and SE values considered as categorical values using 40 and 60 threshold values. The three resultant categories (below 40, between 40 and 60, and more than 60) indicate for each parameter whether it corresponds to an excessively deep, adequate, or inadequate hypnosis level. Two BIS/SE pairs separated by Agreement between BIS and SE on a per patient basis was quantified using the aforementioned proportion Po, which is the crude proportion of agreement between the two indices.
An error of judgment (EJ) was classified as Type 1 EJ if one parameter, BIS or SE, is less than 40 and the other greater than 60 simultaneously, or as Type 2 EJ if simultaneous BIS and SE values are on different sides of a threshold (40 or 60) with various degrees of difference between BIS and SE. To investigate the influence of this difference on the judgment outcome, three subtypes of Type 2 EJ were defined, each with a minimal absolute difference ( Demographic data are expressed as count, mean value ± sd and range (values in brackets) when their distribution was normal and as median and 25–75 percentiles in other cases [values in square brackets]. Kappa coefficient is expressed as value ± 95% confidence interval. Proportion of agreement is expressed as median and 25–75 percentiles [values in square brackets]. NEJ is expressed as a count for Type 1 NEJ and as median and 25–75 percentiles for Type 2 NEJ. A P value <0.05 was considered to be statistically significant in all analyses. Data analysis was performed using SAS® version 8 (SAS Institute Inc., Cary, NC).
Twenty-two men and 36 women with a mean age of 54.5 ± 14.6 (28–70) yr, weight of 69.8 ± 13.6 (46–110) kg, and height of 165 (161–170) cm were evaluated in this study. Twenty-eight patients were classified ASA 1, 27 patients ASA 2, and 3 patients ASA 3. They were scheduled for abdominal (20), gynecological (19), urological (13), or orthopedic (6) surgery. The median duration of anesthesia maintenance was 105.5 (79.0–169.0) min. No patient reported intraoperative recall at the follow-up interviews. BIS-signal quality index was lower than 50 and SE was lacking during 446 and 169 min of monitoring respectively; and these data were eliminated from the analysis. The data which were analyzed included 7792 concurrent artifact-free paired BIS and SE values. Distribution of BIS and SE according to the threshold values of 40 and 60 is presented in Table 1. Twenty-four percent of all the recorded pairs corresponded to instances when BIS and SE values were on different sides of a threshold (40 or 60). Only two pairs corresponded to situations where one parameter was lower than 40 and the other one was greater than 60.
Figure 1 shows the Bland-Altman plot of all the data points of the maintenance period, reflecting the differences between each paired reading from BIS and SE. Bias, the median difference between BIS and SE, was –2; the upper and lower limits of agreement were –18 and 9. Precision limits were –7 and 2.
The Kappa BIS/SE coefficient, calculated using the 52 patients for whom duration of anesthesia maintenance was
Type 1 NEJ occurred in 2 instances of the 7792 paired measurements. Median and interquartile values of Type 2 NEJ are presented in Table 2, both for the whole period of maintenance of anesthesia and per hour.
Our observational study of simultaneously measured BIS and SE values during maintenance of sufentanil-sevoflurane-nitrous oxide anesthesia demonstrates weak comparability between them, as shown by large limits of agreement (Bland and Altman analysis) and Kappa value indicative of moderate agreement. However, making completely contradictory decisions (e.g., deepening the anesthetic based upon one parameter and lightening anesthetic based upon the other) would have been an exceptional consequence of the observed difference. More common is the risk of an error between no change versus increasing or decreasing anesthetic depth. SE has been compared favorably to BIS using clinical end points [loss of verbal contact or loss of consciousness (17–21)], Observers Assessment of Alertness/Sedation Scale (22,23), or pharmacological data such as measured (24) or calculated plasma concentrations of propofol (19,25), and end-tidal concentrations of sevoflurane (26). But most of these studies focused on the transition between awake and anesthetic states (18–20,23,25,26) or between high and low end-tidal sevoflurane concentrations (26). Studying such transition periods may not provide a true assessment of agreement, as the indices compare more favorably during induction and emergence because the clinical changes are more dramatic. Focusing the comparability study on the maintenance of anesthesia is important, because anesthesiologists will administer anesthetics according to the value of a hypnosis index. Although the reliability of BIS as a depth of anesthesia monitor is still controversial (27), comparing devices such as Narcotrend (28), Patient State Index (29), or SE with BIS is an appropriate, although not ideal, method in the absence of firm clinical end points that correlate with the depth of anesthesia. Clinicians use MAC and some variants (MACawake and MACbar) and minimum infusion rates or plasma concentrations (50% and 95% effective infusion rates or plasma concentration preventing the movement response to noxious stimuli) values to guide volatile and IV anesthetic administration respectively. More recently, commercially available target-controlled infusion systems estimate plasma and site-effect concentrations using pharmacokinetic models. Conducting anesthesia using pharmacokinetic parameters is complicated by the combination of drugs (usually an analgesic, a hypnotic, and a muscle relaxant) and would be easier if clinicians had the possibility of measuring the depth of hypnosis. Strictly speaking, BIS and SE do not measure hypnosis directly but, rather, are intended to indicate if anesthesia is "not too light" or "not too deep." Studies performed during maintenance of anesthesia using various statistical analysis tests report divergent results. A recent observational study performed during laparoscopic surgery showed that mean BIS and SE values are not significantly different, but that the standard deviation is large (21). A high correlation coefficient has been reported between BIS and SE during minor gynecological surgery (17) and during cardiopulmonary bypass (30). Another study reported a sigmoidal correlation between these variables (26). Finally, BIS and SE have been compared using the Bland-Altman method with a bias of 0.1 and upper and lower limits of agreement of +19.9 and –19.6 (19). Bland-Altman analysis is a useful statistical method to evaluate the degree of agreement between two measurement techniques. Theoretically this analysis is only accurate if a single pair of values is obtained from each patient. However, it is common in the literature to find Bland-Altman analysis with a batch of values for each patient. We report limits of agreement, much more interesting information regarding the degree of agreement than the bias, close to values reported in studies performed during the induction phase of anesthesia (19,20). Reporting limits of agreement begs the question of which limits would be considered "acceptable." There is no consensus to determine what limits are acceptable and the opinions are empiric and not based in fact. Bonhomme and Hans (31) argued that the definition of acceptable limits of agreement must be based on the magnitude of the variable scales and on the clinical relevance of the chosen interval; they suggested that limits of agreement of ±10 between BIS and SE would be reasonable. We have a divergent opinion, because this interval (e.g., 20) is of the same magnitude as the range 40–60, the manufacturers recommended limits. We report 95% limits of agreement of –18 and 9; this interval expressed as an absolute value is more than 20 and thus seems unacceptable. The mathematical approach provided by Bland-Altman analysis should be complemented by Kappa value of agreement and by the calculation of the NEJ, which are more clinically relevant parameters, because they provide insight into how the discrepancies between two monitors might influence clinical decisions. Kappa value of agreement, per patient crude agreement and NEJ, were based upon thresholds (40 and 60) in concert with most clinical studies and those of the manufacturers (4,11). Such thresholds are easily accepted in clinical practice because they provide a simple way to administer hypnotic drugs: too much, enough or too little. The strength of agreement between BIS and SE is mostly moderate according to the classification proposed by Landis and Koch (14). Few studies have compared BIS and SE as categorical variables. Tiren et al. (30) reported during cardiopulmonary bypass that only 62/109 simultaneous recorded pairs of BIS and SE values were in agreement (i.e., both values <40 or between 40 and 60, or more than 60). Schmidt et al. (17) reported, during minor gynecological surgery, that BIS values of 40–65 were associated in 84% of the values with SE values of 30 to 59; these data are displayed incidentally in their article without any information about the method used. NEJ calculation helps to clarify the clinical implications of our study, because this parameter measures the differences in judgment, which could have happened, and consequently provides insight into how the discrepancies between two monitors might influence clinical decisions. Making completely contradictory decisions (e.g., deepening the anesthetic based on one parameter and lightening it based on the other) would have been exceptional (only 2 of 7792 paired BIS and SE values). A risk of error between "no change" versus "increasing or decreasing" anesthetic depth would have been more common, as indicated by the fact that median values of such errors during the whole anesthetic period range from 4.5 to 1, depending on the considered interval between BIS and SE values (from 5 to 15). Because BIS and SE use different algorithms, it is not surprising that BIS and SE monitors provide different instantaneous results. The BIS proprietary algorithm is based on several EEG-derived parameters that have been published (32), but the exact degree to which each of the underlying parameters contributes to the index value at different ranges is proprietary. Description of the Entropy algorithm as applied in the Datex-Ohmeda S/5 Entropy Module has been published in full (3). SE is computed over the frequency range of 0.8–32 Hz and reflects the level of hypnosis, whereas Response Entropy is computed over the frequency range of 0.8–47 Hz and reflects electromyographic activity. There are some limitations to our study. Patients received an opioid and nitrous oxide as a component of the anesthetic technique. Opioid contribution to the clinical depth of hypnosis was not clearly captured by the monitors, but it has been demonstrated that during painful stimulations, as in our study which was focused on the maintenance phase of anesthesia (surgical phase), a lack in antinociception is reflected by an increase in BIS (33). A similar figure is not expected from SE, but rather from Response Entropy (3), a parameter not studied in this work. Supplementing sevoflurane with nitrous oxide induces a decrease in SE and no modification of BIS whereas supplementing sevoflurane with nitrous oxide, with a concomitant reduction in sevoflurane concentration to maintain a constant MAC, resulted in an increase in both BIS and SE (34). The last figure corresponds best to our study design, consequently use of nitrous oxide could not be considered as a potential confounder. Although some guidelines were provided for the titration of general anesthesia, there were obviously some subjective changes in anesthetic delivery. Another choice would have been to allow the clinicians access to BIS, SE, or both. A potential risk of such a proposition would have been to obtain most BIS and SE values in a narrower range. We did not randomize the location of each sensor. The BIS sensor was always below the SE sensor because we thought that this position was the closest to the manufacturers recommendations. It is possible that this could influence the results, because the lower sensor is furthest from brain tissue and the frontal sinuses could limit the signal conduction. However, a thorough reading of the data shows that the difference between concurrent BIS and SE values for each patient were negative or positive in a predominantly random pattern, even on consecutive readings. The significant intrapatient variability in BIS-SE differences makes this hypothetical limitation unlikely to be clinically relevant. Finally, we compared BIS and SE without considering the reproducibility of each of these variables, which has been recently disputed, particularly with the presentation of a patient who displayed conflicting concurrent BIS values (35). In conclusion, despite wide clinical use of monitors of hypnosis, their comparative performance is not well established. Ideally, two monitors intended to provide the same information should yield comparable data. This was not the case in the current study, as demonstrated by the results of Bland and Altman analysis and the Kappa value. Most important is that BIS and SE could lead the clinician to make different decisions between no change versus increasing or decreasing anesthetic depth. There is no evidence to suggest that one index is truly better than the other.
The authors thank GE-Datex who loaned the authors an S5 monitor and the probes.
Accepted for publication July 19, 2007. The work should be attributed to the Department of Anesthesiology, Hôpital Foch, Suresnes, France. No financial relationship between any of the authors and any commercial brand. Address for correspondence and reprints requests to M. Fischler, Department of Anesthesiology, Hôpital Foch, 40 rue Worth, 92151 Suresnes, France. Address e-mail to m.fischler{at}hopital-foch.org.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|