| JOURNAL HOME | CME HOME | THIS MONTH | PAST ISSUES | ETOC | COLLECTIONS |
| AUTHORS | REVIEWERS | EDITORIAL BOARD | FEEDBACK | RSS | HELP |
 
|
|
|
||
|
|
Anesth Analg 2004;98:1354-1358 © 2004 International Anesthesia Research Society doi: 10.1213/01.ANE.0000105869.91626.3A
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
,
, |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
The testing of pulse oximeter probes is generally limited to the integrity of the electrical circuit and does not include the optical properties of the probes. Few pulse oximeter testers evaluate the accuracy of both the monitor and the probe. We designed a study to compare the accuracy of nonproprietary probes (OSS Medical) designed for use with Nellcor, Datex-Ohmeda, and Criticare pulse oximeter monitors with that of their corresponding proprietary probes by using a commercial off-the-shelf pulse oximeter tester (Index). The Index pulse oximeter tester does include testing of the optical properties of the pulse oximeter probes. The pulse oximeter tester was given a controlled input that simulated acute apnea. Desaturation curves were automatically recorded from the pulse oximeter monitors with a data-collection computer. Comparisons between equivalent proprietary and nonproprietary probes were performed. Data were analyzed by using univariate and multivariate general linear model analysis. Five OSS Medical probe models were statistically better than the equivalent proprietary probes. The remainder of the probes were statistically similar. Comparative and simulation studies can have significant advantages over human studies because they are cost-effective, evaluate equipment in a clinically relevant scenario, and pose no risk to patients, but they are limited by the realism of the simulation.
IMPLICATIONS: We studied the performance of pulse oximeter probes in a simulated environment. Our results show significant differences between some probes that affect the accuracy of measurement.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Monitoring the arterial blood oxygen saturation (SpO2) with a pulse oximeter is a de facto medical standard for patients at risk for oxygen desaturation. Pulse oximetry systems consist of the pulse oximeter monitor and a probe that is applied to the patient (usually to a finger, ear, or toe).
Pulse oximeters and their probes are usually evaluated for accuracy by using humans breathing hypoxic gas mixtures to reduce their oxygen saturation to levels as low as 70% (1–3). Because it is not possible to measure PaO2 continuously, another pulse oximeter is typically used to determine the point in time when the next measurement should be taken, which can lead to very low PaO2 values. Human trials are expensive and time consuming and require that the patient have an arterial line so that data from the pulse oximeter monitor can be compared with arterial blood gas data. Arterial lines have been associated with considerable morbidity (4).
We studied the compatibility and functional equivalence of a new brand of pulse oximeter probes compared with those already on the market. When using the same monitor and input signals for the purpose of comparing the new brand of compatible pulse oximeter probes with their equivalents, patient testing is not required if the input signals are reproducible and adequately simulate the optical properties of a finger.
Pulse oximeters are routinely tested in hospitals to ensure safe and accurate operation. There are, however, few pulse oximeter testers that evaluate the accuracy of both the monitor and the probe. Generally, probe tests are limited to testing the electrical wiring of the probe. The Index (Bio-Tek Instruments, Inc., Winooski, VT) is one pulse oximeter tester that tests both the probe’s monitor and its optical properties. This device is called a "transfer standard." This standard can be used to take measurements of an unknown device (a probe and monitor, in our case) and evaluate how closely it compares to a known device with the same transfer standard (in our case, monitors and probes that have been clinically validated).
The Index contains a simulated finger that attenuates the light emitted by the light-emitting diodes in the probe. The attenuation is accomplished by measuring the light intensity with a photo detector and emitting an attenuated replica of the original signal on the other side of the simulated finger. Light attenuation depends on the wavelength of the light emitted by the pulse oximeter probe and the saturation level programmed into the Index (5). Unlike other pulse oximeter testers, no electrical connection between the probe and the tester is required; this allows simulation of the optical properties of the probes. This device is approved as a testing device by the Food and Drug Administration (FDA) [510(k) approval K971273]. Performance data from the 510(k) application were obtained from the FDA under a Freedom of Information Act request. The performance data are plotted in Figure 1. The data are within the 1% deviation specified in the performance specification of the Index.
|
We designed a study to compare the accuracy of nonproprietary probes (OSS Medical, Singapore) designed for use with Nellcor, Datex-Ohmeda, and Criticare pulse oximeter monitors with that of their corresponding proprietary probes. These studies were performed with the Human Patient SimulatorTM (HPS) (Medical Education Technologies, Inc., Sarasota, FL) to simulate an acute apneic event that resulted in SaO2 values as low as 60%. The HPS controlled the SaO2 input of the Index pulse oximeter tester. The acute apneic event was selected because it more closely resembles a real-world clinical scenario. Each OSS probe was compared only with its equivalent brand probe. Inputs to the tester, pulse oximeter, and other influences were kept constant. The only difference was the probe. The null hypothesis that the probes would be equivalent was used.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The HPS contains physiologic models that provide values for cardiovascular variables, such as heart rate, arterial blood pressure, and blood oxygen saturation (6). These models were used to control the SpO2 and heart rate of the Index pulse oximeter tester. The HPS inhales oxygen and exhales carbon dioxide at physiologic levels. Respiratory muscle action can be set in the computer model and was used in this study to create 100% neuromuscular blockade and, thus, apnea in the simulated patient. When the neuromuscular blockage was set to 100%, the blood SaO2, based on the physiologic model of the HPS, decreased gradually from 98% to 65% in approximately 2 min—the expected time course (7). Software was written to simultaneously collect output from the HPS and the pulse oximeter (Nellcor N-180, Ohmeda Biox 3700, or Criticare 504) (8). The saturation input to the Index tester was considered the "gold standard." The monitors were set to the fastest averaging mode (generally 3 s) and communicated an output of one reading per second (resolution was 1% saturation). Each proprietary probe (Nellcor, Ohmeda, and Criticare) was tested against its equivalent nonproprietary probe (OSS Medical).
A pilot study was designed to determine the variability caused by the measurement (variability in the generated desaturation curve and in the pulse oximeter tester) and to evaluate the number of probes that should be tested, the number of times each probe should be tested, and the number of times each probe should be placed on the pulse oximeter tester. The pilot study was conducted with three proprietary (Nellcor Durasensor) and three nonproprietary (OSS 2010) probes. For each probe, three runs were performed; the probe was placed on the Index pulse oximeter tester for one run, was removed and replaced for another run, and was finally removed and replaced for the last run. This evaluated the intraprobe variability, the effect of finger position on the pulse oximeter tester, and other factors that might influence the experiment.
Consultation with a statistician after completion of the pilot study revealed that it was unnecessary to remove and replace the probes between runs. Therefore, for the subsequent studies, five probes of each model were used, with three runs performed without removal and replacement of the probes on the tester. Three Ohmeda and five Criticare models were evaluated against the equivalent OSS Medical models.
Data were analyzed by using a univariate and multivariate SAS (SAS Institute, Cary, NC) general linear model procedure. For this study, only the differences in the data sets were evaluated. Statistical differences were not studied at specific saturations, but rather the whole desaturation curves were evaluated. Thus, although there may be statistically significant differences between the saturation curves, individual saturation values may not be significantly different (P < 0.05 was considered statistically significant).
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
There was no statistically significant difference among the saturation curves as generated by the simulator (set SaO2 values) over the different experiments. There was no statistical difference among the multiple runs of the same probes. We conclude that our experimental design is sound and does not contribute to any significant difference in the results.
Five OSS Medical probe models were statistically different from the equivalent proprietary probe (Table 1). In all these instances, the OSS Medical probes were closer to the SaO2 selected on the Index. All probes performed better at the higher SaO2 values (Table 2). Generally the Nellcor and the equivalent nonproprietary probes showed an SpO2 that was less than that calculated by the HPS. The Ohmeda and Criticare and their equivalent nonproprietary probes showed an SpO2 that was higher than that calculated by the HPS.
|
|
Our study also demonstrates differences between different probes of the same model and manufacturer (Fig. 2). However, these differences were not statistically significant. Data from this study, presented in standard Bland-Altman plots (9), show the percentage pulse oximeter SpO2 probe deviations from the Index SaO2 (Fig. 3). We have used the Index SaO2 as the standard.
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
This study demonstrates that the nonproprietary probes manufactured by OSS are statistically equivalent to or, in some cases, better than their commercial counterparts. We also showed that a simulator can be used to reliably provide input signals for the evaluation of pulse oximeter probes.
Simulation studies have significant advantages over human studies. First, simulation studies are cost-effective. Second, simulators such as the HPS and Index evaluate the pulse oximeter probes in a clinically relevant scenario (a patient with an acute apneic episode). Third, human evaluations can test at only a limited number of SpO2 values because studies at SpO2 values <70% would be unethical. Fourth, use of simulation studies eliminates the potential morbidity associated with the arterial catheters required for human studies. Fifth, simulation studies allow the researcher to gather large amounts of reproducible data in short periods of time. More probes would have been required to study humans because of the marked variability in the human physiologic response to hypoxemia. Although simulation studies are not a substitute for clinical studies, they can help design the clinical studies better by identifying possible problem areas and may reduce the number of subjects needed to show the reproducibility and accuracy of a measurement. Care must be taken, however, to make sure that the simulations are appropriate for a particular study and that the limitations of such simulations are clearly understood.
This study demonstrates statistically significant and probably clinically significant differences in pulse oximeter probes designed for use with the same pulse oximeter monitor from two manufacturers. For 3 of the 6 Nellcor pulse oximeter probes, the equivalent OSS Medical probes are statistically better, as shown by the smaller deviations in SpO2 from the tester. Also, the variability between probes is significantly less, particularly at lower saturation levels. Nellcor pulse oximeter probes tend to overestimate the degree of desaturation. For several Nellcor probes, the difference between the tester reading and the probe reading exceeded the manufacturer’s claim of a <3% error at 70% saturation. Intuitively, it may seem better to have a probe overestimate rather than underestimate the degree of desaturation; however, either case can lead to unnecessary medical intervention. A probe that reads accurately is clearly best.
Our study was designed to evaluate 3 different possibilities for the variability in pulse oximeter probes. First, the study evaluated whether an individual probe produced reproducible results on repeated runs (replication evaluation). Second, the study evaluated whether the position of the probe on the "finger" of the pulse oximeter tester affected the results. Third, the study evaluated probe-to-probe variability. The results indicate that for an individual probe, there is virtually no variation from run to run. Thus, individual probes give reproducible results with a simulator. The study also shows that positioning of the probe on the finger tester has no effect on the results.
This study has some limitations. The probes were studied by using a tester with an artificial instead of a human finger; thus, factors such as pigmentation, arterial vascular disease, and perfusion were not present. The pulse oximeter tester may not have been precisely calibrated for the pulse oximeter monitors used in this study; however, because the same tester was used for all the probes, the results would be valid for comparison among measurements. Finally, the proprietary probes were commercial off-the-shelf production probes, whereas the nonproprietary probes were preproduction prototypes. It remains to be seen whether OSS Medical can maintain the high degree of reproducibility in their production probes.
In conclusion, this study demonstrates that the preproduction prototype probes provided by OSS Medical are equivalent to or better than their commercially available counterparts. We have also shown that a simulator can be used to reliably provide input signals for the evaluation of pulse oximeter probes.
|
Acknowledgments |
|---|
Partial funding of this study was provided by OSS Medical (now Dolphin Medical) under a grant to the University of Florida.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
- Severinghaus JW, Naifeh KH, Koh SO. Errors in 14 pulse oximeters during profound hypoxia. J Clin Monit 1989; 5: 72–81.[Web of Science][Medline]
- Trivedi NS, Ghouri AF, Lai E, et al. Pulse oximetry performance during desaturation and resaturation: a comparison of seven models. J Clin Anesth 1997; 9: 184–8.[Web of Science][Medline]
- Barker SJ. "Motion-resistant" pulse oximetry: a comparison of new and old models. Anesth Analg 2002; 95: 967–72.
[Abstract/Free Full Text] - Weiss BM, Gattiker RI. Complications during and following radial artery cannulation: a prospective study. Intensive Care Med 1986; 12: 424–8.[Web of Science][Medline]
- Haas P, inventor; Bio-Tek Instrumentation, Inc, assignee. Process and system for simultaneously simulating arterial and non-arterial blood oxygen values for pulse oximetry. US patent 6,141,572. October 31, 2000
- van Meurs WL, Good ML, Lampotang S. Functional anatomy of full-scale patient simulators. J Clin Monit 1997; 13: 317–24.[Medline]
- Cooke JE. When pulse oximeters fail: motion and low perfusion [abstract]. Anesthesiology 2000; 93: A554.
- van Oostrom JH, Melker RJ, Whibbs VJ III. Tool to improve physiologic data capture [abstract]. Br J Anaesth 1999; 82 (Suppl 1): 20.
[Abstract/Free Full Text] - Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307–10.[Web of Science][Medline]
This article has been cited by other articles:
|
|
 |
|
  S. Zafar, I. Ayappa, R. G. Norman, A. C. Krieger, J. A. Walsleben, and D. M. Rapoport Choice of Oximeter Affects Apnea-Hypopnea Index Chest, January 1, 2005; 127(1): 80 - 88. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. D. Mannheimer, P. B. Batchelder, M. Larsen, H. Gehring, E. Konecny, J. H. van Oostrom, and R. J. Melker The Use of Pulse Oximeter Functional Testers in Evaluating SpO2 Accuracy * Response Anesth. Analg., December 1, 2004; 99(6): 1871 - 1871. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
