| JOURNAL HOME | CME HOME | THIS MONTH | PAST ISSUES | ETOC | COLLECTIONS |
| AUTHORS | REVIEWERS | EDITORIAL BOARD | FEEDBACK | RSS | HELP |
| ||||||||||||||
| ||||||||||||||
|
|
|
|
||||||||||||
|
||||||||||||||
*Department of Anesthesiology and the Human Simulation and Patient Safety Center of Duke University;
Global Perioperative Research Organization; and
Duke Clinical Research Institute, Durham, North Carolina
Address correspondence and reprint requests to Jeffrey M. Taekman, MD, Department of Anesthesia, Box 3094, Duke University Medical Center, Durham, NC 27710. Address e-mail to jeffrey.taekman{at}duke.edu
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
IMPLICATIONS: A complex clinical research protocol was replicated in the Duke University Human Simulation and Patient Safety Center. Clinical research coordinators trained in simulation showed significantly greater confidence in their ability to perform the research protocol. We conclude that simulation training should be considered as a training method of research coordinators.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Little attention has been paid to the study of research personnel education, learning, competence, and performance. Current research coordinator training includes methods such as self-study, investigator meetings, and lectures. Modern theories of adult learning highlight the value of interactivity, immediate feedback, integration of experience, and placing information in context (11). Interactivity, as achieved in simulation, has been shown to be a superior method of teaching desired behaviors to medical professionals (12,13).
Therefore, using confidence as a surrogate for competence, we tested the hypothesis that the addition of an interactive simulation exercise to the education of research coordinators training to achieve mastery of a complex perioperative trial protocol would increase their confidence compared with traditional methods alone.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
A nine-question self-assessment (Appendix A) was designed to probe the 3 areas of learning as defined by Bloom and Krathwohl (6): psychomotor, cognitive, and affective. The questionnaire consisted of three psychomotor questions, two cognitive questions, and three affective questions targeted at key components of the trial. The questionnaire used a scale from 0 (not confident) to 10 (confident). In addition, there was a single question item that addressed participants’ overall impression of simulation as a learning method. Each study coordinator completed the questionnaire before and after their simulation experience.
Table 2 demonstrates representative behaviors of this complex clinical protocol. Psychomotor tasks were those involving equipment setup, study drug administrations, and data capture. Cognitive tasks involved performance of calculations, timing and administration of drugs, and correct responses to alterations in patient condition. Affective tasks were comfort with the study environment, equipment, and the ability to communicate effectively with operating room personnel.
|
Each study coordinator received a packet of mock study drug syringes and data collection sheets for use in the simulation exercise. Each coordinator was expected to treat the simulated patient as their first enrolled study patient. The protocol required frequent interaction and collaboration with the anesthesiologist. Representative activities include: (a) requests for procedural times, (b) requests for blood samples, (c) request for study drug administration, (d) request for dosages and times of standard drugs administered, and (e) notification of alterations in clinical status. The clinical trial principal investigator (MSS) was embedded in the exercise giving personalized, constructive, real-time feedback on the coordinator’s performance. Each subject, in turn, coordinated the simulated clinical trial in the simulation center. Study coordinators were expected to communicate with the anesthesiologist at appropriate times in the simulated trial, direct the timing of the study drugs, and collect data. The study coordinator was expected to recognize and vary the protocol in response to clinical cues.
The experience was observed by all other coordinators via a live video feed in a distant room. A facilitator was present in the observation room to answer questions during the simulation exercise. The experience was nonevaluative.
The audio and video of the simulation exercise were captured to mini digital videotape and to a computerized digital video array. After the 15- to 30-min exercise, the principle investigator, study subject, observing study coordinators, and coordinator facilitator gathered to debrief on the simulation. The debrief covered key positive and negative behaviors observed by the other study coordinator and principal investigator. The process was then repeated. Each study coordinator worked through the protocol in the simulation center. The simulation duration of 15–30 minutes was developed as a practical compromise between concerns of reduced educational value from a time-compressed exercise and the diminishing benefit monitors seemed to derive during our pilot simulation exercises from observing more than four to five simulations over 4 h.
The primary analysis was the mean change in questionnaire responses from presimulation to postsimulation (combining affective, cognitive, and psychomotor domains). Secondary analyses examined the four subcategories separately. Pre- and postscores were compared using paired t-tests. The study-wide significance level was set at 0.05. Bonferroni adjustment for 4 multiple comparisons required each individual test to have a P value of 0.01 or less to be considered significant.
Although questionnaire items were chosen to adhere to the three domains of Bloom’s Taxonomy, we performed a confirmatory factor analysis to verify that our items conformed to this structure (14).
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Today’s clinical trials require complex behaviors and an extensive comprehension of the study protocol. Mastery of a complex protocol requires a significant learning curve. Training of research coordinators fails to take into account one of the basic tenants of modern learning theory, interactivity. Current coordinator training consists of focused self-study, lectures, and small-group discussions. We believe a great deal of learning currently takes place at the patient’s bedside with enrolled patients. Simulation is a training method used extensively in the military, aviation, and nuclear industries to shorten the learning curve and improve performance. Many education and technology experts consider simulation the most powerful modality available for teaching complex integrative tasks.
Many studies have examined the role of simulation in health care professional training (15–20). Simulation allows the learning of complex behaviors without the risk to patients. An emerging body of work demonstrates improved real-world performance for those trained in simulation (21). Simulators are also used to examine the cause of medical mishaps and limitations in equipment and human performance (20,22,23). Simulators should similarly be used to discover, analyze, and resolve issues with clinical research protocols.
There were several limitations to our study. Study logistics limited our ability to designate a control group or randomize participants in this study. Because we lacked a control group, we could not determine if the effect seen was because of the simulation exercise or the longer nonspecific exposure to the trial protocol. Our future studies will attempt to include a control group and subject randomization.
There were several notable limitations to our survey instrument. We used confidence as a surrogate measure of competence. Ultimately, we are interested in assessing coordinator competence in performing the complex tasks of a clinical trial. Although there is literature demonstrating a correlation between confidence and competence, our future studies will use objective measures of coordinator performance and data integrity to assess competence directly.
Miller (7) described the progression of competence through a series of interdependent steps represented as a triangle. A learner may be able to achieve the "knows" or "knows how" levels of competence through self-study. But only through performance of the protocol could they demonstrate the higher levels of competence ("shows how" and "does").
Cognitive gains using other less labor-intensive forms of simulation (24) have been described. However, we suspect the greatest psychomotor and affective gains will be made by training in high-fidelity simulated or true clinical environments. Training in simulated environments offers the additional benefit of not placing patients (or study data) at risk. Regardless of the cause, the ability to affect performance argues for a hands-on simulation experience.
There are several potential problems with the simulated environment. Despite the sophistication of today’s simulation centers, learners realize they are working in an artificial environment. Adding to the artificiality is the need to compress time (compressing the key components of a 3 hour procedure into a 15–20 minute simulation) to accommodate all participants. Further studies need to address whether time compression during training has any effect on real-world performance.
In addition, participants tend to be hyper-aware during a simulation exercise. Many expect a problem to occur with every training session. This heightened state of awareness likely leads to greater vigilance and faster response times of the subjects when reacting to alterations in the protocol. Thus, what is seen in the simulation center may or may not reflect real-world behavior. We attempted to control this phenomenon by randomly choosing between normal and abnormal pathways during the simulation.
The application of simulation in medicine has been limited to students, physicians, nurses, and para-health professionals involved in direct patient care. Our study demonstrates that simulation centers may be used to train other personnel in vitally important medically related activities. We conclude that simulation should be considered in the training of study coordinators involved in complex clinical trials. Use of simulation in training study coordinators will expand the potential user-base of simulation centers and will contribute to improved patient safety through safer trials and better data integrity.
In this study, there were several interesting observations that deserve further investigation. We noticed a wide disparity of coordinator performance in the simulated protocol. The cause of this disparate performance was unclear. Potential sources of this observation include differing personality styles, differing or aberrant learning styles, teacher-learner disparities, or differing levels of preparation. Further investigation is warranted to uncover the cause of these disparities. A few coordinator performances raised an important ethical question: Should simulation be used to screen individuals involved in complex (especially potentially life-threatening) trials? This question requires further investigation.
Several individuals had combined confidence scores that worsened after the simulation exercise. The reality check of working through the replicated protocol brought these aberrant scores in line with other subjects. The worsening of confidence after simulation highlights a limitation of using confidence as a surrogate for competence. Although confidence worsened, performance may have remained the same (or even improved). The root cause of overconfidence, the effect of overconfidence on protocol performance, and the effect of simulation in correcting overconfidence are all areas for further investigation.
Another observation worth noting is that several changes were made to the protocol by the principle investigator after observing the study coordinator’s performance in simulation. Observation of the study coordinators brought to light several timing, data collection issues, and physical task impediments that were not obvious during the paper-and-pencil preparation of the protocol. The potential use of high-fidelity simulated environments for complex clinical trial protocol development warrants further investigation.
Clinical trials have a wide range of complexity and inherent danger to the patient. The cost of developing and delivering a simulated protocol training experience is significant. Further studies need to be conducted to determine the type of clinical trials most amenable to simulation.
In summary, the current work is an assessment of the role of simulation in the education of practitioners of clinical research. We believe our preliminary findings identify potential ways to improve the quality of clinical research including patient safety issues relative to other teaching methods; however, it will be important to confirm our thesis. This is particularly significant because many other alternate teaching modalities (e.g., one-on-one teaching, group teaching, videotape review of procedures, and web-based instruction) are significantly less costly. In the next phase of our studies, we hope to show, using objective measures of performance, that simulation training improves the quality of data obtained in a clinical trial compared with a control group that receives an equivalent amount of traditional training. Our ultimate goal is to improve both the safety and efficiency of clinical research trials.
|
|
|
Acknowledgments |
|---|
|
Footnotes |
|---|
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  C. L. Lake Simulation in Cardiothoracic and Vascular Anesthesia Education: Tool or Toy? Seminars in Cardiothoracic and Vascular Anesthesia, December 1, 2005; 9(4): 265 - 273. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
