| JOURNAL HOME | CME HOME | THIS MONTH | PAST ISSUES | ETOC | COLLECTIONS |
| AUTHORS | REVIEWERS | EDITORIAL BOARD | FEEDBACK | RSS | HELP |
| ||||||||||||||
| ||||||||||||||
|
|
|
|
||||||||||||
|
||||||||||||||
From the *Department of Anesthesiology, Nucleus for Research in Medical Education, Hospital Governador Celso Ramos, Florianópolis, SC, Brazil; and Center for Development of Medical Education, University of São Paulo Medical School, São Paulo, SP, Brazil.
Abstract
BACKGROUND: We designed this study to determine the academic performance of anesthesia residents as related to their differential characteristics on some affective-motivational variables, represented by perceptions about their educational environment, subjective quality of life, and learning and study strategies.
METHODS: The study sample consisted of 63 anesthesia residents who completed the World Health Organization Quality of Life Inventory, the Dundee Ready Educational Environment Measure, the Learning and Study Strategies Inventory, and a progress test on basic sciences on two to four measurement occasions during a 2-year period. A growth curve model was fit to the academic performance. Mantel-Haenszel tests identified independent predictors of academic performance on progress tests.
RESULTS: Mean rating at the first measuring occasion was 41%. There was a statistically significant improvement over time (slope = 7% per 6-m period; P < 0.01). Analysis of the random effects showed significant individual differences in the intercept. The residents’ scores improved at an equivalent rate over the course of the residency. The independent predictors of academic performance were anxiety, motivation, and ability in selecting main ideas.
CONCLUSIONS: Knowledge growth on basic sciences during anesthesia residency is significantly associated to the level of anxiety related to study and achievement, to the motivation for learning and for personal improvement, and to the ability in selecting main ideas from subject matters to which residents are exposed during learning episodes.
According to the Outcome Project of the Accreditation Council for Graduate Medical Education, resident performance is the measure of each program’s educational efficacy. Outcomes refer to the achievement of competency inpatient care, medical knowledge, practice-based learning, interpersonal skills, professionalism, and systems-based practice (1).
A number of factors may affect resident performance, including learning styles and personality characteristics, social and financial influences, preferences for practices, personal health, and responses to job environment. However, a review of the literature revealed that, resident performance can only be related to personality traits, resident burnout, stress/distress at work, and job satisfaction (2).
Among factors related to individual resident’s life, physical, psychological, social, and environmental issues can potentially affect performance (2). These are domains of the subjective quality of life as defined by the World Health Organization as "the perception of the individual of his position in life in the context of culture and value system in which he lives and in relation to his objectives, expectations, patterns, and worries." There are no available data about the relation between resident performance and the subjective quality of life. However, several of its facets, such as the quality of sleep and rest, the opportunity to get new information, personal relationships, atmosphere at home, and the general quality of life, have been negatively affected by violations of resident duty hours and supervision standards (3).
There is no evidence that resident performance is influenced by the learning environment (2). However, one study showed direct correlation between undergraduates’ perceptions about the learning milieu and their academic achievement. Accordingly, compared with under-achievers, academic achievers scored significantly higher on perceptions regarding teachers, academic atmosphere, and social self-perceptions (4).
Academic performance, as an outcome of residency (2,5), also depends on the quality of residents’ studying and learning strategies. It has been postulated that the way individuals choose or prefer to learn might be compared with a multilayer structure, in which the outer layer represents study and learning strategies, the middle comprises learning styles, and the inner layer represents the personality characteristics. Learning styles remain relatively constant throughout adult life (6). Personality traits remain constant during residency (7), and are hardly amenable to change because of educational interventions. Learning and study strategies can be learned, trained, and improved by specific educational interventions (8,9).
The hypothesis behind the present study was that distinct patterns of perceptions about the subjective quality of life and the learning environment, associated with differing learning and study strategies, could be associated with distinct patterns of knowledge improvement during residency. Based on this hypothesis, the study was designed to determine the academic performance of anesthesia residents as related to their differential characteristics on some affective-motivational variables, represented by perceptions about their educational environment, subjective quality of life, and learning and study strategies.
METHODS
Three training centers in anesthesiology participated in this study, with the respective Ethics Committees’ approval. The three centers are traditional teaching institutions, accredited by the Brazilian Society of Anesthesiologists and by the National Committee for Medical Residency of the Ministry of Education. All institutions are positioned above the 75th percentile in the rank of training centers, promoted by the Brazilian Society of Anesthesiology.
Thirty-three residents admitted in February 2003 and 30 residents admitted in February 2004 composed the study sample. Residents signed written informed consent. The study period was July 2003 through January 2005. Measurement occasions were July 2003, January 2004, July 2004, and January 2005. The instruments included:
–1 logit, 33% between –1 and +1 logit, and 1/3 of item above +1 logit. This item selection approach excluded very easy and very difficult items. Most items (58%) addressed applied pharmacology, followed by applied physiology (23%), anatomy (11%), and physics applied to anesthesiology (9%). This item distribution is in accordance with the basic sciences curriculum established by the Brazilian Society of Anesthesiology (15). Scores on progress tests are presented as percentages of correct responses to the criterion-items instead of logit scores to facilitate interpretation.
Both residents and teaching faculty were blinded to test results. Academic performance data from the subset of subjects entering the study as first-year residents were analyzed using growth curve methods to determine the pattern of changes in academic performance over the study period. Individual growth curves were constructed using a Level I (unconditional) hierarchical linear model. Parameters of individual growth curves (intercept and slopes) were estimated using ordinary least squares. Homogeneity of intercepts and slopes of individual curves were tested by comparing the sum of the squared deviations of the maximum likelihood estimates from the pooled estimate to the corresponding cutoff of the 
2 distribution (16).
Residents were classified as having poor academic performance if they were below the median of each progress test scores for two or more of their evaluations. Otherwise, residents were classified as having good academic performance. The environmental and study strategies tests produced 20 different domain scores for each resident (10 on LASSI, 5 from WHOQOL, and 5 of DREEM). Each of the scores was divided into quartiles, and the outcomes were examined on ROC curves for suggestions of appropriate categories for the statistical analyses. On this basis, scores were grouped into two categories (at or above, or below cutoff points), and 2 tests of association were used to identify which of these measures showed evidence of association with academic performance. Between-cell comparisons were performed by z tests for proportions. A value of P < 0.05 was the criterion used for this screening process.
Our units of analysis were individual measures on each test occasion. We decided that a minimum difference of 20% would be worthwhile to detect in comparing proportions between cells of 2 x 2 tables (intra- and interclass comparisons), which resulted in a minimum sample size of 35 measures per class for one-sided comparisons, at 
= 0.05 and β = 0.2 (Statistica 6.0, Statsoft, Tulsa).
RESULTS
Growth curve analyses were limited to the subset of subjects who entered the study as first-year residents and completed all four evaluations (n = 27). The mean rating at the first measuring occasion (intercept) was 41%. There was a statistically significant improvement over time (slope = 7% per 6-mo period; P < 0.01) (Fig. 1). Analysis of the random effects showed significant individual differences in the intercept [2 (26 df) = 340.7, P < 0.01]. Random effects for slopes were not statistically significant [2 (26 df) = 26.32, P = 0.45]. Reliability of the model was 0.96. Table 1 compares PT scores and model parameters between residents classified as having good or poor academic performances. Significant between-group differences were found between PT scores and intercepts of the growth curves. No differences were found on the slopes.
|
|
Of the 63 participating residents, 3 (5%) left the program for personal reasons. The study sample comprised 39 male and 24 female residents. The median age was 28 yr. Of the 60 participating residents who completed the study, 22 (37%) were classified as having poor academic performance based on their scores on PTs. Not all residents completed tests on the four measurement occasions, so that 161 (89%) of the 180 expected complete data sets were obtained. Sets of 21 variables (10 on LASSI, 5 from WHOQOL, 5 of DREEM, and 1 from the PT) obtained from each resident completing tests on each measurement occasion were defined as units of analysis.
Of the 20 affective-motivational variables, 3 (from LASSI) met our criterion (P < 0.05) for a univariate predictor variable: anxiety, motivation, and ability in selecting main ideas. Table 2 reveals that significant percentages of good performances on PTs were associated with low scores on anxiety, motivation, and selection of main ideas scales, indicating that, above the cutoff points, these variables are poor predictors of good academic performance. Conversely, the percentages of poor performances that were accompanied by high scores on predictor variables were significantly smaller than the percentage of poor performances accompanied by low scores on predictive variables, which suggests that scores below cutoff points on anxiety, motivation, and selection of main ideas scales are good predictors of poor academic performance.
|
Univariate predictors were significantly associated with each other (r(ANX x MOT) = 0.35; r(ANX x SMI) = 0.54; r(MOT x SMI) = 0.36; P < 0.01). Based on this and the relatively small sample size, we believed that these data were not appropriate for multivariate parametric analyses. Instead, data are presented as contingency tables that allowed tests of the independence of association among each of the three univariate predictor variables. The Mantel-Haenszel test was used to test for statistical significance. For these analyses, the remaining variable and the academic performance were included as layers in the 2 x 2 contingency tables to examine the independence of association between pairs of variables (Tables 3–5).
|
|
|
DISCUSSION
This study was unable to detect any significant association between academic performance and residents’ perceptions about their subjective quality of life or about the educational environment of their residencies. These findings might suggest that knowledge gain occurred during residency independently on how residents perceived their quality of life or their educational environment. However, caution should be exercised in interpreting these results. Only violations to resident duty hours standards and low quality of supervision have been shown to negatively affect the perceptions of residents about their learning environment and their quality of life (3). Considering that the participating institutions strictly adhere to residency national standards, as suggested by their position above the 75th percentile of the national ranking (heavily based on adherence to such standards), these findings may have resulted from a selection bias of the convenience sample used in the present study. Further studies also including low-positioned institutions, which fail to follow residency rules, might clarify this issue.
Only factors associated with strategic learning were found to independently predict academic performance. Such factors were anxiety, motivation, and ability to properly select main ideas.
The anxiety scale of the LASSI assesses the degree to which learners worry about their academic performance. High levels of such anxiety can divert attention away from completing academic tasks, such as studying or taking a test (9). Academic performance-related anxiety can be an inhibiting factor in learning by impairing performance in cognitive functions such as attention, memory, concept formation, and problem solving (17). It is closely related to arousal, attention, and motivation. Tests and examinations, which involve a decision or judgment, are common precursors of anxiety in educational settings. Academic performance-related anxiety can be reduced by: 1) instructions that minimize stress and prepare individuals; 2) increased use of positive feedback during a task; and 3) reduced opportunities for failure in a task (18).
The motivation scale assesses learners’ diligence, self-discipline, and willingness to exert the effort necessary to successfully complete academic requirements. Learners who score low on this scale need to accept more responsibility for their academic outcomes and learn how to set and use goals to help accomplish specific tasks (9). Motivation is a pivotal concept in most theories of learning. It is closely related to arousal, attention, anxiety, and feedback/reinforcement. Motivation to achieve is a function of the individual’s desire for and expectancy of success, and the incentives provided (19).
Selecting main ideas signify a learner’s ability to select important information to concentrate on during classes or while reading and studying in order to achieve academic goals. Active listening and metacognitive thinking allows students to identify which topics are most important to learn (9).
The measuring instruments of this study consisted of reliable inventories (9–11). Academic achievement was measured on items showing high measuring reliability and displaying balanced proportions of item difficulties (20). The analysis of individual growth curves showed that there was a significant difference between residents at baseline measurements followed by a modest growth in knowledge over the 2-yr study period, with homogeneity of slopes of individual curves, so that the relative ranking of residents did not change over the study period, regarding their knowledge of basic sciences. In a previous study (21), attitudes toward the relevance of basic sciences to clinical anesthesiology were compared between residents and attending anesthesiologists. Residents were less positive than anesthesiologists toward the construct. They were also more favorable to a more superficial approach represented by learning general concepts instead of a deeper approach, which was preferred by attendings. We hypothesize that lack of perceived relevance of learning basic sciences during residency might, at least partially, explain the modest growth of knowledge found in the present study.
Unconditional hierarchical linear models are used to analyze individual growth trajectories in social sciences. Conditional models can be used to study the effects of variables to predict individual differences in individual growth curves, but are sensitive to significant correlations between variables (16). Because of the limited sample size, the boundary between high and low performances was set at the 50th percentile. This limitation left the question unanswered regarding predictors of extremes and intermediate levels of performance. The dependency of predictors on our criterion for classifying performance must be considered when generalizing our results to other levels of academic achievement.
We conclude that knowledge growth on basic sciences occurs at a relatively modest rate during anesthesia residency and is significantly associated with the level of anxiety related to academic performance, the motivation for learning, and the ability in selecting main ideas. Testing residents for these factors and for their individual basic knowledge at the beginning of the training period could help identify residents with high probabilities of academic failure and direct remediation measures.
Footnotes
Accepted for publication February 14, 2007.
Address for correspondence and reprints to Getúlio R. de Oliveira Filho, PhD, Department of Anesthesiology, Nucleus for Research in Medical Education, Hospital Governador Celso Ramos, Florianópolis, SC, Brazil. Address e-mail to grof{at}grof.med.br.
REFERENCES
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
