This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via Web of Science (2) Citing Articles via Google Scholar Google Scholar Articles by Janssen, K. J. M. Articles by Vergouwe, Y. Search for Related Content PubMed PubMed Citation Articles by Janssen, K. J. M. Articles by Vergouwe, Y. Related Collections Economics and Health Care Research Pain Medicine Pain

---

ECONOMICS, EDUCATION, AND POLICY

The Risk of Severe Postoperative Pain: Modification and Validation of a Clinical Prediction Rule

Kristel J. M. Janssen^*, Cor J. Kalkman, Diederick E. Grobbee^*, Gouke J. Bonsel, Karel G. M. Moons^*, and Yvonne Vergouwe^*

From the *Julius Center for Health Sciences and Primary Care; Department of Anesthesiology, Division of Perioperative Care and Emergency Medicine, University Medical Center Utrecht, The Netherlands; and Department of Public Health, Academic Medical Center, Amsterdam, The Netherlands.

Address correspondence and reprint requests to Kristel J. M. Janssen, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, PO Box 85500, 3508 GA Utrecht, The Netherlands. Address e-mail to k.j.m.janssen{at}umcutrecht.nl.

Abstract

BACKGROUND: Recently, a prediction rule was developed to preoperatively predict the risk of severe pain in the first postoperative hour in surgical inpatients. We aimed to modify the rule to enhance its use in both surgical inpatients and outpatients (ambulatory patients). Subsequently, we prospectively tested the modified rule in patients who underwent surgery later in time and in another hospital (external validation).

METHODS: The rule was originally developed from the data of 1395 adult inpatients. We modified the rule with the data of 549 outpatients who underwent surgery between 1997 and 1999 in the same center (Academic Medical Center Amsterdam, The Netherlands). Furthermore, we tested the performance of the modified rule in 1035 in- and outpatients who underwent surgery in 2004, in the University Medical Center Utrecht, The Netherlands (external validation). Performance was quantified by the rule's calibration (agreement between observed frequencies and predicted risks) and discrimination (ability to distinguish between patients at high and low risk).

RESULTS: Modification of the original rule to enhance prediction in outpatients included reclassification of the predictor "type of surgery," addition of the predictor "surgical setting" (ambulatory surgery: yes/no) and addition of interaction terms between surgical setting and the other predictors. One-third of the patients in the Utrecht cohort reported severe postoperative pain (36%), compared to 62% of the patients in the Amsterdam cohort. The distribution of most predictors was similar in the two cohorts, although the patients in the Utrecht cohort were slightly older, more often underwent ambulatory surgery and had large expected incision sizes less often than patients in the Amsterdam cohort. The modified prediction rule showed good calibration, when an adjusted intercept was used for the lower incidence in the Utrecht cohort. The discrimination was reasonable (area under the Receiver Operating Characteristic curve 0.65 [95% confidence interval 0.57–0.73]).

CONCLUSIONS: A previously developed prediction rule to predict severe postoperative pain was modified to allow use in both inpatients and outpatients. By validating the rule in patients who underwent surgery several years later in another hospital, it was shown that the rule could be generalized in time and place. We demonstrated that, instead of deriving new prediction rules for new populations, a simple adjustment may be enough to recalibrate prediction rules for new populations. This is in line with the perception that external validation and updating of prediction rules is a continuing and multistage process.

Moderate to severe acute postoperative pain occurs frequently after a variety of surgical procedures. Incidences of up to 50% in inpatients and 40% in outpatients (patients undergoing ambulatory surgery) have been reported.1–4 Severe postoperative pain may result in patient discomfort, reduced patient satisfaction, delayed discharge from the postoperative anesthesia care unit (PACU) and hospital, and limited mobility and return to normal activities.5 Moreover, it can promote delirium in the elderly6 and may develop into chronic pain syndromes.7

Prediction rules for various postoperative outcomes such as mortality have been developed and are used for risk management. Surprisingly, there were no rules for preoperative estimation of the risk of acute or late postoperative pain. Such rules could preoperatively distinguish between patients at high risk and low risk to direct appropriate preventive pain treatment. Given the reported high incidences of severe acute postoperative pain, the approach to prevent pain in current practice seems insufficient for timely identification and treatment of patients at high risk.2,8

A multivariable prediction rule (including only predictors that are easy to obtain during a preoperative visit) was developed to preoperatively predict the risk of severe pain in the first postoperative hour in surgical inpatients.9 It would be useful if this rule could also predict severe acute postoperative pain in outpatients. We therefore modified the inpatients rule to be used in inpatients and outpatients. Subsequently, we prospectively tested this modified rule in patients who underwent surgery several years later in another hospital (external validation). We used state-of-the-art methods for modification and validation of clinical prediction rules that can be applied to prediction rules for any clinical problem. Hence, this paper may also serve as a methodological illustration of the modification and validation of clinical prediction rules in general, which is a continuing and multistage process.

METHODS

The original prediction rule was developed from the data of 1395 adult inpatients who underwent surgery between 1997 and 1999 in the Academic Medical Center Amsterdam, The Netherlands. We modified the rule with data of 549 outpatients who underwent surgery during the same period and in the same hospital. We subsequently assessed the predictive performance of the rule in 1035 new patients10,11 (external validation) who underwent surgery in 2004 in the University Medical Center Utrecht, The Netherlands.

Original Prediction Rule for Inpatients
The development of the original prediction rule has been described.9 In brief, patients were 18–85 yr and all types of surgery were included, except cardiac surgery and intracranial neurosurgical procedures. Exclusion criteria were emergency surgery, pregnancy, ASA physical status 4 and morbid obesity (weight >120 kg). Induction of anesthesia was achieved with thiopental in patients randomized to isoflurane/nitrous oxide, and with propofol in patients randomized to total IV anesthesia with propofol/air. Anesthesia was maintained with propofol or isoflurane in nitrous oxide according to the randomization. Intraoperatively, the anesthesiologist was free to use opioid analgesics (typically fentanyl or sufentanil in appropriate doses) and muscle relaxants as needed.12 The patients did not receive regional anesthesia or combined general/regional anesthesia, except for inpatients undergoing upper abdominal procedures (n = 30), in whom it was considered standard practice to combine general anesthesia with thoracic epidural analgesia.

The dichotomous outcome of the prediction rule was the presence or absence of severe acute postoperative pain. Presence was defined as a numerical rating scale (NRS) score equal to or higher than 8 (where 0 indicates no pain at all, and 10 the most severe pain imaginable), occurring at least once within the first hour at the PACU. A trained and blinded research nurse recorded the severity of pain every 15 min with a NRS. If patients were not awake they received a NRS score of 0 (no pain) for that time point.

The prediction rule was presented as a formula and as an easy-to-use nomogram (Appendix). The included predictors were gender, age, type of surgery (ophthalmology, laparoscopy, ear/nose/throat, orthopedic surgery, intra-abdominal, and other type of surgery), expected incision size 10 cm, a preoperative pain score, an anxiety score and a need for information score. The predictors anxiety and need for information score were based on the Amsterdam Preoperative Anxiety and Information Scale (APAIS) questionnaire, which consists of six questions, each scored on a 5-point Likert scale from 1 (not at all) to 5 (extremely). The APAIS is specifically designed to assess the patient's preoperative anxiety score (4 questions, score range 4–20) and an information-seeking behavior score to assess the patient's need for information regarding the scheduled surgery and anesthesia (2 questions, score range, 2–10).13

Modifying the Prediction Rule for Surgical Outpatients
To modify the rule, we used data of 549 surgical outpatients who underwent surgery between 1997 and 1999 in the Academic Medical Center Amsterdam (the inpatients and outpatients from this center are further referred to as the Amsterdam cohort). The same methods of data collection were applied as for the inpatients. Hence, the same predictors and outcome definitions could be used. As the prediction rule performed insufficiently in these outpatients, we modified the rule so it was applicable and valid to both inpatients and outpatients. To improve the performance, we made three adjustments to the rule.

First, we used a more widely accepted definition of severe acute postoperative pain for both in- and outpatients, i.e., NRS score 6 instead of 8, to better adhere to the indication for administering acute pain treatment in current practice.14

Second, the classification of type of surgery that was used as a predictor in the rule for inpatients was initially developed for prediction of postoperative nausea and vomiting rather than for acute postoperative pain.9 For the latter purpose no suitable classification could be found in the literature; therefore we developed this classification. We identified 27 groups of surgical procedures based on clinical experience, current practice and interviews with surgeons and anesthesiologists (Table 1). Subsequently, the univariable association between each surgical group and severe acute postoperative pain was estimated. Groups with similar associations were further combined.

View this table: [in this window] [in a new window]   Table 1. Surgical Procedures Conducted in Patients of the Amsterdam Cohort, Ordered by Increasing Incidence of Severe Acute Postoperative Pain (Defined as 6 on a Numerical Rating Scale)

Third, we included an additional predictor "surgical setting" (ambulatory surgery: yes versus no), as we expected that inpatients may have a higher risk of postoperative pain than outpatients.

Subsequently, the regression coefficients of the seven predictors of the original rule (Appendix) plus surgical setting were estimated in a multivariable logistic regression model15 in the combined data of the inpatients and the outpatients. We hypothesized a priori that the predictive effect of gender and type of surgery could be different for surgical inpatients and outpatients. Possible differences in the effect of the predictors between inpatients and outpatients were tested with interaction terms. To prevent problems of multiple testing, we used one overall test that considered all interaction terms together, with P < 0.30, to conform to current statistical guidelines.16

The incidence of severe postoperative pain might be different in other patients, as incidences change over time due to changes in treatment protocols. By adjusting the intercept of the prediction rule, this difference in incidence can be accounted for. To anticipate potential different incidences, we presented the modified rule with alternative intercept values so that the rule could be applied in other populations with different incidences of severe pain.

Prediction rules usually show overly optimistic performance in the patients from which they are developed.10,11,17,18 We therefore estimated the amount of optimism with bootstrapping techniques.18–20 Further, regression coefficients are usually too extreme. As a consequence, low predicted risks are too low in new patients and high predicted risks are too high. We therefore shrunk the logistic regression coefficients of the modified rule with a shrinkage factor that was smaller than one. The shrinkage factor was also derived with bootstrapping.

External Validation
Predictive performance of prediction rules needs to be tested in new patients before the rules can be applied in daily clinical practice.10,11,17,18 To test whether the rule was generalizable across time and place, we studied the predictive performance of the rule in 1035 new consecutive patients from a prospective cohort that underwent surgery between February and December 2004 in a different academic hospital (University Medical Center Utrecht, The Netherlands; the cohort is further referred to as the Utrecht cohort). The same predictors, outcome definitions and measurements were used as in the Amsterdam cohort.

Predictive Performance Measures
To study the predictive performance of the modified prediction rule in the Utrecht cohort, we assessed the calibration and discrimination of the rule. Calibration refers to the agreement between the predicted risks and observed incidences of severe acute postoperative pain in the new patients. This was graphically assessed with a calibration plot18 and tested with the Hosmer-Lemeshow statistic, where an insignificant test indicates good model fit.15 Discrimination is the ability of the rule to distinguish between patients with severe pain and patients without severe pain, and was quantified with the area under the Receiver Operating Characteristic curve (ROC area). An ROC area ranges from 0.5 (no discrimination; same as flipping a coin) to 1.0 (perfect discrimination).18

RESULTS

The Modified Prediction rule for Surgical Inpatients and Outpatients
The new classification of type of surgery resulted in a predictor with five categories (Table 1); i.e., lowest expected incidence of pain (observed incidence of severe postoperative pain 33%), low expected incidence of pain (47%), moderate expected incidence of pain (61%), high expected incidence of pain (68%) and highest expected incidence of pain (84%). Twenty-one patients with an unknown or rare surgical procedure (i.e., less than five patients in the database, surgery of the penis, bone marrow, and trachea) were excluded from the analysis, since their effect on postoperative pain could not be reliably estimated. Some surgical groups still had small numbers (more than 5 but <20 patients, such as carotid endarterectomy and vaginal hysterectomy, Table 1). These groups were explicitly retained in the analysis to enhance generalizability of the final results, even though the uncertainty in the outcome incidence will be higher for these groups than in groups with, for example, more than 100 patients. The effects of the tested interaction terms were statistically significant and included in the prediction rule.

As we decreased the threshold for "severe" postoperative pain from a NRS score 8 to a NRS score 6, other variables than included in the original rule could have become important. Hence, we performed an extra analysis to quantify whether other variables, such as Body Mass Index (BMI), duration of surgery and type of anesthesia (IV versus inhaled), had an added predictive effect. All these other variables were far from significant and no important predictors were missed.

Of the surgical outpatients, 48% (265/549) reported a NRS score 6 within the first hour after arriving at the PACU, versus 67% (935/1395) of the inpatients (Table 2, second and third column). Outpatients were on average younger and less often had a large incision size (20 vs 44%). Outpatients underwent types of surgery with high or highest expected pain incidences less often than inpatients.

The model was presented as a formula (Table 3) and as an easy to use score chart (Fig. 1). Table 3 shows the intercept and the regression coefficients of the predictors in the modified prediction rule. The risk of severe postoperative pain was increased by a large expected incision size (larger than 10 cm), high preoperative pain and anxiety scores, and decreased by female gender, older age and higher need for information scores. The interaction terms indicated that the effects of gender and type of surgery were different in inpatients and outpatients. For example, the effect of the type of surgery on the risk of severe postoperative pain was smaller (indicated by the negative interaction term) for outpatients than for inpatients who were scheduled for the same type of surgery.

View larger version (9K): [in this window] [in a new window]   Figure 1. Score chart to predict the risk of severe acute postoperative pain for inpatients and outpatients. The scores are based on the regression coefficients of the prediction rule. For each patient, a sumscore can be calculated by counting the scores that correlate to the characteristics of the patient. The total sumscore can be linked to the individual risk using the box. For example, in an outpatient setting (score = –4), a female patient (score = 3) of age 43 (score = –2), has a preoperative pain score of 9 (score = 5) is scheduled for a high expected risk of pain procedure (score = 5) with a small expected incision size (score = 0), and has a preoperative anxiety score of 16 (score = 4) and a preoperative need for information score of 4 (score = –1). This patient has a total sumscore of 10. This total sumscore refers to a risk of severe postoperative pain of 83% (using the lower part of the figure). When we use the formula of the prediction rule to estimate the risk of severe postoperative pain for this patient (Table 3), we find a risk of 85%. For patients in settings with a different incidence (prior probability) of severe postoperative pain, the corresponding constant should be used.

We estimated intercept values for different incidences of severe postoperative pain (Table 3), so that when the rule is applied in a population with a different incidence, the intercept of the rule can be adjusted.

The calibration of the modified rule was adequate, also expressed by the nonsignificant Hosmer-Lemeshow statistic (P: 0.09). The ROC area of the rule was 0.71 (95% confidence interval: 0.66, 0.76).

External Validation of the Modified Rule in the Utrecht Cohort
One-third of the patients in the Utrecht cohort reported severe postoperative pain (36%), compared to 62% of the patients in the Amsterdam cohort (Table 2). The distribution of most predictors was similar in the two cohorts, although the patients in the Utrecht cohort were slightly older and underwent ambulatory surgery more often, reflecting the current trend towards more outpatient surgery. They also had large expected incision sizes less often than patients in the Amsterdam cohort. We tested the modified prediction rule with an adjusted intercept for an incidence of 35% (intercept of –1.53 instead of –0.42, Table 3, Fig. 1).

Figure 2 shows the calibration line of the modified prediction rule in the Utrecht cohort. The dotted line shows the ideal situation in which the predicted risks and the observed frequencies of postoperative pain are completely in agreement. The solid line shows the observed association between the predicted risks and the observed frequencies. The prediction rule showed good calibration when the rule was tested in the patients of the Utrecht cohort (Fig. 2a). Note that when the predicted risks were higher than 60%, the predicted risks were slightly too high. To illustrate the necessity to adjust the intercept according to the lower incidence of severe postoperative pain, we also present the calibration line when we would have used the original intercept of –0.42 (Fig. 2b). In that case, the predicted risks would be systematically higher than the observed frequencies.

View larger version (9K): [in this window] [in a new window]   Figure 2. Calibration line of the modified prediction rule in the Utrecht cohort with adjusted intercept corresponding to the different incidence (prior probability) of severe postoperative pain (35% vs 62% in the Amsterdam cohort) (a) and with the original intercept that corresponds to the incidence in the Amsterdam cohort (b). Triangles indicate the observed frequency of severe acute postoperative pain per decile of predicted risk. The solid line shows the relation between observed outcomes and predicted risks. Ideally, this line equals the dotted line that represents perfect calibration, in which the predicted risks equal the observed frequencies of severe postoperative pain.

Table 4 shows the observed number of patients with and without severe postoperative pain across score and risk categories estimated by the modified rule in the Utrecht cohort. The incidence of severe postoperative pain per risk stratum increased from 18% to 65%. For example, for patients with a score of –10, the observed incidence of severe postoperative pain was 21%, whereas for patients with a score of 1, the observed incidence was 65%. Again, when the predicted risks were higher than 60%, the predicted risks were slightly too high (Fig. 2a).

The ROC area of the modified prediction rule in the Utrecht cohort was 0.65 (0.57–0.73).

DISCUSSION

We modified a previously developed rule to preoperatively predict the risk of severe acute postoperative pain in surgical inpatients in order to make it applicable to both in- and outpatients. We used a less stringent definition of severe acute postoperative pain, i.e., NRS 6, to adhere to current quality indicators and indications for administering analgesics. We revised the surgical classification and added surgical setting to the model. We validated the modified rule in inpatients and outpatients in a cohort of patients who were treated later in time and in another hospital. Since the incidence of severe postoperative pain was lower (36% vs 62%), we used an intercept that corresponds to an incidence of 35%. The modified rule showed good calibration and reasonable discrimination. To allow reliable identification of patients who might benefit from more aggressive preemptive analgesic strategies, ideally the discrimination of the rule should be higher. If future studies are able to identify strong additional predictors, these may be added to the model to increase the rule's discriminative ability.

Researchers are often tempted to develop a new prediction rule when a new patient sample shows slightly different results. If every new patient sample would lead to a new prediction rule, the information that is captured in a previous prediction rule is neglected.39,40 This is counterintuitive to the notion that research should be based on as much data as possible. The principle of using knowledge of previous studies has been recognized in etiologic and intervention research, in which cumulative meta-analyses are more common. We demonstrated that the original developed prediction rule could be modified and simply adjusted for new groups of patients rather than developing a new rule. This is in line with the perception that validation and updating of prediction rules is a continuing and multistage process.

The data to modify the prediction rule were obtained from patients who underwent surgery between 1997 and 1999. Since then, several surgical protocols have changed. For example, types of surgery that used to be performed on inpatients only may now be applied in ambulatory surgery as well. Also, the incidence of severe postoperative pain may have decreased over time as result of increased attention to protocols for postoperative pain management. Therefore, we specifically tested the modified rule in patients from a later period in another hospital that indeed had a lower incidence of severe postoperative pain. The modified rule showed good predictive performance. Interest in the field is shifting to include consideration of late (up to 24 h) postoperative pain, but preventing the occurrence of NRS scores > = 6 or 7, has become a quality of care indicator in various countries, including the Netherlands. Our continuing studies also focus on pain in the first 24 h. Nonetheless, severe postoperative pain in the PACU remains a highly relevant outcome for patients.

Type of surgery was an important predictor in the original prediction rule for inpatients. However, the classification of surgical procedures was developed for the prediction of postoperative nausea and vomiting. It was most likely not sensitive enough for the prediction of postoperative pain. Previous classifications of surgical procedures on prediction of postoperative pain were not suitable for our study, as they did not cover all procedures nor reflect the current trend towards more outpatient procedures.21 Therefore, we developed a new classification of surgical procedures based on the proportion of patients with severe pain in the first hour. However, additional validation of this surgical classification is required, especially since some surgical groups were under-represented in our sample.

For some of the predictors, gender and type of surgery, the effect was different for inpatients and outpatients, as reflected by a significant interaction term (Table 3). Gender shows no predictive effect in inpatients, whereas in outpatients the risk is twice as high for female patients. Also, the effect of all types of surgery on the risk of severe postoperative pain was smaller (indicated by the negative interaction term) for outpatients than for inpatients who were scheduled for the same type of surgery. Hence, an outpatient scheduled for the same surgery has a lower risk than an inpatient (with the same characteristics). A simple explanation cannot be given. Possibly, in- and outpatients scheduled for the same surgery differ in other characteristics not covered by the predictors in our rule. For example, it is conceivable that severity of the disorder requiring surgery may be different between in- and outpatients, leading to different risks of severe postoperative pain.

Testing the hypothesis of different regression coefficients for in- and outpatients was done by adding interaction terms between the variable "surgical setting" and gender and type of surgery. Since the interaction terms were statistically significant, the differences in the effects were accounted for in the final model (Table 3). These differences were confirmed when the entire analysis was repeated for inpatients and outpatients separately. An important reason to develop a single model for both inpatients and outpatients by including the interaction effects as presented, instead of two separate models, is that the estimated regression coefficients will be based on more data making the prediction model more stable enhancing its generalizability. Moreover, we believe that one "parsimonious" prediction rule for both in- and outpatients simplifies the application of the rule in practice.

The effects of most predictors included in the modified rule have been described, such as the risk-increasing effect of high preoperative pain and anxiety scores,22–25 and the risk-decreasing effect of age.21,22,26 The effect of gender remains the subject of debate. We confirmed the risk increasing effect of female gender on postoperative pain only in outpatients.14 Although there are studies in which female gender increased the risk of severe postoperative pain in inpatients,22,27 most studies (including ours) found no or only a small effect.26,28,29 Type of surgery has always been known to be an important predictor of postoperative pain.22,30–33 We found that expected incision size is an independent predictor in addition to type of surgery. It may seem peculiar to include a predictor of which the value can only be observed pre- or postoperatively in a rule that is to be applied preoperatively. However, in nonemergency surgery, expected incision size can be reliably estimated beforehand, given the large number of detailed surgical protocols in current practice. We are not aware of other studies that examined the predictive effect of expected incision size on postoperative pain.

A few other potential predictors of severe postoperative pain that were not included in our rule have been described in the literature, notably BMI26,30 and duration of surgery,24,26,28,30,31 although with conflicting results. In the original study among surgical inpatients, duration of surgery and BMI had no independent predictive value. Also, in our current analysis among in- and outpatients combined, these factors, including type of anesthesia, showed no additional predictive effect: the predictive performance of the rule was not increased by addition of these three factors.

Our study has several limitations. We used only predictors that can be easily obtained at the preoperative clinic. It is conceivable that genotype or specific testing for individual pain thresholds may yield important additional predictive information. Indeed, some pain threshold tests, such as cold and thermal stimuli, suprathreshold pain stimuli and burn tests, have been shown to predict the occurrence of acute postoperative pain.26,34–38 However, such tests require specific equipment, are time-consuming and may be burdensome for the patient. Thus far, they have not found widespread application in preoperative care, and their added predictive value beyond the more easily obtainable predictors included in the present rule remains to be quantified.

At present it is unknown whether clinical application of our prediction rule for severe postoperative pain will improve the quality of postoperative pain management. The most logical application would be to identify patients who are at high risk, and in these patients apply a more aggressive approach to prevention and treatment of postoperative pain. This could include multimodal pain therapy, including regional/general anesthesia combination techniques, and/or patient-controlled delivery of analgesics. For proper use of the modified prediction rule in clinical practice, a specific risk threshold for the application of preemptive or more extensive pain treatment needs to be chosen. Practitioners are always free to choose any threshold desired, although it is obvious that when a high risk threshold is chosen, fewer patients will receive treatment (with attendant reduction in side effects and costs). In contrast, patients who might benefit from such treatment will not be treated (which may in turn create additional costs). Conversely, when a low risk threshold is chosen, more patients will receive extensive pain treatment, which will likely reduce the incidence of severe acute postoperative pain, albeit at the expense of over-treatment (potentially leading to more side effects and higher costs). A definition of the most cost-effective threshold and assessing which more extensive preemptive pain approaches would be best for patients of different risk categories were beyond the scope of our study.

Although many prediction rules are available for postoperative complications, surprisingly few exist to predict postoperative pain. One was developed on patients undergoing orthopedic and intraperitoneal surgery.24 This rule included anesthetic technique, expectation of postoperative pain and chronic sleeping difficulties. However, only chronic sleeping difficulties showed a predictive effect in the validation set. Moreover, the rule was developed on a small dataset (304 patients, of which 153 experienced severe postoperative pain) using far too many predictors (62). In general, at least 10 events are needed for each predictor considered.18 This means that at least 620 patients with severe postoperative pain would have been needed instead of 153. Therefore, this rule is likely "overfitted" and the predictive value when applied in new patients may be suboptimal.

In conclusion, we modified a previously developed rule to predict severe postoperative pain for use in both inpatients and outpatients. External validation in patients who were treated more recently and in another center showed that the rule can be generalized in time and place. The rule can be applied to other patient populations with different incidences of severe postoperative pain, by using an alternative model intercept as we presented. We showed that this simple adjustment to the rule was sufficient for recalibration, instead of developing a new prediction rule. When future studies are able to identify strong additional predictors, these may be added to the model to increase the rule's discriminative ability. If the prediction rule proves to be robust in various other settings, its application might improve the quality of postoperative pain management by timely identification of patients who will benefit from more extensive analgesic regimens.

APPENDIX

Figure 1

View larger version (16K): [in this window] [in a new window]

Footnotes

Accepted for publication May 30, 2008.

Supported by The Netherlands Organization for Scientific Research Grant ZON-MW 917.46.360.

K.J.M.J. had the idea for the article, performed the analyses, interpreted the results and wrote the article. K.G.M.M. and Y.V. had the idea for the article and interpreted the results. C.J.K., K.G.M.M., D.E.G., and G.J.B. developed the original prediction rule and interpreted the results. All authors commented on drafts of the article and approved the final manuscript.

REFERENCES

1. Chauvin M. State of the art of pain treatment following ambulatory surgery. Eur J Anaesthesiol 2003;20:3–6[Web of Science]
2. Huang N, Cunningham F, Laurito CE, Chen C. Can we do better with postoperative pain management? Am J Surg 2001;182:440–48[Web of Science][Medline]
3. Shaikh S, Chung F, Imarengiaye C, Yung D, Bernstein M. Pain, nausea, vomiting and ocular complications delay discharge following ambulatory microdiscectomy. Can J Anaesth 2003;50:514–18[Web of Science][Medline]
4. Svensson I, Sjostrom B, Haljamae H. Assessment of pain experiences after elective surgery. J Pain Symptom Manage 2000; 20:193–201[Web of Science][Medline]
5. Myles PS, Williams DL, Hendrata M, Anderson H, Weeks AM. Patient satisfaction after anaesthesia and surgery: results of a prospective survey of 10,811 patients. Br J Anaesth 2000;84:6–10[Abstract/Free Full Text]
6. Lynch EP, Lazor MA, Gellis JE, Orav J, Goldman L, Marcantonio ER. The impact of postoperative pain on the development of postoperative delirium. Anesth Analg 1998;86:781–5[Abstract]
7. Katz J, Jackson M, Kavanagh BP, Sandler AN. Acute pain after thoracic surgery predicts long-term post-thoracotomy pain. Clin J Pain 1996;12:50–5[Web of Science][Medline]
8. Apfelbaum JL, Chen C, Mehta SS, Gan TJ. Postoperative pain experience: results from a national survey suggest postoperative pain continues to be undermanaged. Anesth Analg 2003;97:534–40[Abstract/Free Full Text]
9. Kalkman CJ, Visser K, Moen J, Bonsel GJ, Grobbee DE, Moons KG. Preoperative prediction of severe postoperative pain. Pain 2003;105:415–23[Web of Science][Medline]
10. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med 1999;130: 515–24[Abstract/Free Full Text]
11. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med 2006;144:201–9[Abstract/Free Full Text]
12. Visser K, Hassink EA, Bonsel GJ, Moen J, Kalkman CJ. Randomized controlled trial of total intravenous anesthesia with propofol versus inhalation anesthesia with isoflurane-nitrous oxide: postoperative nausea with vomiting and economic analysis. Anesthesiology 2001;95:616–26[Web of Science][Medline]
13. Moerman N, van Dam FS, Muller MJ, Oosting H. The Amsterdam Preoperative Anxiety and Information Scale (APAIS). Anesth Analg 1996;82:445–51[Abstract]
14. Rosseland LA, Stubhaug A. Gender is a confounding factor in pain trials: women report more pain than men after arthroscopic surgery. Pain 2004;112:248–53[Web of Science][Medline]
15. Hosmer DW, Lemeshow S. Applied logistic regression. New York: John Wiley and Sons, Inc., 1989
16. Harrell FE Jr. Regression modeling strategies. Springer-Verlag, New York, 2001
17. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med 2000;19:453–73[Web of Science][Medline]
18. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361–87[Web of Science][Medline]
19. Efron B. Censored Data and the Bootstrap. J Am Stat Assoc 1981;76:312–19[Web of Science]
20. Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774–81[Web of Science][Medline]
21. Macintyre PE, Jarvis DA. Age is the best predictor of postoperative morphine requirements. Pain 1996;64:357–64[Web of Science][Medline]
22. Thomas T, Robinson C, Champion D, McKell M, Pell M. Prediction and assessment of the severity of postoperative pain and of satisfaction with management. Pain 1998;75:177–185[Web of Science][Medline]
23. Chapman CR. Psychological aspects of pain patient treatment. Arch Surg 1977;112:767–72[Abstract/Free Full Text]
24. Mamie C, Bernstein M, Morabia A, Klopfenstein CE, Sloutskis D, Forster A. Are there reliable predictors of postoperative pain? Acta Anaesthesiol Scand 2004;48:234–42[Web of Science][Medline]
25. Scott LE, Clum GA, Peoples JB. Preoperative predictors of postoperative pain. Pain 1983;15:283–93[Web of Science][Medline]
26. Bisgaard T, Klarskov B, Rosenberg J, Kehlet H. Characteristics and prediction of early pain after laparoscopic cholecystectomy. Pain 2001;90:261–9[Web of Science][Medline]
27. Aubrun F, Salvi N, Coriat P, Riou B. Sex- and age-related differences in morphine requirements for postoperative pain relief. Anesthesiology 2005;103:156–60[Web of Science][Medline]
28. Kotzer AM. Factors predicting postoperative pain in children and adolescents following spine fusion. Issues Compr Pediatr Nurs 2000;23:83–102[Medline]
29. Logan DE, Rose JB. Gender differences in post-operative pain and patient controlled analgesia use among adolescent surgical patients. Pain 2004;109:481–7[Web of Science][Medline]
30. Chung F, Ritchie E, Su J. Postoperative pain in ambulatory surgery. Anesth Analg 1997;85:808–16[Abstract]
31. Dahmani S, Dupont H, Mantz J, Desmonts JM, Keita H. Predictive factors of early morphine requirements in the post-anaesthesia care unit (PACU). Br J Anaesth 2001;87:385–9[Abstract/Free Full Text]
32. Ellstrom M, Olsen MF, Olsson JH, Nordberg G, Bengtsson A, Hahlin M. Pain and pulmonary function following laparoscopic and abdominal hysterectomy: a randomized study. Acta Obstet Gynecol Scand 1998;77:923–8[Web of Science][Medline]
33. Nguyen T, Malley R, Inkelis SH, and Kuppermann N. Comparison of prediction models for adverse outcome in pediatric meningococcal disease using artificial neural network and logistic regression analyses. J Clin Epidemiol 2002;55:687–95[Web of Science][Medline]
34. Granot M, Lowenstein L, Yarnitsky D, Tamir A, Zimmer EZ. Postcesarean section pain prediction by preoperative experimental pain assessment. Anesthesiology 2003;98:1422–6[Web of Science][Medline]
35. Hsu YW, Somma J, Hung YC, Tsai PS, Yang CH, Chen CC. Predicting postoperative pain by preoperative pressure pain assessment. Anesthesiology 2005;103:613–18[Web of Science][Medline]
36. Kehlet H, Rung GW, Callesen T. Postoperative opioid analgesia: time for a reconsideration? J Clin Anesth 1996;8:441–5[Web of Science][Medline]
37. Kim H, Neubert JK, Rowan JS, Brahim JS, Iadarola MJ, Dionne RA. Comparison of experimental and acute clinical pain responses in humans as pain phenotypes. J Pain 2004;5:377–84[Web of Science][Medline]
38. Werner MU, Duun P, Kehlet H. Prediction of postoperative pain by preoperative nociceptive responses to heat stimulation. Anesthesiology 2004;100:115–19[Web of Science][Medline]
39. Janssen KJ, Moons KG, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 2008;61:76–86[Web of Science][Medline]
40. Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 2004;23:2567–86[Web of Science][Medline]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via Web of Science (2) Citing Articles via Google Scholar Google Scholar Articles by Janssen, K. J. M. Articles by Vergouwe, Y. Search for Related Content PubMed PubMed Citation Articles by Janssen, K. J. M. Articles by Vergouwe, Y. Related Collections Economics and Health Care Research Pain Medicine Pain