This Article Abstract Full Text (PDF) Data Supplement Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Chan, M. T. V. Articles by Gin, T. Search for Related Content PubMed PubMed Citation Articles by Chan, M. T. V. Articles by Gin, T. Related Collections Ambulatory Postanesthetic Care Unit Outcomes

---

AMBULATORY ANESTHESIOLOGY

Psychometric Testing of the Chinese Quality of Recovery Score

Matthew T. V. Chan, MBBS, FANZCA^*, Charlotte C. K. Lo, MBChB, BMedSc, FANZCA^*, Candy K. W. Lok, MPH, MSc^*, Tak Wing Chan, DPhil(Oxon), Kai C. Choi, PhD, and Tony Gin, MD, FRCA, FANZCA^*

From the *Department of Anaesthesia and Intensive Care, and Centre for Epidemiology and Biostatistics, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong; and Department of Scoiology, The University of Oxford, Oxford, UK.

Address correspondence to Matthew Chan, MBBS, FANZCA, Department of Anaesthesia and Intensive Care, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, New Territories, Hong Kong. Address e-mail to mtvchan{at}cuhk.edu.hk.

Abstract

BACKGROUND: We evaluated a Chinese translation of the quality of recovery (QoR) score for measuring health status in patients after surgery and anesthesia.

METHODS: The Chinese QoR score was developed by a panel of linguistic experts using a series of forward and backward translations. We then compared the psychometric performance of the Chinese QoR score with the original English version in bilingual Chinese patients undergoing a variety of surgeries. Using a crossover design, 210 patients were randomly assigned to complete both versions of the QoR score, 1 h apart, in the morning after surgery according to one of the two sequences: Chinese followed by the English QoR score or the reverse order. Test-retest and interrater reliabilities were calculated by comparing scores administered 6–8 h later and those completed by the patients and duty nurses, respectively.

RESULTS: Patient responses to the Chinese QoR score agreed well with the English version (weighted kappa statistic, _w = 0.92). Both QoR scores correlated with patient satisfaction scores (Chinese version, = 0.82; English version, = 0.79) confirming convergent validity. There was also significant negative correlation between the QoR scores and female gender, duration of hospital stay, or magnitude of surgery (discriminant construct validity). We found the Chinese QoR score has good internal consistency (Cronbach’s = 0.91), interobserver (_w = 0.77), and test-retest reliability (_w = 0.83). These values were similar to those of the original English version (Cronbach’s = 0.89, interobserver _w = 0.84, test-retest reliability _w = 0.88).

CONCLUSION: The Chinese QoR score is conceptually, semantically, and operationally equivalent to the English version. Both scales provide valid, reliable, and responsive assessment of the QoR after surgery and anesthesia.

Health-related quality of life (HRQoL) is an important attribute of patient care.1,2 It reflects a patient’s experience of the quality of health care received, and it indicates the impact of adverse events, such as pain or other organ dysfunction during the period of care. A number of instruments have been developed to measure the HRQoL after surgery and anesthesia.3–11 All these instruments were developed in countries with unique cultures and many were administered in English.3–10 Therefore, it is difficult to apply these instruments to patients in different cultures, speaking different languages. In this regard, a validated instrument available in several languages is important for multicenter studies involving different cultural groups.

The purpose of the present study was to develop the Chinese version of the quality of recovery (QoR) score.5 We chose the QoR score because it is a simple instrument containing nine questions to evaluate postoperative changes in emotion, well-being, social function, and physical disability. Each of the nine items is rated on a scale of 0–2, with a maximum score of 18. The QoR scale has proven psychometric properties,5 and more importantly, it is the most frequently cited instrument for measuring HRQoL after anesthesia and surgery.12–14 Using a crossover design, we compared the validity and reliability of the Chinese QoR score with the original English version in 210 bilingual Hong Kong Chinese patients undergoing a variety of surgical procedures.

METHODS

The translation and validation procedures were conducted according to the International Quality of Life Assessment project.15

The Translation Procedure
Three professional translators, who were native Chinese speakers, independently translated the original English version of the QoR score into written Chinese. The drafts were critically reviewed by a panel consisting of the translators and four bilingual authors (M.T.V.C., C.K.W.L., C.C.K.L., and T.W.C.), item by item, before a revised draft was agreed upon. It was then back-translated to English separately by another three linguistic experts, who had no prior knowledge about the QoR score. These were compared with the original English version. The Chinese QoR score was further modified until the final text was accepted by the panel.

Psychometric Testing
We studied the equivalence of the Chinese and English QoR scores using a crossover design. Surgical patients undergoing a diverse group of procedures with general anesthesia were eligible for the study if they were ethnically Chinese, aged between 18 and 70 yrs, and were able to read a newspaper in English and Chinese. We excluded patients with major psychosis or drug and alcohol abuse. Patients with significant visual impairment or other physical disability that precluded complete cooperation were also excluded.

In the morning after surgery, we asked patients to rate their QoR using both versions of the QoR scores sequentially. The questionnaires were self-administered. Patients were randomly assigned so that half of the patients completed the Chinese QoR score first (Period 1), and then 1 h later (Period 2) the English version. The remaining patients completed the questionnaires in the reverse order. The order of administration was stratified by the magnitude of surgery (minor versus major), using permuted blocks, according to a computer-generated random number concealed in an opaque envelope. We collected the first questionnaire before the second questionnaire was administered to prevent cross-referencing of answers. At the same time, the nurse primarily responsible for the care of the patient was also asked to complete the same set of questionnaires for the assessment of interrater agreement.

Immediately after we collected the second questionnaire, patients were asked to indicate their overall recovery using a 100-mm visual analog scale printed on a slide rule bar (Astra USA, Westborough, MA). The 0 point represented "the worst imaginable recovery" when the patient experienced severe pain, nausea, and vomiting, might also have been confused and was unable to move, eat, or communicate with others. The 100 point was recovery "as good as it can be." Thus, the patient was alert, active, and comfortable, was also free of pain, enjoying food, and was able to communicate freely with others.

Later the same day, 6–8 h afterwards, we asked the patients to complete the questionnaires for measuring test-retest reliability. To determine the responsiveness of the scores to the changes in health status, patients completed the English and Chinese QoR scores on Day 2, 3, 5, and 14 after surgery. Patients who were discharged home were interviewed by telephone.

We also collected perioperative data that may indicate the QoR. Specifically, we recorded the duration of stay in the postanesthetic care unit (PACU) from the end of surgery to the time eligible for discharge to the wards, defined as an Aldrete score 9.16 We also recorded the total hospital stay from the start of surgery to actual hospital discharge. The study was approved by the Clinical Research Ethics Committee and all patients gave written informed consent during the preoperative visits.

Statistical Analyses
We anticipated that the item scores and psychometric properties of the English and Chinese QoR scores are equivalent. Therefore, we modeled the sample size according to the smallest possible difference between scales (i.e., one-point difference or 5.6% of the total score). Given that the variation of the QoR score in the literature was 10%–15%,5,12–14 we calculated that at least 180 patients were required to demonstrate the difference with 90% power at 5% two-sided Type I error. We recruited 210 patients to allow for dropouts and missing data.

Scale Equivalence
Data were presented as frequency (%) or mean ± sd, as appropriate. Agreement was expressed as the raw concordance rate between scores. We also determined the extent of agreement using the weighted kappa statistic, _w.17,18 In general, a _w value of 0–0.2 indicates poor agreement, 0.21–0.4 is fair agreement, 0.41–0.60 is moderate agreement, 0.61–0.8 is substantial agreement, and 0.81–1 shows almost perfect agreement.17 To determine whether the order of scale administration affects a patient’s response to the questionnaires, we calculated the language (English versus Chinese), period (Period 1 versus Period 2), and carryover (treatment x period interaction) effects for a two-period crossover study, using the method proposed by Hills and Armitage.19 Multiple comparisons were adjusted by Bonferroni correction.

Reliability Testing
We compared the scores among items in each questionnaire using polychoric correlation.20 Internal consistency was then calculated with Cronbach’s statistic. Agreement between QoR scores obtained 6–8 h apart (test-retest reliability) and those completed by the patients or the duty nurses (interobserver reliability) were measured by weighted kappa statistic.

Validity Testing
We determined the convergent validity of the QoR scores by comparing the total scores with the patient satisfaction score using Spearman rank correlation. Similarly, we measured the association between QoR scores and the duration of stay in the recovery room or total hospital stay as alternate indices of QoR. We also evaluated the divergent validity of the QoR scores to distinguish patients with poorer QoR. Specifically, we compared the QoR scores in women with men, and in those patients undergoing major with minor surgery using the Mann-Whitney U-test.

Responsiveness to Change
Differences in postoperative QoR scores were analyzed by analysis of variance with repeated measures and were compared between patients after major or minor surgery. We also determined the sensitivity of the scores to detect a change in health status by calculating the standardized response mean.21,22 This is the ratio between the mean change in scores and the standard deviation of the change. All statistical tests were two-sided and a P value <0.05 was considered statistically significant.

RESULTS

Nineteen preliminary drafts were prepared before the final text of the Chinese QoR score was approved by the panel. The major difficulties and disagreements arose from the last three questions of the QoR score: "free from headaches, backache, or muscle pains," "free from nausea, dry-retching, or vomiting," and "free from experiencing severe pain or constant moderate pain." With direct Chinese translation, these phrases contained double negative messages. The questions were therefore rephrased in a positive affirmative fashion. The Likert scale was also reversed for these questions accordingly, so that 2 = "None of the time" and 0 = "Most of the time" (Supplementary Appendix).

All 210 patients completed the Chinese and English QoR scores in the assigned order. Two patients required further surgery in the afternoon of the first postoperative day and were unable to complete the second set of questionnaires. Therefore, the test-retest reliability was calculated from 208 patients. Table 1 shows the characteristics and surgical details of all patients. The average time required to complete the Chinese QoR scale, 2.3 ± 0.3 min, was similar to that for the English version, 2.2 ± 0.4 min, P = 0.92. The order of administration (period effect) did not affect the time to complete the questionnaire (P = 0.62). There was also no carryover effect (P = 0.27), such that the completion of the scale in Period 1 did not influence the time required in Period 2.

Overall, the recorded responses in the Chinese QoR score were similar to that of the original English version (Table 2). The average _w value was 0.92 indicating a near perfect agreement. Subgroup analysis in patients undergoing major (n = 105) and minor surgery (n = 105) showed no significant difference in the agreement (average _w 0.93 vs 0.89, respectively), suggesting that the agreement was uniform across the whole range of QoR score (detail results are available in the Web Supplement available at www.anesthesia-analgesia.org). There was no carryover or period effect, suggesting that score equivalence was not influenced by learning of the responses in the first questionnaire during Period 1. Table 3 shows the interitem correlation matrix. The Cronbach’s for the Chinese and English QoR scores were 0.91 and 0.89, respectively. Both QoR scores showed substantial to perfect agreement between tests performed 6–8 h apart (_w 0.67–0.94) and among observers, patients, or primary nurses (_w 0.74–0.93, Table 4).

View this table: [in this window] [in a new window]   Table 4. Test-Retest and Interobserver Agreement of the Chinese and English Quality of Recovery (QoR) Score. Values were Weighted Kappa, w, Using Fleiss-Cohen Weights17

The QoR scores correlated well with patient satisfaction scores (Chinese QoR score, = 0.82, P < 0.001; English QoR score, = 0.79, P < 0.001), confirming convergent validity. The correlation was largely unaffected by the types or magnitude of surgery (Chinese QoR score, = 0.53–0.82; English QoR score, = 0.64–0.79). Both the Chinese and English QoR scores were significantly lower in women (P = 0.01) compared with men. The mean differences were 3.3 (95% confidence intervals [CI]: 2.4–4.3) and 3.1 (2.1–4.0) for the Chinese and English QoR scores, respectively. Similarly, patients undergoing major surgery scored much lower in either scales compared with those after minor surgery (mean difference was 6.4 [95% CI: 5.8–7.0] for the Chinese QoR score and 6.7 [95% CI: 6.1–7.3] for the English QoR score). There was also a negative correlation between the QoR scores with the duration of hospital stay (Chinese QoR score, = –0.73, P < 0.001; English QoR score, = –0.71, P < 0.001) or length of time spent in the PACU (Chinese QoR score, = –0.39, P < 0.001; English QoR score, = –0.33, P = 0.01). These data suggested that both scores were comparable in distinguishing patients with poor recovery (discriminant validity).

Figure 1 shows the change in scores of both questionnaires after surgery. Patients undergoing major surgery reported a lower score compared with minor surgery (P < 0.001). The standardized response means for Chinese and English QoR scores were 0.78 and 0.76, respectively.

View larger version (14K): [in this window] [in a new window]   Figure 1. Changes in the Chinese () and English () quality of recovery (QoR) scales after major (solid line, n = 105) and minor (broken line, n = 105) surgery. Display for the English QoR scale is offset for clarity.

DISCUSSION

The QoR score is an established measure of HRQoL after anesthesia and surgery.5,12 Its utility, however, is restricted to English-speaking patients. To facilitate health status assessment in international multicenter study, an instrument available in different languages with equivalent psychometric properties is required. The QoR score has recently been translated into German. In 577 patients undergoing a variety of surgical procedures, Eberhart et al.23 reported a negative correlation between the German QoR score and magnitude of surgery or duration of anesthesia. The internal consistency of the German version was acceptable (Cronbach’s = 0.63). Although these results were comparable to the original English version,5 there were no data comparing the psychometric performance between versions. It is therefore unclear whether the German QoR score will produce an identical rating for the same health status as the original English version. We have designed the present study to compare the Chinese and English QoR scores directly.

In this study, we found a near perfect agreement inpatient responses between the Chinese and English QoR scores. Both versions have good test-retest (median _w 0.82) reliabilities. Internal consistency (Cronbach’s = 0.89–0.91) and construct validity were also similar between scales. The time required to complete the questionnaires did not differ between scales, indicating that they present similar challenges to patients in the postoperative period. Although there is no "gold standard" for assessing QoR, we were able to identify patients who were likely to have poorer health status after surgery and anesthesia (e.g., after major surgery with prolonged PACU or hospital stays) using either version of the QoR scores. There was a positive correlation between QoR scores and patient satisfaction. These data strongly argue for construct validity of both scales. We believe the Chinese QoR score is conceptually, semantically, and operationally equivalent to the English version. It is therefore appropriate to compare the Chinese QoR score with the English data from other populations and in different countries.22 Given that the scores can now be pooled and analyzed together, it is also feasible to include Chinese-speaking patients in international multicenter studies using the QoR score to evaluate the recovery after surgery and anesthesia.14 We also found a good agreement in rating QoR by different observers using the QoR scores (median _w = 0.88). For patients who have difficulty completing the questionnaires, our data suggest that it is valid to obtain QoR scores with experienced nurses. A similar approach was found useful in a more elaborated version of the questionnaire (QoR-40).24

Although there was an excellent agreement between the Chinese and English QoR scores, these results may be biased because we asked patients to complete the questionnaires sequentially within a short period of time. Therefore, learning in the first questionnaire may have confounded their responses in the second scale. We anticipated this problem and have randomized the questionnaire administration in this crossover study.17 This design allowed us to quantify the effect of "cross-reference" between scales. In this regard, we found no period effect in our study, suggesting that order of questionnaire administration did not influence patient response. There was also no carryover effect, so that completing the first questionnaire did not affect their responses in the second one. We believe patients treated the two versions of the QoR scores independently and that the scores are equivalent. An alternate approach to minimize recall of answers would be to ask the questions in random order during sequential administration of the QoR scores. Nevertheless, the study design required bilingual patients who could read both English and Chinese. This may also have introduced selection bias, because the participants were more likely to be well educated and to have superior language skills.

We found a substantial change in QoR scores after surgery. After major procedures, there was a >40% decrease in score on Day 1. This gradually returned to normal by the end of the week. The change after minor surgery was also more than 10%. The corresponding standardized response mean was about 0.7.21,22 This is a relatively large effect size, suggesting the QoR score is sensitive to change in health status after surgery. Accordingly, between 30 and 50 patients per group are required to detect a clinically relevant change in health status after a perioperative intervention.21

Although there are more than 10 commonly spoken dialects in China, there is only one written language among Chinese. We have therefore formulated the Chinese QoR score in standard written Chinese. We believe this version is applicable to all Chinese in other parts of the world. Nonetheless, the ideal practice would be to calibrate the QoR score locally to account for the subtle difference in the values and cultures in these regions.15

In conclusion, we have developed the Chinese QoR score to determine health status after anesthesia and surgery. The reliability, validity, responsiveness, interpretability, feasibility, and acceptability of the score are indistinguishable from the original English version. The availability of valid assessment tools in Chinese and other different languages presents an opportunity for conducting large cross-cultural clinical studies.

ACKNOWLEDGMENTS

We thank Professor Paul Myles for granting us permission to translate the QoR Scale in Chinese.

Footnotes

This article has supplementary material on the Web site: www.anesthesia-analgesia.org.

Accepted for publication May 7, 2008.

Supported, in part, by a competitive earmarked research grant CUHK4400/02M from the Research Grant Council, Hong Kong SAR.

Tony Gin is editor of Anesthetic Clinical Pharmacology for the Journal. This manuscript was handled by Peter S. A. Glass, Section Editor of Ambulatory Anesthesiology and Tony Gin was not involved in anyway with the editorial process or decision.

Reprints will not be available from the author.

REFERENCES

1. Guyatt GH, Cook DJ. Health status, quality of life, and the individual. JAMA 1994;272:630–1[Abstract/Free Full Text]
2. Gill TM, Feinstein AR. A critical appraisal of the quality of quality-of-life measurements. JAMA 1994;272:619–26[Abstract/Free Full Text]
3. Dexter F, Aker J, Wright WA. Development of a measure of patient satisfaction with monitored anesthesia care: the Iowa Satisfaction with Anesthesia Scale. Anesthesiology 1997;87: 865–73[Web of Science][Medline]
4. Swan BA, Maislin G, Traber K. Symptom distress and functional status changes during the first seven days after ambulatory surgery. Anesth Analg 1998;86:739–45[Abstract]
5. Myles PS, Hunt JO, Nightingale CE, Fletcher H, Beh T, Tanil D, Nagy A, Rubinstein A, Ponsford JL. Development and psychometric testing of a quality of life recovery score after general anesthesia and surgery in adults. Anesth Analg 1999;88:83–90[Abstract/Free Full Text]
6. Kleinbeck SV. Self-reported at-home postoperative recovery. Res Nurs Health 2000;23:461–72[Web of Science][Medline]
7. Myles PS, Weitkamp B, Jones K, Melick J, Hensen S. Validity and reliability of a postoperative quality of recovery score: the QoR-40. Br J Anaesth 2000;84:11–5[Abstract/Free Full Text]
8. Hogue SL, Reese PR, Colopy M, Fleisher LA, Tuman KJ, Twersky RS, Warner DS, Jamerson B. Assessing a tool to measure patient functional ability after outpatient surgery. Anesth Analg 2000;91:97–106[Abstract/Free Full Text]
9. Oakes CL, Ellington KJ, Oakes KJ, Olson RL, Neill KM, Vacchiano CA. Assessment of post-anesthesia short term quality of life: a pilot study. AANA J 2002;70:267–73[Medline]
10. Talamini MA, Stanfield CL, Chang DC, Wu AW. The surgical recovery index. Surg Endosc 2004;18:596–600[Web of Science][Medline]
11. Auquier P, Pernoud N, Bruder N, Simeoni MC, Auffray JP, Colavolpe C, Francois G, Gouin F, Manelli JC, Martin C, Sapin C, Blache JL. Development and validation of a perioperative satisfaction questionnaire. Anesthesiology 2005;102:1116–23[Web of Science][Medline]
12. Herrera FJ, Wong J, Chung F. A systematic review of postoperative recovery outcomes measurements after ambulatory surgery. Anesth Analg 2007;105:63–9[Abstract/Free Full Text]
13. Kwok RF, Lim J, Chan MTV, Gin T, Chiu WK. Preoperative ketamine improves postoperative analgesia after gynecologic laparoscopic surgery. Anesth Analg 2004;98:1044–9[Abstract/Free Full Text]
14. Myles PS, Leslie K, Chan MTV, Forbes A, Paech MJ, Peyton P, Silbert BS, Pascoe E; ENIGMA Trial Group. Avoidance of nitrous oxide for patients undergoing major surgery: a randomized controlled trial. Anesthesiology 2007;107:221–31[Web of Science][Medline]
15. Ware JE, Keller SD, Gandek B, Brazier JE, Sullivan M; IQOLA Project Group. Evaluating translations of health status questionnaires: methods from the IQOLA project. Int J Tech Assess Health Care 1995;11:525–51
16. Aldrete JA. The post-anesthesia recovery-score revisited. J Clin Anesth 1995;7:89–91[Web of Science][Medline]
17. Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 1973;33:613–9[Web of Science]
18. Landis RJ, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74[Web of Science][Medline]
19. Hills M, Armitage P. The two-period cross-over clinical trial. Br J Clin Pharmacol 1979;8:7–20[Web of Science][Medline]
20. Olsson U. Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika 1979;44:443–60[Web of Science]
21. Katz JN, Larson MG, Phillips CB, Fossel AH, Liang MH. Comparative measurement sensitivity of short and longer health status instruments. Med Care 1992;30:917–25[Web of Science][Medline]
22. Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chron Dis 1987;40:171–8[Web of Science][Medline]
23. Eberhart LHJ, Greiner S, Geldner G, Wulf H. Patient evaluation of postoperative recovery. An evaluation of the QoR scores in 577 patients. Anaesthesist 2002;51:463–6[Web of Science][Medline]
24. Gower ST, Quig CA, Hunt JO, Wallace SK, Myles PS. A comparison of patient self-administered and investigator administered measurement of quality of recovery using the QoR-40. Anaesth Intensive Care 2006;34:634–8[Web of Science][Medline]

This Article Abstract Full Text (PDF) Data Supplement Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Chan, M. T. V. Articles by Gin, T. Search for Related Content PubMed PubMed Citation Articles by Chan, M. T. V. Articles by Gin, T. Related Collections Ambulatory Postanesthetic Care Unit Outcomes