Rheumatology Advance Access originally published online on May 23, 2006
Rheumatology 2007 46(1):87-92; doi:10.1093/rheumatology/kel040

? The Author 2006. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.org
The online version of this article has been published under an open access model. Users are entitled to use, reproduce, disseminate, or display the open access version of this article for non-commercial purposes provided that: the original authorship is properly and fully attributed; the Journal and Oxford University Press are attributed as the original place of publication with the correct citation details given; if an article is subsequently reproduced or disseminated not in its entirety but only in part or as a derivative work this must be clearly indicated. For commercial re-use, please contact journals.permissions@oxfordjournals.org

Cross-cultural adaptation, reliability and validity of the German Shoulder Pain and Disability Index (SPADI)

F. Angst, J. Goldhahn1, G. Pap, A. F. Mannion1, K. E. Roach2, D. Siebertz, S. Drerup1, H. K. Schwyzer and B. R. Simmen

Department of Upper Extremity and Hand Surgery, 1Department of Musculoskeletal Research, Schulthess Klinik, Zurich, Switzerland and Department of Rheumatology/Institute of Physical Medicine, University Hospital, Zurich, Switzerland and 2Department of Physical Therapy, Miller School of Medicine, University of Miami, Florida, USA.

Correspondence to: F. Angst, Schiedhaldenstrasse 1, 8700 K?nacht, Switzerland. E-mail: fangst@datacomm.ch


   Abstract
 
Objective. To cross-culturally adapt the Shoulder Pain and Disability Index (SPADI) from English into German, and to test the reliability and validity of the German version.

Methods. Cross-cultural adaptation of the SPADI was performed according to international guidelines. One hundred and eighteen patients who had undergone shoulder arthroplasty, on average 4 yr previously, completed a questionnaire booklet containing the German SPADI, the Short Form 36 (SF-36), the Disability of the Arm, Shoulder and Hand (DASH) questionnaire, and the American Shoulder and Elbow Surgeons (ASES) questionnaire for the shoulder to assess SPADI's construct validity. One week later, they completed the SPADI again to assess test?retest reliability.

Results. The six-step cross-cultural adaptation procedure revealed no major problems with the content or language. The intraclass correlation coefficients for the individual items of the SPADI were between 0.68 and 0.89, and that for the SPADI total score was 0.94. The SPADI total score showed a correlation of 0.61?0.69 with the SF-36 physical scales, of 0.88 with the DASH and of 0.92 with the ASES.

Conclusions. The German SPADI is a practicable, reliable and valid instrument, and can be recommended for the self-assessment of shoulder pain and function.

KEY WORDS: Shoulder, Questionnaire, Reliability, Validity, SPADI
   Introduction
 
The shoulder is one of the most common sites of musculoskeletal pain, second only to back pain, with 1-month to 1-yr period prevalences in the general population of 6?37%. Shoulder problems are hence associated with considerable personal and economic costs in relation to the individual sufferer and the health-care system alike [1]. It is becoming apparent that the patient's subjective perception of their disease status is decisive for both the diagnostic work-up and the subsequent therapeutic management [2, 3]. Consequently, compared with the evaluation of clinical and biomedical factors, self-assessment is gaining increasing importance in determining the functional and psychosocial (participation) consequences of disease (World Health Organization International Classification of Functioning, Disability, and Health: ICF levels 2 and 3) [4].

The shoulder consists of five functional joints, which form a functional entity to perform complex movements in almost every activity of daily living [1]. A joint-specific assessment of shoulder disorders should cover a range of functions and movements of this functional entity that is as wide as possible but exclude functions of the elbow and wrist as far as possible. The WHO's ICF demands the use of well-tested instruments in the assessment of health and disease. The Disability of the Arm, Shoulder and Hand questionnaire (DASH) [5], the American Shoulder and Elbow Surgeons (ASES) questionnaire for the shoulder [6], and the Shoulder Pain and Disability Index (SPADI) [7] are the most thoroughly tested and most commonly used self-assessment instruments for the shoulder [8, 9]. The DASH was originally developed for the assessment of all three joints of the upper extremity and hence is not shoulder-specific. The ASES (patient self-assessment part), a questionnaire that resulted from a Delphi process carried out amongst shoulder surgeons, is longer than the SPADI (18 items vs 13 items) and has not been tested extensively. A German version of the DASH already exists [10] and that of the ASES will soon be published [J. Goldhahn, F. Angst, G. Pap, A. F. Mannion, S. Drerup, H. K. Schwyzer and B. R. Simmen; [11]. Previous studies have shown that it is feasible to implement the English original version of the SPADI in clinical practice and that it is a valid and highly responsive instrument for assessing shoulder pain and function [11?13]. The aim of this study was to cross-culturally adapt the SPADI for use with German-speaking patients and to determine its clinimetric properties, in particular, the reliability and validity.


   Materials and methods
 
Translation and cross-cultural adaptation
The SPADI comprises a series of 13 items (five for pain, eight for function), each scored with a visual analogue scale (VAS) from 0 (no pain/no difficulty, i.e. best) to 11 (worst pain imaginable/so difficult required help, i.e. worst) [7]. We followed the description of the original SPADI and adopted this likely unusual scaling ?A numeric score was calculated for each item by arbitrarly dividing the horizontal line into 12 segments of equal length. A number ranging from 0 to 11 was attached to this segment to produce a score for each item? [7]. For the score transformation, there is no problem whether the item maximum is scored 10 or 11: the divisor is 11 instead of 10.

Cross-cultural adaptation of the SPADI was performed following the guidelines of the American Association of Orthopedic Surgeons (AAOS) Outcomes Committee [14]. This process consists of six steps, each of which is documented with a written report: stage I, forward translation (English to German) by an informed translator (i.e. a health professional, T1) and uninformed translator (T2); stage II, synthesis of T1 and T2, resolving any discrepancies, leading to version T-12; stage III, back-translation (German to English) of the version T-12 by two native English-speaking back-translators (BT1 and BT2), naive to the purpose of the instrument; stage IV, expert committee review consisting of all four translators, one methodologist and one language professional to reach consensus on discrepancies or ambiguities, and to establish a pre-final version (German); stage V: pretesting of the target language version to examine the layout, wording, ease of understanding and ease of completion of the questionnaire; stage VI, submission of the documentation of stages I?V to the developers of the original questionnaire or the AAOS to ensure that the process has been carried out correctly and that a reasonable translation has been achieved.

Subjects
The reliability and validity of the SPADI were examined by a catamnesis of a group of patients who had undergone primary shoulder arthroplasty (mostly due to osteoarthritis or rheumatoid arthritis) and who had been invited to attend for a follow-up examination, on average 4.0 yr after their operation. The first questionnaire was completed during the visit to the hospital and a second questionnaire was completed by the patient at home, 1 week later, and returned by mail. The study protocol was approved by the local ethics committee and written signed informed consent was obtained from all participants.

Reliability
The intraclass correlation coefficient (ICC) was used to quantify test?retest reliability or stability over time, i.e. the extent to which the same test results are obtained for repeated assessments when no real change is expected in the intervening period. The ICC was calculated for the agreement between the two (test and retest) responses for each item in the questionnaire, for the SPADI subscales (pain and disability) and for the total SPADI score. The ICC can range from 0.00 (no agreement) to 1.00 (perfect agreement). Cronbach's alpha (CA) is computed as the average correlation (0.00, no correlation; 1.00, perfect correlation) between the items of a scale to quantify the internal consistency of the item(s) within the scale. Detailed explanations of the ICC and CA can be found in [15, 17].

Several indicators of the precision (or the error) of measurement, as additional indices of reliability, can be found in the literature. One such measurement is the ?standard error of measurement? (SEM; not to be confused with the standard error of the mean in statistics) and another is the ?typical error? (TE) [17]. The SEM is equal to the standard deviation of the group's scores(s) divided by the square root of (1 ? CA) in the one-point measurement case and of (1 ? ICC) in the test?retest data case, as used for our data [15, 18]. The typical error (TE) is equal to the standard deviation of the differences between the test and retest measurements for each patient (s{Delta}) divided by Formula ; it is also given by the square root of the within-person residual mean square from the analysis of variance (ANOVA) of the test and retest results [17]. The 95% confidence interval (CI) for these measurements can be used to quantify the precision of measurement as the mean of the difference (m{Delta}): m{Delta}?tn?1,0.025 x SEM [18] or m{Delta}?tn?1,0.025 x s{Delta} [17], where tn?1,0.025 is the t-value of the Student's t-distribution and s{Delta} = TE x Formula . Further explanations of the two concepts can be found in the Results.

Validity
The validity of the SPADI was examined by determining how well the scores correlated with those of the Short Form 36 (SF-36), the DASH and the ASES. The results were compared with those previously reported in the literature, as revealed by a Pubmed/Medline review of the primary and linked secondary literature for the three most commonly used shoulder instruments, the DASH, the ASES and the SPADI, and for the generic SF-36 [19, 20]. The SF-36 assesses physical, mental and psychosocial health and ability [21]. An overview of the most important shoulder assessment instruments can be found in [8, 9]. The version of the German SPADI used in our previous validity study was identical to that presented in this study [21].

The following concepts have been well defined and explained [22, 23]. Face or content validity is concerned with how well the items or questions seem to make sense; i.e., in the case of the SPADI, how relevant they seem to be to the assessment of shoulder pain and function. Criterion validity describes the extent of agreement between the measure of interest (i.e. the SPADI) and the gold standard (the criterion), i.e. real shoulder pain and function. In the absence of a gold standard, the correlation between the scores on the new instrument and those on a measure that is similarly constructed and has already been validated (e.g. the DASH and patient ASES) can be examined, to determine the extent to which the new instrument corresponds to the theoretical concept of shoulder pain and function (construct validity). The criterion and the construct validity are usually quantified by non-parametric Spearman rank correlation coefficients, as the latter do not require that the data be distributed (statistically) in any specific manner. High correlations are expected between instruments with similar constructs (e.g. the SPADI and the ASES), proving convergent validity and low correlations between instruments with different constructs (e.g. the SPADI and the SF-36) proving discriminant validity [16].

Data were collected using standardized evaluation forms that were then scanned into a database. Prior to analysis, the five pain items and the eight function items (scaled from 0 = best to 11 = worst) of the SPADI were transformed to a scale ranging from 0 (worst) to 100 (best) for ease of comparison with previous studies in which scores were expressed on a 0?100 scale [21]. For example, the pain score is given by (11 ? mean)/11x100, where mean is the mean of the five pain items. All analyses were carried out using the statistical software package SPSS 11.0 for Windows? (SPSS, Chicago, IL, USA).


   Results
 
Cross-cultural adaptation
The original English version is shown in Table 1 and the final German version is available as supplementary data at Rheumatology Online. Forward and back-translation of the SPADI revealed no major problems or language difficulties. Most of the discrepancies in the forward and back translations (stages I?III) concerned synonyms for particular expressions. For example, ?how severe is your pain? (items 1?5): severe -> stark -> strong, ?at its worst? (item 1): worst -> ?am schlimmsten?, ?removing something from your back pocket? (item 13): removing -> ?aus ... nehmen? -> ?take ... out?, etc.


Item 4, ?How severe is your pain when touching the back of your neck?, needed consensus discussion (stage IV). In German, the word ?Nacken? is used for the back of the neck. However, this was then back-translated as simply ?neck? rather than ?back of the neck? (i.e. ?how severe is your pain when touching your neck?). Strictly speaking, this translation is not incorrect, and probably arose as a result of the back-translators being unaware of the purpose of the questionnaire. ?10 pounds? (item 12), which is actually 4.54 kg, was expressed as ?etwa 5 kg? (approximately 5 kg) for ease of understanding. Finally, consensus had to be gained in the translation of ?putting on a shirt that buttons down the front? (item 9) by ?wenn Sie ein Hemd oder eine Bluse anziehen, die vorn zugekn?ft wird?, since ?that buttons down the front? cannot easily be translated word by word, and back-translation resulted in ?which has buttons up the front?.

The extreme (worst) anchor for pain resulted in ?schlimmste Schmerzen? (worst pain) instead of ?worst pain imaginable? because ?schlimmste vorstellbare Schmerzen? was considered a strange formulation and would have been difficult to understand. The extreme (worst) anchor for function (?so difficult that required help?) was translated as ?T?igkeit nicht ausf?rbar? (activity not feasible/unable to do), which was considered to be synonymous with the original response category but easier to judge (because availability and sources of help may differ between patients) and analogous to other instruments, such as the DASH. Pretesting of the pre-final German version (stage V) revealed no further difficulties with the questionnaire in nine patients randomly selected from the hospital waiting room, five health professionals and 42 participants in an earlier, related study [21]. The cross-cultural adaptation process and the final version were approved by K. E. Roach, who developed the English SPADI in 1991 (stage VI) [7].

Subjects
A total of 176 German-speaking patients, who had undergone total shoulder or hemi-arthroplasty of the shoulder at least 1 yr previously (for rheumatoid arthritis or osteoarthritis), were invited to attend the hospital for a follow-up assessment, as part of a larger clinical study (to be published separately). Thirty-two patients (18.2%) could not be assessed: seven had died, 15 were too ill to attend the hospital or to fill in a questionnaire (cancer, dementia, etc.), five had moved to an unknown address and five refused to participate. Of the 144 patients examined, 23 (16.0%) returned incomplete data, i.e. filled in fewer than 2/3 of the items in the pain or function scales, which contravened the previously established missing data rule [21], and three were excluded between the test and retest examinations, due to injury or re-operation of the shoulder. On checking the missing data items, no special preference was detected which could hint, for example, that an item is specially difficult to understand.

This resulted in 118 patients with complete test and retest data (resulting in 125 questionnaire sets because seven patients had been operated bilaterally), examined 1.2?7.9 yr (mean 4.0 yr, S.D. 1.2 yr) after their operation. Their ages ranged from 32.7 to 88.9 yr (mean 69.2 yr, S.D. 11.6 yr) and 77 (65.3%) were female. Twenty out of 125 (16.0%) of all operations were hemi-arthroplasties and 105/125 (84.0%) total arthroplasties.

Reliability
Table 1 shows the descriptive data for the individual items, the subscales and the total SPADI scale, along with the test?retest ICCs. The items showed relatively high ceiling effects (i.e. item score = 0)?up to 61% for item 10 (difficulty in putting on your pants)?whereas the floor effects (i.e. item score = 11) were moderate to low, with a maximum of 19% for item 7 (difficulty in washing your back). Consequently, none of the items or scores was normally distributed (Komolgorov?Smirnov test). The pain, function and total scores were around 70 points (100 = best) and showed very low floor and low ceiling effects.

Of all possible items, 47.7% (125 questionnaires x 13 items = 1625 values) had identical scores at the test and retest (integer item numbers/values from 0 to 11); the lowest agreement was found for item 1 (how severe is your pain at its worst? 37.6% identical values), and the highest for item 10 (difficulty in putting on your pants; 68.8% identical values). The item ICCs ranged from 0.68 to 0.89; the least reliable item was item 9 (difficulty putting on a shirt that buttons down the front) and the most reliable were items 6 (difficulty washing your hair) and item 12 (difficulty in carrying a heavy object of 10 pounds).

Of the 13 items, eight showed ICCs greater than 0.80. The ICCs of the scales were 0.89 for pain, 0.93 for function and 0.94 for the SPADI total score, i.e. the higher the number of items in the scale, the higher the ICC. Internal consistency as measured by the CA was 0.92 for pain, 0.93 for function and 0.95 for the SPADI total score.

Table 2 shows the results describing the precision of measurement. The values for the typical error (TE) indicate that when a given patient is tested and retested, a change of t124,0.025 x s{Delta} = 1.979 x 9.11 = 18 points represents the minimum difference in the total SPADI score required to state (with 95% confidence) that a real change is responsible for the difference and not just measurement error, i.e. 18 is the minimum detectable change score (MDC95%) [17]. In contrast, the concept of Roddey (and Portney) interprets the interval of ?15 to + 10 as the range in which the examiner can be 95% confident that the individual's true total SPADI score is located [15, 18].


Because of the relatively large ceiling effects, post hoc analysis was performed for all single items and scores that did not represent the minimum and maximum score possible (data not shown in detail). The items revealed ICCs from 0.60 (item 13) to 0.79 (item 6), with the exception of item 9, which had an ICC of 0.44. The ICC was 0.87 for the pain score, 0.91 for the function score and 0.93 for the total score.

Validity
Face validity was established in the original English version of the SPADI and was also considered to be adequate for the German SPADI, following discussions within the expert committee (consensus committee at stage IV), i.e. the content of the translated items made sense in the assessment of shoulder pain and function.

The Spearman rank correlation coefficients describing the extent of the correlation between the SPADI scores and those of the comparison questionnaires (i.e. to examine SPADI's construct validity) are shown in Table 3, along with similar data reported in the literature for the original English SPADI [12, 13, 21]. The correlations between the SPADI and the SF-36 physical dimensions, and those between the SPADI and the SF-36 overall score were between 0.61 and 0.69; the correlations between the SPADI and the upper extremity specific measures were somewhat higher: 0.76?0.92 (all P<0.001; Table 3).



   Discussion


The German SPADI proved to be a reliable and valid measure of shoulder pain and function that was feasible to implement in clinical practice. The questions are short, easily understandable and the whole form requires approximately 2 min to complete [21]. The reliability and internal consistency were excellent for both the subscores and the total score. The reliability was also good for the individual items, with the exception of item 9 (difficulty in putting on a shirt that buttons down the front), for which it was still moderate. Compared with the original English SPADI, the German SPADI showed similar or stronger correlations with the SF-36 physical dimensions, the DASH and the (patient) ASES.

The reliability of the German SPADI was higher (SPADI total score, ICC = 0.94) than that of the English version (ICC = 0.66 [7] and 0.91 [12]). The same is true for the subscores: German SPADI, ICC = 0.89 vs original SPADI, ICC = 0.63 for pain and ICC = 0.93 vs 0.64 for function [7]. According to Bot et al. [9], an ICC ≥0.90 allows the reliable assessment of individual patients. This figure was reached for the total score and the function subscore, and was almost reached by the pain subscore.

The internal consistency of the German SPADI was equal to or slightly higher than that of the original SPADI: CA = 0.92 for pain, 0.93 for function and 0.95 for the total German SPADI vs 0.86, 0.93 and 0.95 [7], and 0.89, 0.95 and 0.96 [18] in the original SPADI. CA values greater than 0.90 are interpreted to mean that some items are probably redundant, which may limit the content validity [15].

The standard errors of measurement (SEM) of the German SPADI (Table 2; e.g. the SPADI total score, 6.2) were somewhat higher than those reported by Roddey et al. (e.g. SPADI total score, 4.8) [18]. The typical error (TE) of the German SPADI (total score, TE = 6.4) was higher than that reported for other musculoskeletal symptom-specific questionnaires, e.g. for the Oswestry (Low Back) Disability Index (TE = 3.4), which is also scaled 0 (worst) to 100 (best) [24]. This means that the German SPADI showed lower precision of measurement than the original SPADI and than another musculoskeletal pain and disability instrument. Notably, the concepts of Hopkins and Roddey led to different 95% CIs, which means that there is no generally accepted single method.

The reliability was still acceptable (reducing by just 0.00?0.21 coefficient points), even when the scores that represented a minimum or maximum possible value for a given item/sum-score were excluded. A sample with high floor and/or high ceiling effects will tend to overestimate the reliability, since a test value of 0 (e.g. best health, ceiling) is more likely to be rated identically at the retest than is a score in the middle of the range (e.g. item score 3 = low pain). To our knowledge, this post hoc analysis is not usually performed in most studies examining the psychometric properties of questionnaires, but we recommend that it be done in order to assess the stability of the reliability data.

The face and construct validities of the German SPADI were good, and comparable to (or slightly better than) that of the original SPADI (Table 3) [11, 13, 21]. As expected, because of the differences in the constructs, the correlations between the scores for the condition-specific SPADI and those of the generic SF-36 were low to moderate (proving discriminant validity), whereas the correlations between the SPADI scores and those of the condition-specific instruments (DASH, ASES) were (very) high (proving convergent validity) [16]. The correlations reported in the present study and in the literature to date were relatively consistent across the different settings (Table 3), except those between the SPADI and the patient ASES (r = 0.77, 0.81, 0.92); this should be verified by further studies.

The study's strengths include the standardized methods used for all procedures, especially for the cross-cultural adaptation, and the large sample size. Assessment of cross-cultural adaptation by reliability and validity does not depend on specific diagnoses of shoulder affections (and by that it does not depend on a specific patient selection) since all comparisons are made within the same patient(s) (within-patient data comparisons) and not between different patients. Relevant limitations in relation to the generalizability of the results include the fact that the study was performed only in the German-speaking part of Switzerland and only in shoulder arthroplasty patients. In Germany and Austria, the same (written) German is used as in Switzerland, and language differences between the countries relate only to the spoken dialects. Hence, we expect the questionnaire to perform just as well in these German-speaking countries, although this would have to be verified in further studies. The high ceiling effects observed (2?19% for the items)?perhaps a consequence of examining postoperative patients, in whom symptoms would be expected to be lower?may have inflated the scores for the reliability and validity of the questionnaire: a patient who scores 100 (no pain/no difficulty) in the first assessment would be more likely to score 100 again at retest, which results in perfect reliability for that patient. However, this did not appear to have any notable influence on the results, as post hoc analysis detected no large bias, and other studies that could also be expected to have been affected by similarly high ceiling effects showed comparable results (e.g. [11]).

From a clinical perspective, ceiling effects reveal important limitations of an assessment. Two patients with ceiling scores cannot be (further) differentiated or individually characterized by the instrument. This is not a problem in pain if both do not have pain, but one patient could have better function (e.g. able to lift 10 kg) than the other (e.g. able to lift 20 kg). Additionally, a patient with a ceiling score at baseline cannot improve but only remain stable or worsen at the follow-up. This phenomenon is mainly responsible for the regression-to-the-mean effect of effects measured by closed scales.

Previous studies have shown that, in assessing clinical change in shoulder patients, the English SPADI is more responsive than the SF-36, the modified ASES for the shoulder and four other shoulder-specific instruments [12]. Our future prospective studies will examine the sensitivity to change (responsiveness) and the minimal clinically important (perceptible) difference for the German SPADI, in order to complete the assessment of its clinimetric properties.

Formula

 

 

 
   Acknowledgments
 
Funding to pay the Open Access publication charges for this article was provided by ...

We gratefully thank Astrid Junge, Geoffrey Klein and Vivien Robinson for their help with the cross-cultural adaptation, Gordana Balaban and Katrin Knecht for their help with the data collection, and all the patients who participated in the study. This study was supported by the Georg und Bertha Schwyzer-Winiker Foundation, Zurich, Switzerland.

There are no competing interests to declare.


   References
 

  1. McBeth J and Macfalane GJ. (2002) The prevalence of regional and widespread musculoskeletal pain symptoms. In Linton SJ (Ed.). New avenues for the prevention of chronic musculoskeletal pain and disability(Elsevier, Amsterdam) pp. 7?22.
  2. Stucki G and Kroeling P. (2003) Principles of rehabilitation. Rheumatology 3rd edn (MosbyIn Hochberg MC, Silman AS, Smolen JS, Weinblatt ME, Weisman E (Eds.). , London) Vol. 3:11.1?11.14.
  3. Angst F, Stucki G, Aeschlimann A. (2003) Quality of life assessment in osteoarthritis. Expert Rev Pharmacoeconomics Outcome Res 3:623?36.[CrossRef]
  4. World Health Organization. (2001) ICF ? International Classification of Functioning, Disability and Health(World Health Organization, Geneva, Switzerland) pp. 10?20.
  5. McConnell S, Beaton DE, Bombardier C. (1999) The DASH outcome measure: a user's manual(Institute for Work and Health, Toronto, Ontario).
  6. Richards RR, An KN, Bigliani LU, et al. (1994) American Shoulder and Elbow Surgeons. A standardized method for the assessment of the shoulder function. J Shoulder Elbow Surg 3:347?52.
  7. Roach KE, Budiman-Mak E, Songsiridej N, Lertrantanakul Y. (1991) Development of a shoulder and pain disability index. Arthritis Care Res 4:143?9.[Medline]
  8. McClure P and Michener L. (2003) Measures of adult shoulder function. Arthritis Rheum 49:5SS50?8.[CrossRef]
  9. Bot SDM, Terwee CB, van der Windt DAWM, Bouter LM, Dekker J, de Vet HCW. (2004) Clinimetric evaluation of shoulder disability questionnaires: a systematic review of the literature. Ann Rheum Dis 63:335?41.[Abstract/Free Full Text]
  10. Offenbaecher M, Ewert T, Sangha O, Stucki G. (2002) Validation of a German version of the disabilities of arm, shoulder, and hand questionnaire (DASH-G). J Rheumatol 29:401?2.[ISI][Medline]
  11. Beaton DE and Richards RR. (1996) Measuring function of the shoulder. A cross-sectional comparison of five questionnaires. J Bone Joint Surg Am 78A:882?90.[Abstract/Free Full Text]
  12. Beaton DE and Richards RR. (1998) Assessing the reliability and responsiveness of 5 shoulder questionnaires. J Shoulder Elbow Surg 7:565?72.[CrossRef][ISI][Medline]
  13. Beaton DE, Katz JN, Fossel AH, Wright JG, Tarasuk V. (2001) Measuring the whole or parts? Validity, reliability, and responsiveness of the Disabilities of the Arm Shoulder and Hand outcome measure in different regions of the upper extremity. J Hand Ther 14:128?46.[Medline]
  14. Beaton DE, Bombardier C, Guillemin F, Bosi Ferraz M. (2000) Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 25:3186?91.[CrossRef][ISI][Medline]
  15. Portney LG and Watkins MP. (2000) Statistical measures of reliability. In Portney LG and Watkins MP (Eds.). Foundation of clinical research: applications to practice 2nd edn (Prentice Hall Health, New Jersey, New Jersey) pp. 557?86.
  16. Portney LG and Watkins MP. (2000) Validity of instruments. In Portney LG and Watkins MP (Eds.). Foundation of clinical research: applications to practice 2nd edn (Prentice Hall Health, New Jersey, New Jersey) pp. 79?110.
  17. Hopkins WG. (2000) Measures of reliability in sports medicine and science. Sports Med 30:1?15 Available at: http://www.sportsci.org/resource/stats/precision.html. Accessed 24 January 2006.[CrossRef][ISI][Medline]
  18. Roddey TS, Olson SL, Cook KF, Gartsman GM, Hanten W. (2000) Comparison of the University of California-Los Angeles Shoulder Scale and the Simple Shoulder Test with the Shoulder Pain and Disability Index: single administration reliability and validity. Phys Ther 80:759?68.[Abstract/Free Full Text]
  19. Ware JE, Snow KK, Kosinski M, Gandek B. (2000) SF-36 Health survey: manual and interpretation guide 2nd edn (QualityMetric Incorporated, Lincoln, RI).
  20. Bullinger M and Kirchberger I. (1998) SF-36 Fragebogen zum Gesundheitszustand. Handanweisung(Hogrefe, G?tingen) [The SF-36 questionnaire to assess health status. A manual].
  21. Angst F, Pap G, Mannion AF, et al. (2004) Comprehensive assessment of clinical outcome and quality of life after total shoulder arthroplasty. Usefulness and validity of subjective outcome measurement. Arthritis Rheum 51:819?28.[CrossRef][ISI][Medline]
  22. Jensen MP. (2003) Questionnaire validation: a brief guide for readers of the research literature. Clin J Pain 19:345?52.[CrossRef][ISI][Medline]
  23. Abrahamson JH and Abrahamson ZH. (2001) Validity of a measure. In Abrahamson JH and Abrahamson ZH (Eds.). Making sense of data. A self-instruction manual on the interpretation of epidemiologic data 3rd edn (Oxford University Press, Oxford) pp. 132?4.
  24. Mannion AF, Junge A, Fairbank JCT, Dvorak J, Grob D. (2006) Development of a German version of the Oswestry Disability Index. Part 1: cross-cultural adaptation, reliability, and validity. Eur Spine J 15:55?65.[CrossRef][ISI][Medline]