Responsiveness and Validity of 3 Outcome Measures of Motor Function After Stroke Rehabilitation

 Measurements NEVER GET ANYONE RECOVERED! I'd have you all fired for incompetency in not solving stroke!

Responsiveness and Validity of 3 Outcome Measures of Motor Function After Stroke Rehabilitation

To the Editor: With interest we read the recently published letter of Dr Sivan, “Interpreting Effect Size to Estimate Responsiveness of Outcome Measures,” 1 as a response to a paper by Hsieh et al 2 in which they provided indices of the magnitude of treatment-related intraindividual change assessed with the Fugl-Meyer Assessment (FMA), Action Research Arm Test (ARAT), and Wolf Motor Function Test performance time (WMFT-TIME) and functional ability scores (WMFT-FAS). As an effect size index, Hsieh et al used the method of the so-called standardized response mean (SRM) by which mean change in scores over time is divided by the SD of these change scores (see Formula A). (A) SRM= X  change SD (Xchange) As Sivan argued in his letter, 1 the interpretation of the magnitude of intraindividual change estimated with a SRM may lead to overestimation or underestimation of treatment- related effects when the widely used thresholds of Cohen 3 are used. These thresholds for classification of the magnitude of mean differences were developed with an effect size index based on standardizing these mean differences using the pooled SD (see Formula B). Dunlap et al convincingly argued that only the pooled SD should be used to compute effect size (ES) for correlated designs and concluded that if the SD pooled is corrected for the amount of correlation between the measures, then the ES estimate will be an overestimate of the actual ES. 4 It is essential for clinical investigators to under- stand the differences between the SRM and ES in classifying treatment-related change in terms of Cohen’s thresholds (ES 0.20 indicating a “trivial” change, ES between 0.20 and 0.50 “small,” ES of 0.50 to 0.80 a moderate, and ES 0.80 a large change). 3 (B) ES p = X  change SD (pooled) Hsieh et al refer to our earlier work concerning the risk of misclassification of an SRM when using Cohen’s thresh- olds 5,6 in their response to Sivan’s critic. However, their calculation of adjusted ES estimates 7 is based on a false assumption. Consequently, adjusting ES for the size of the correlation between baseline and follow-up as computed by Hsieh et al 7 in the Table leads to an ES estimate more than twice the magnitude of the ES computed using the SD pooled when the correlation between the baseline and follow-up scores is at least 0.8. 4 Adjustment of a SRM to ES comprises 2 components. First, Cohen introduced a (2) correction as necessary for an appropriate use of his tables for sample size calculation. This correction for looking in Cohen’s power tables is necessary because these assume 2(N-1) degrees of freedom (2 indepen- dent samples), whereas in, for example, pre-/posttest evalu- ation, only n-1 are actually available 3 (pp 46 to 48). Thus, following Cohen’s theory, “multiplying SRM by 2 (ap- proximately 1.41) compensates for the sample size tables’ assumption of double the error variance” 3 (p 46). Second, because the t test prescribed in “own control” study designs (baseline to follow-up) is based on correlated means 3 (p 48), we also have to compensate for the correlation (r) between paired observations. Therefore, according to Cohen, the relative size of the standardizing unit for the SRM to the ES pooled is not (C) d= d'  1-r but (D)  2 1-r) Thus, the difference between means for paired (dependent) samples needs to be standardized by a value “which is 2 (1-r) as large as would be the case were they independent” 3 (p 49). As was shown in an earlier publication, (d'/2)/(1-r) is equivalent to the SRM and alternatively SRM *  2* (1-r) Table. Effect Size Estimates SRM, ESP, and ES as Adjusted by Hsieh et al Scale r r 2 A Mean Change B SD change C SD pooled A/B Effect Size d (SRM) A/C Effect Size pooled (ESP) Adjusted Effect Size 7 (ESP)/(1-r) FMA 0.901 0.81 5.75 4.06 8.67 1.42 0.66 2.06 ARAT 0.915 0.84 4.68 4.95 11.24 0.95 0.41 1.40 WMFT-TIME 0.594 0.35 2.13 5.56 5.91 0.38 0.36 0.56 WMFT-FAS 0.951 0.90 0.30 0.23 0.73 1.30 0.41 1.85 FMA indicates Fugl-Meyer Assessment; ARAT, Action Research Arm Test; WMFT, Wolf Motor Function Test performance time (WMFT-TIME) and functional ability scores (WMFT-FAS).

Tables and equations are better seen at the link.