The Global Burden of Disease Study 2019 shows an increase in overall stroke incidents and exposes stroke as the third-leading cause of death and disability combined. About 80% of stroke survivors are affected by motor deficits, with approximately 50% experiencing persistent upper limb impairments [17, 27, 42].
Considering the involvement of arm and hand functions in activities of daily living (ADL), facilitating upper limb motor recovery following a stroke is essential. According to the current state of scientific consensus, motor recovery is distinguished by true recovery and motor compensation [8, 24, 32]. The Stroke Recovery and Rehabilitation Roundtable (SRRR) defines true recovery as regaining the same movement patterns as available before the injury and motor compensation as the development of new motor patterns by using intact body structures to accomplish an activity goal [8]. The definitions provided by Levin et al. [32] and Kleim [24] distinguish between motor compensation and true recovery across three different levels of the international Classification of Functioning, Disability and Health (ICF [62]) domains: health condition, body function/structure and activity. According to Levin et al. [32] changes at the Health Condition level occur through processes at the neuronal level. On this level, motor compensation is characterized by structural reorganization within the brain, where other brain regions assume the functions of damaged areas. At the body function/structure level, compensation is reflected in alternative movement patterns. At the activity level, compensation is evident when different limbs or end effectors are used compared to the premorbid status. Achieving a desired goal with the impaired arm can promote its use and prevent learned non-use, potentially enhancing functional capacity and independence in activities of daily living [21, 22]. However, compensatory movements can lead to musculoskeletal changes, increasing the risk of chronic pain [12, 13, 32]. Furthermore, compensation with the unimpaired limb can negatively impact neural reorganization, potentially hindering the true recovery of the impaired limb [22, 46]. Therefore, it is important to support the true recovery on the body function/structure level during rehabilitation, rather than allowing alternative movement patterns, to minimize adverse side effects. However, to facilitate the identification and treatment of motor compensation, a comprehensive assessment is necessary [10, 26]. Compensatory movements can be assessed by measuring the quality of upper limb movement patterns through motion analysis. Possible approaches to conduct this analysis include qualitative descriptions of movements based on visual observation, observer-based scoring using standardized assessment tools or kinematic motion analysis technologies, such as marker-based motion capture systems [50].
In research settings, marker-based motion capture systems are considered the gold standard for motion analysis. Unfortunately, its implementation in clinical practice is only possible to a limited extent because of high costs, duration of the assessment as well as requirement for high-quality instruments and specialized training [60].
Core measurement sets recommended to be applied in motor rehabilitation and recovery trials post-stroke describe the Fugl-Meyer Assessment (FMA, [16]) as essential tool for evaluating upper limb body functions, and the Action Research Arm Test (ARAT, [33]) for assessing motor function at the activity level [25, 41]. However, previous research indicated that these assessments simply focus on the ability to complete a task, without capturing the movement quality. Hence they do not differentiate between the recovery of movement patterns and the use of compensatory movement strategies [26, 32, 44]. In response to the need for an assessment specifically designed to measure motor compensation, Levin et al. [30] developed the Reaching Performance Scale for Stroke (RPSS). The RPSS [30] is an ordinal-scale assessment tool that evaluates compensatory movements of the upper extremity during two specific reaching tasks, excluding drinking motions. Recent studies by Subramanian et al. [54, 55] on the RPSS [30] provide promising results, describing it as a valid, reliable, and responsive scale for visually assessing motor compensation. However, the assessment is susceptible to ceiling effects due to its ordinal scaling. To our knowledge the RPSS is not yet commonly used in clinical practice. Consequently, therapists assess compensatory movements qualitatively through visual observation relying on their clinical expertise and experience [14, 28, 44, 47].
In recent years, considerable research has focused on developing assessments for motor compensation. However, due to the complexity of developing accurate and sensitive observer-based scoring assessments and the lack of consensus regarding the use of marker-based motion capture systems, further research is required to construct reliable, feasible and affordable measurement tools [26, 29, 47, 51].
Previous research has demonstrated that kinematic motion analysis technologies, such as marker-based motion capture systems, wearable sensors, marker-free vision sensors (e.g. simple cameras, or Microsoft Kinect depth sensors) or sensors embedded in rehabilitation training systems, provide objective, consistent, and precise detection of compensatory movements through automated processes, devoid of any floor and ceiling effects respectively [23, 28, 36, 60]. Recent reviews [39, 47] have highlighted the advantages of Machine Learning (ML) algorithms, which include high agreement levels, high accuracy, and low-cost, unobtrusive home based monitoring of patients. Supervised ML algorithms depend on labeled data, which serves as ground truth, to derive an optimal model capable of accurately predicting outcomes or classifying new, unseen data by leveraging learned patterns from the training dataset [5, 40]. In stroke rehabilitation, a promising domain for applying ML algorithms lies in the automated analysis of movement patterns in impaired limbs [39].
Therefore, we are currently conducting larger project to develop a ML-supported tool to analyze compensatory movements of the upper extremities and trunk on the body function/structure level, which can be used remotely with simple methods in the homes of patients after stroke [52, 56]. In this larger study, we plan to utilize simple webcam or smartphone cameras to assess compensation during a drinking task [1]. During the development of the ML algorithm, the completion of the task is recorded with a webcam and manually labeled by experienced therapists, who assess the extent of compensatory movements. This data was intended to be used to create a ground truth for the ML algorithm. While therapists are trained to assess movement quality in person (3-dimensional), it is unclear if they can do so when rating a drinking task based on videos (2-dimensional). The findings from Martinez et al. [35], Bernhardt et al. [6] and Bernhardt et al. [7] demonstrated good reliability in assessing reaching and object-lifting tasks using video analysis. However, these studies did not specifically evaluate the drinking task.
We seek to imitate human decision-making behavior using ML-supported models. To train an ML algorithm to imitate human intelligence, i.e. to build a ground truth that reflects human knowledge, we must make implicit therapeutic decisions, such as rating compensatory movements, explicitly available. This raises a broader discussion, which we intend to explore in this article. If we want ML to learn from human intelligence, a key challenge is determining whether humans can reliably assess movements during a drinking task from 2-dimensional video recordings in the first place. Depending on the findings, it remains uncertain whether basing a ground truth for assessment on human decisions would be a viable or effective approach.
Therefore, we conducted this study to investigate the inter-rater and intra-rater reliability of experienced therapists’ assessment of compensatory movement behavior based on video recordings and hence infer if such ratings are suitable for creating a ground truth for an ML application.
