Automatic Temporal Segmentation for Post-Stroke Rehabilitation: A Keypoint Detection and Temporal Segmentation Approach for Small Datasets

FYI, no clue.

 Automatic Temporal Segmentation for Post-Stroke Rehabilitation: A Keypoint
Detection and Temporal Segmentation Approach for Small Datasets

isoo Lee

Arizona State University

jlee815@asu.edu

Tamim Ahmed

University of Southern California

tamimahm@usc.edu

Thanassis Rikakis

University of Southern California

rikakis@usc.edu

Pavan Turaga

Arizona State University

Pavan.Turaga@asu.edu

Abstract

Rehabilitation is essential and critical for post-stroke

patients, addressing both physical and cognitive aspects.

Stroke predominantly affects older adults, with 75% of

cases occurring in individuals aged 65 and older, under-

scoring the urgent need for tailored rehabilitation strategies

in aging populations. Despite the critical role therapists

play in evaluating rehabilitation progress and ensuring the

effectiveness of treatment, current assessment methods can

often be subjective, inconsistent, and time-consuming, lead-

ing to delays in adjusting therapy protocols. This study aims

to address these challenges by providing a solution for con-

sistent and timely analysis. Specifically, we perform tem-

poral segmentation of video recordings to capture detailed

activities during stroke patients’ rehabilitation. The main

application scenario motivating this study is the clinical as-

sessment of daily tabletop object interactions, which are

crucial for post-stroke physical rehabilitation. To achieve

this, we present a framework that leverages the biomechan-

ics of movement during therapy sessions. Our solution di-

vides the process into two main tasks: 2D keypoint detec-

tion to track patients’ physical movements, and 1D time-

series temporal segmentation to analyze these movements

over time. This dual approach enables automated labeling

with only a limited set of real-world data, addressing the

challenges of variability in patient movements and limited

dataset availability. By tackling these issues, our method

shows strong potential for practical deployment in physi-

cal therapy settings, enhancing the speed and accuracy of

rehabilitation assessments.

1. Introduction

Stroke is a medical condition that occurs when the blood

supply to the brain is interrupted, resulting in brain tissue

damage, which can lead to disability or even death. Aging

is a major contributor to stroke, with the risk doubling ev-

ery decade after the age of 55, and approximately 75% of

stroke patients being 65 or older [24]. Rehabilitation ther-

apy is essential for minimizing disability and helping pa-

tients recover physical and cognitive functions. This need is

especially critical for older adults, as aging leads to greater

vulnerability in both physical and cognitive functions, mak-

ing rehabilitation even more necessary.

Effective rehabilitation relies on thorough assessments to

tailor treatment plans. Currently, therapists monitor these

assessments, evaluating the patient’s physical and cognitive

progress to refine therapy protocols accordingly. However,

this process is time-intensive and may vary among thera-

pists, creating a need for more objective, automated solu-

tions.

Advancements in deep learning have enabled automation

across various fields, including healthcare. In rehabilitation,

deep learning offers the potential to streamline and enhance

the assessment process. Despite this potential, challenges

remain, including the limited availability of real-world pa-

tient data, difficulties in using synthetic data, and the com-

putational demands of processing video data, which often

involve spatial and temporal complexities.

To overcome these challenges, we propose a novel

framework that decomposes complex tasks into smaller,

more manageable sub-tasks based on domain-specific in-

sights. By focusing on critical hand-object interactions

during the Action Research Arm Test (ARAT), we isolate

key movements and extract 2D joint coordinates for de-

tailed analysis. This targeted approach allows us to re-

duce model complexity and mitigate overfitting, even with

 a small dataset. We utilize the ASAR (Affective State for

ARAT Rehab) dataset [2], which consists of video record-

ings of stroke patients performing the standardized Action

Research Arm Test (ARAT) [25]. ARAT assesses upper ex-

tremity motor function in stroke patients by evaluating their

ability to perform tasks such as grasping, moving, and lift-

ing objects, aiding in tracking rehabilitation progress.

Additionally, given the subjective nature of the ARAT

assessment, the criteria for segmenting actions may vary de-

pending on the therapist. To address this, instead of retrain-

ing the entire model, we propose adjusting only the tempo-

ral segmentation phase to accommodate the changed crite-

ria. This approach offers significant flexibility by allowing

adaptation to different segmentation standards without the

need for complete model retraining.

More at link.