Influence of virtual reality and task complexity on digital health metrics assessing upper limb function

 I consider 'assessments' to be totally worthless. Nothing here gets survivors recovered!

Influence of virtual reality and task complexity on digital health metrics assessing upper limb function

• Christoph M. Kanzler,
• Tom Armand,
• Leonardo Simovic,
• Ramona Sylvester,
• Nadine Domnik,
• Antonia M. Eilfort,
• Carola Rohner,
• Roger Gassert,
• Roman Gonzenbach &
• Olivier Lambercy 

Journal of NeuroEngineering and Rehabilitation volume 21, Article number: 125 (2024) Cite this article

• Metrics details

Abstract

Background

Technology-based assessments using 2D virtual reality (VR) environments and goal-directed instrumented tasks can deliver digital health metrics describing upper limb sensorimotor function that are expected to provide sensitive endpoints for clinical studies. Open questions remain about the influence of the VR environment and task complexity on such metrics and their clinimetric properties.

Methods

We aim to investigate the influence of VR and task complexity on the clinimetric properties of digital health metrics describing upper limb function. We relied on the Virtual Peg Insertion Test (VPIT), a haptic VR-based assessment with a virtual manipulation task. To evaluate the influence of VR and task complexity, we designed two novel tasks derived from the VPIT, the VPIT-2H (VR environment with reduced task complexity) and the PPIT (physical task with reduced task complexity). These were administered in an observational longitudinal study with 27 able-bodied participants and 31 participants with multiple sclerosis (pwMS, VPIT and PPIT only) and the value of kinematic and kinetic metrics, their clinimetric properties, and the usability of the assessment tasks were compared.

Results

Intra-participant variability strongly increased with increasing task complexity (coefficient of variation + 56%) and was higher in the VR compared to the physical environment (+ 27%). Surprisingly, this did not translate into significant differences in the metrics’ measurement error and test–retest reliability across task conditions (p > 0.05). Responsiveness to longitudinal changes in pwMS was even significantly higher (effect size + 0.35, p < 0.05) for the VR task with high task complexity compared to the physical instrumented task with low task complexity. Increased inter-participant variability might have compensated for the increased intra-participant variability to maintain good clinimetric properties. No significant influence of task condition on concurrent validity was present in pwMS. Lastly, pwMS rated the PPIT with higher usability than the VPIT (System Usability Scale + 7.5, p < 0.05).

Conclusion

The metrics of both the VR haptic- and physical task-based instrumented assessments showed adequate clinimetric properties. The VR haptic-based assessment may be superior when longitudinally assessing pwMS due to its increased responsiveness. The physical instrumented task may be advantageous for regular clinical use due to its higher usability. These findings highlight that both assessments should be further validated for their ideal use-cases.

Introduction

Upper limb disability is common in neurological disorders, such as persons with multiple sclerosis (pwMS), which strongly contributes to an inability to perform daily life activities and increases dependency on caregivers [1]. In clinical studies, assessments are of fundamental importance to advance our understanding of the types of upper limb impairments and their underlying mechanisms [2]. In addition, assessments are essential to provide sensitive and reliable endpoints that can be used to evaluate the effectiveness of pharmacological or rehabilitation interventions.

The most commonly applied assessments in clinical studies subjectively describe movement quality on ordinal scales or capture the time to complete functional tasks [2]. While these assessments have high usability, provide a good overview of the disability level of a patient, and are well-accepted by the clinical community, they have a limited ability to serve as detailed, insightful endpoints for clinical studies [2, 3]. This is because ordinal scales typically have ceiling effects and low sensitivity, while subjective assessments are prone to rater-induced bias [4]. In addition, time-based assessments are not able to provide information on the mechanism underlying suboptimal task performance; for example, they cannot distinguish whether grip force control or gross movement control is impaired. Because of these limitations, there is a consensus in the research community that novel, complementary and more sensitive endpoints are urgently required to provide more detailed insights into the mechanisms of upper limb impairments and the effect of therapeutic interventions [3, 5, 6].

Technology-based assessments can record objective sensor-based data on upper limb movement patterns and hand grip forces during functional manipulation tasks [7, 8]. These can be transformed into digital health metrics (discrete one-dimensional metrics extracted from health-related sensor data such as movement kinematics and kinetics) with ratio scales, thereby promising novel, sensitive, and insightful endpoints [9, 10]. Technology-based assessments often consist of a robotic interface (e.g., haptic devices) that serves as a control input (i.e., joystick) and a virtual reality (VR) environment with a goal-directed manipulation task rendered, for example, on a 2D computer screen [11], [12,13,14,15].

VR environments are a unique element of technology-based assessments, as they provide flexibility in the implementation of assessment tasks with different levels of complexity to target specific sensorimotor and cognitive impairments. Also, VR environments promise to increase engagement and motivation of participants, and VR-based depth cues can support a realistic representation of 3D movements on a 2D screen [16, 17]. However, when compared to physical environments, VR environments and the different levels of task complexity they may generate are also known to influence the kinematics of goal-directed movements. This can be, for example, in terms of reduced smoothness and speed, or increased movement variability [15, 18,19,20,21,22]. Crucially, it remains an open question whether this change in kinematics and variability also influences the extracted digital health metrics and in particular their clinimetric properties. These properties include test–retest reliability, measurement error, responsiveness, and concurrent validity and ultimately determine whether digital health metrics can be used as insightful and robust endpoints in clinical studies [9, 10, 23].

The aim of this work is to describe the influence of VR and task complexity on the clinimetric properties of digital health metrics extracted from a goal-directed, technology-based upper limb assessment. The secondary aim is to describe the influence of these two factors on the magnitude of the metrics, the observed intra-participant variability, and the usability of the technology-based assessment.

For this purpose, we relied on the Virtual Peg Insertion Test (VPIT, Fig. 1), a previously established haptic end effector- and VR-based assessment describing upper limb movement patterns and hand grip force control during the insertion of nine virtual pegs into nine holes. To assess the impact of both task complexity and VR, we developed two distinct assessment tasks: the VPIT-2 Hole (VPIT-2H, Fig. 1) requires inserting only two virtual pegs into two virtual holes, thereby simplifying the original VPIT. To examine the influence of VR, we introduced the Physical Peg Insertion Test (PPIT, Fig. 1). The PPIT uses the same end effector device as the VPIT, but instead of a virtual task, it uses a physical pegboard with two magnetic pegs and physical holes, and an electromagnet to transport the magnetic pegs. These assessments were administered in an observational longitudinal study with 27 able-bodied participants (VPIT, VPIT-2H, and PPIT; test and retest) and 31 pwMS (VPIT and PPIT; admission and discharge to a rehabilitation program; Fig. 2).

Fig. 1

Assessment platforms VPIT (A), PPIT (B), and virtual display of the VPIT-2H (C). These are used to study the influence of task complexity and virtual reality on the clinimetric properties of digital health metrics

More at link.