Predictive performance of machine learning models in acute ischemic stroke: a systematic review and meta-analysis

 I'd fire anybody doing useless prediction research! This doesn't get any survivor recovered, does it?

Predictive performance of machine learning models in acute ischemic stroke: a systematic review and meta-analysis

• 

  Uzma Khanum 1

• 

  Vasudeva Guddattu 2

• S

  Shasthara Paneyala 3

• 

  Asha Srinivasan 4

• 

  Chaithra Nagaraju 1*

• 1. Division of Medical Statistics, School of Life Sciences, JSS Academy of Higher Education and Research, Mysuru, India

• 2. Department of Applied Statistics and Data Science, Manipal Academy of Higher Education, Udupi, India

Abstract

Introduction: 

Acute ischemic stroke (AIS) is a leading cause of global mortality and disability worldwide. Machine learning (ML) models enhance prognostic accuracy by analysing complex, multidimensional clinical data. The aim of this systematic review and meta-analysis is to identify the gaps in the current ML models, along with methodological and performance outcomes in AIS. Further, the study objective was to identify the most frequently used algorithms and compare their relative effectiveness, thereby supporting future research to develop novel ML-based predictive models for stroke care management.

Methods: 

The systematic review followed PRISMA guidelines with PROSPERO registration. A comprehensive search was performed in PubMed, Scopus, and Web of Science using MeSH keywords. Data extraction captured study characteristics, ML algorithms, and outcome metrics. We used the PROBAST and TRIPOD-AI to assess the qualities and bias of included studies. Meta-analysis of AUC values across ML models were conducted to synthesize model performance used a random-effects model to summarize and analyse the data and assessed heterogeneity (I2) statistic using SPSS-29 and R-Studio-4.2.0.

Results: 

A total of 14 studies were included in the systematic review, with 12 eligible for meta-analysis. The pooled AUC of ML models was 0.87 (95% CI, 0.83–0.91), demonstrating strong predictive performance despite substantial heterogeneity (I2 = 99%). Random forest (RF) (AUC = 0.85) and SVM (AUC = 0.82) outperformed logistic regression (LR) (AUC = 0.75), while XGBoost showed stable performance (AUC = 0.82); heterogeneity was mainly driven by study design, publication year, and algorithm type (p < 0.001).

Conclusion: 

ML-based models show potential for improving prognostic assessment in AIS; however, substantial heterogeneity and methodological limitations across studies limit the generalizability of pooled performance estimates.

Systematic review registration: 

https://www.crd.york.ac.uk/PROSPERO/view/CRD420251033217, (Registration number: CRD420251033217).

Machine Learning Techniques for the Prediction of Functional Outcomes in the Rehabilitation of Post-Stroke Patients: A Scoping Review

In what multiverse do you live where predicting failure to recover is of any use at all to survivors? And your mentors and senior researchers approved this crapola?

Machine Learning Techniques for the Prediction of Functional Outcomes in the Rehabilitation of Post-Stroke Patients: A Scoping Review 

by

Christos Kokkotis

¹,

Serafeim Moustakidis

¹,

Georgios Giarmatzis

¹,

Erasmia Giannakou

¹,

Evangelia Makri

¹,

Paraskevi Sakellari

¹,

Dimitrios Tsiptsios

²,

Stella Karatzetzou

²,

Foteini Christidi

²,

Konstantinos Vadikolias

² and

Nikolaos Aggelousis

^1,*

¹

Department of Physical Education and Sport Science, Democritus University of Thrace, 69100 Komotini, Greece

²

Department of Neurology, School of Medicine, University Hospital of Alexandroupolis, Democritus University of Thrace, 68100 Alexandroupolis, Greece

^*

Author to whom correspondence should be addressed.

BioMed 2023, 3(1), 1-20; https://doi.org/10.3390/biomed3010001

Received: 10 November 2022 / Revised: 18 December 2022 / Accepted: 21 December 2022 / Published: 27 December 2022

Download

Browse Figures

Versions Notes

Abstract

Stroke is one of the main causes of long-term disabilities, increasing the cost of national healthcare systems due to the elevated costs of rigorous treatment that is required, as well as personal cost because of the decreased ability of the patient to work. Traditional rehabilitation strategies rely heavily on individual clinical data and the caregiver’s experience to evaluate the patient and not in data extracted from population data. The use of machine learning (ML) algorithms can offer evaluation tools that will lead to new personalized interventions. The aim of this scoping review is to introduce the reader to key directions of ML techniques for the prediction of functional outcomes in stroke rehabilitation and identify future scientific research directions. The search of the relevant literature was performed using PubMed and Semantic Scholar online databases. Full-text articles were included if they focused on ML in predicting the functional outcome of stroke rehabilitation. A total of 26 out of the 265 articles met our inclusion criteria. The selected studies included ML approaches and were directly related to the inclusion criteria. ML can play a key role in supporting decision making during pre- and post-treatment interventions for post-stroke survivors, by utilizing multidisciplinary data sources.

Keywords:

post-stroke; artificial intelligence; prognosis; outcome assessment; rehabilitation

1. Introduction

Stroke, as a result of either a sudden brain blood supply interruption or local brain blood vessel eruption [1], may cause paralysis (plegia) or weakness (paresis), with detrimental consequences on daily activities, such as dressing, eating, and walking, as well as problems with memory, cognition, speaking, emotion control, numbness, and pain [2,3,4]. An aging population and the interaction of risk factors enhance the risk of stroke, leading to an increased number of people with long-term disabilities [3,5]. Forms of stroke rehabilitation include a mixture of pharmacologic and nonpharmacologic interventions that target physiological and functional deficits; however, traditional one-size-fits-all approaches often leave a considerable portion of patients without effective treatment. The design of effective interventions has proven a challenging task due to the high variability of patients’ level of impairment and symptoms [6,7,8]. Hence, the development of personalized assessment/prognostic tools that could lead to better risk stratification and prediction of functional outcomes, based on past and current data, is necessary. Innovative, evidence-based strategies that can utilize longitudinal, multisource population data could aid individual rehabilitation by shaping personalized interventions, both at admission and throughout the patient’s path of care.

To this end, classical statistical approaches such as linear regression have been employed to model post-stroke rehabilitation and predict functional outcomes, using a binary (good or poor) classification or specific score outcomes. These models, which are typically based on standard scales and relevant clinical data, fail to incorporate meaningful factors that are detrimental to patient-specific recovery pathways, such as the level of family or community support and the cultural level. On the contrary, advanced artificial intelligence (AI)-based correlation models, such as machine learning algorithms, can analyze large, inhomogeneous datasets, map nonlinearities between multiple input and output variables, and extract patterns among various clinical outcomes.

ML is the study of how machines (i.e., learning algorithms) can learn patterns or complex relationships from daily data and produce trained mathematical models linking target variables of interest with a huge number of covariates. Furthermore, deep learning (DL) is defined as a subfield of ML concerned with learning algorithms inspired by the function and structure of the brain [7]. DL provides an alternative architecture system by overcoming the burden of feature engineering. Hence, ML has the ability to cope with high-dimensionality data and complex cases [9,10], going beyond the traditional statistical approaches and overcoming significant limitations in the prediction of the functional outcome in the rehabilitation of post-stroke survivors, offering valuable tools in the field of stroke rehabilitation [3].

Currently, various ML techniques have been employed to model individual disease pathways, and their contribution plays a key role in the scientific community. For example, in the case of knee osteoarthritis, several ML-based patient-specific prediction models have been developed (e.g., ML models for diagnosis, post-treatment assessment, and segmentation in knee osteoarthritis) [11]. Moreover, ML demonstrated excellent performances in predicting the outcome for several neurosurgical conditions [12,13]. In addition, Bivard et al. demonstrated the importance of AI and imaging in stroke management [14]. Furthermore, they presented the need for AI tools for the quick assessment of meaningful imaging data and to support clinical decisions.

In this context, the current scoping review was carried out to (i) investigate ML methods employed to predict functional outcome in stroke rehabilitation, (ii) identify current trends in this field, and (iii) identify the existing literature gap for future scientific approaches. Compared to the already available literature on the field of post-stroke rehabilitation [15], this paper focuses specifically on the various ML models being used to predict functional outcomes in stroke rehabilitation, as well as their specific characteristics and underlying principles. By providing an in-depth examination of these models, the paper aims to shed light on the current state of the field and identify trends and areas for future research. Overall, the primary aim of the paper is to provide a comprehensive overview of the ML approaches being used in this area of study.

 

More at link.

Machine learning methods for functional recovery prediction and prognosis in post-stroke rehabilitation: a systematic review

 I'd fire everyone involved in useless predicting failure to recover research like this.

Machine learning methods for functional recovery prediction and prognosis in post-stroke rehabilitation: a systematic review

• Silvia Campagnini,
• Chiara Arienti,
• Michele Patrini,
• Piergiuseppe Liuzzi,
• Andrea Mannini &
• Maria Chiara Carrozza 

Journal of NeuroEngineering and Rehabilitation volume 19, Article number: 54 (2022) Cite this article

• 93 Accesses

• Metrics details

Abstract

Background

Rehabilitation medicine is facing a new development phase thanks to a recent wave of rigorous clinical trials aimed at improving the scientific evidence of protocols. This phenomenon, combined with new trends in personalised medical therapies, is expected to change clinical practice dramatically. The emerging field of Rehabilomics is only possible if methodologies are based on biomedical data collection and analysis. In this framework, the objective of this work is to develop a systematic review of machine learning algorithms as solutions to predict motor functional recovery of post-stroke patients after treatment.

Methods

We conducted a comprehensive search of five electronic databases using the Patient, Intervention, Comparison and Outcome (PICO) format. We extracted health conditions, population characteristics, outcome assessed, the method for feature extraction and selection, the algorithm used, and the validation approach. The methodological quality of included studies was assessed using the prediction model risk of bias assessment tool (PROBAST). A qualitative description of the characteristics of the included studies as well as a narrative data synthesis was performed.

Results

A total of 19 primary studies were included. The predictors most frequently used belonged to the areas of demographic characteristics and stroke assessment through clinical examination. Regarding the methods, linear and logistic regressions were the most frequently used and cross-validation was the preferred validation approach.

Conclusions

We identified several methodological limitations: small sample sizes, a limited number of external validation approaches, and high heterogeneity among input and output variables. Although these elements prevented a quantitative comparison across models, we defined the most frequently used models given a specific outcome, providing useful indications for the application of more complex machine learning algorithms in rehabilitation medicine.

Background

Vascular problems in nature are the leading cause of death, and stroke is ranked second in worldwide mortality [1]. It accounted for 5.5 million deaths in 2006 [2]. Indeed, for survivors, the burden of stroke is producing an increase in the number of disability-adjusted living years (DALYs). For this reason, the ultimate challenge in stroke rehabilitation research is to improve the rehabilitation protocols by tuning them according to an optimised early outcome prognosis [3]. Therefore, advances in artificial intelligence, machine learning (ML), and more generically data-driven tools, may have a central role in rehabilitation decision-making and protocol development. ML is the methodology that provides computers with the ability to learn from experience. By designing and training algorithms able to learn decision rules from data, automatic solutions able to make predictions on new data can be exploited [4].

ML algorithms have been used often in recent years to predict clinical outcomes [5]. The recent growing interest is due to the increasing complexity and numerosity of available data sets, as well as the presence of multifactorial data with diverse origins, for which more classical methods do not allow accurate results [6, 7].

From this perspective and given the available technologies, a new concept of rehabilitation is arising, namely “Rehabilomics”. This innovative view of the rehabilitative intervention concerns a multifactorial data-driven evaluation of the patient, aiming at the identification of physiological, genetic, biochemical or metabolic biomarkers as factors concurring in the rehabilitation process. The correlation of these biomarkers with the clinical outcome that measures the recovery of the patient could lead to important information for rehabilitation treatment planning.

Considering the latest advances in ML-based predictive models could be employed to promote the development of personalised rehabilitation processes for individual recovery. This would result in a human-centred framework in which the synergy among therapies, biogenetics, imaging techniques, technological devices and data-driven tools has a key role [8].

In the literature, there has been a broad exploration of solutions for outcome prediction in medicine applications [6, 9,10,11], and very few of them are about ML models in stroke rehabilitation [12, 13]. Most of the reviews in this field provide only a narrative description of the studies, without providing a systematic analysis of the results. On the other hand, those prioritising the technical aspects of the models often lack a clinical contextualisation of the findings. For example, Christodoulou et al. [6], ML methods for clinical outcome prediction are compared across pathologies without providing details about the outcomes used. So, although the review was highly detailed from the technical point of view, i.e. regarding the algorithms validation approaches and performance metrics used, the clinical aspects were out of focus. We are convinced that a proper discussion of the results in light of the clinical context (i.e., pathology and measures) in which they are obtained is essential for translational applicability of the solutions developed, from research to the clinical practice.

Thus, there is an urgent need for a study able to integrate and combine clinical and engineering/technical aspects of predictive solutions used in rehabilitation. The aim of this study is to identify the predictive solutions, developed using ML or theory-based algorithms and internally or externally validated, used for functional outcome prognosis in stroke patients after a rehabilitation programme. The predictive solutions were investigated comprehensively, by evaluating their technical characteristics and performances in association with the clinical selection of input and output variables.

More at link.

 

Machine Learning Methods Predict Individual Upper-Limb Motor Impairment Following Therapy in Chronic Stroke

Oh fuck, more prediction crapola. DO YOU NOT KNOW THAT SURVIVORS DON'T CARE ABOUT PREDICTIONS, THEY WANT REHAB RESULTS? Just maybe if you talked to a few survivors before starting worthless research like this.  And they are all Ph.Ds, probably their mentors and senior researchers also. Meaning that educational achievement doesn't necessarily include common sense. 

Machine Learning Methods Predict Individual Upper-Limb Motor Impairment Following Therapy in Chronic Stroke

Show all authors

Ceren Tozlu, PhD, Dylan Edwards, PhD*, Aaron Boes, MD, PhD, ...

First Published March 20, 2020 Research Article Find in PubMed

https://doi.org/10.1177/1545968320909796

Article information

Abstract

Background.
Accurate prediction of clinical impairment in upper-extremity motor function following therapy in chronic stroke patients is a difficult task for clinicians but is key in prescribing appropriate therapeutic strategies. Machine learning is a highly promising avenue with which to improve prediction accuracy in clinical practice.  
Objectives.
The objective was to evaluate the performance of 5 machine learning methods in predicting postintervention upper-extremity motor impairment in chronic stroke patients using demographic, clinical, neurophysiological, and imaging input variables.  
Methods.
A total of 102 patients (female: 31%, age 61 ± 11 years) were included. The upper-extremity Fugl-Meyer Assessment (UE-FMA) was used to assess motor impairment of the upper limb before and after intervention. Elastic net (EN), support vector machines, artificial neural networks, classification and regression trees, and random forest were used to predict postintervention UE-FMA. The performances of methods were compared using cross-validated R².  
Results.
EN performed significantly better than other methods in predicting postintervention UE-FMA using demographic and baseline clinical data (median R2EN=0.91,R2RF=0.88,R2ANN=0.83,R2SVM=0.79,R2CART=0.70;

P < .05). Preintervention UE-FMA and the difference in motor threshold (MT) between the affected and unaffected hemispheres were the strongest predictors. The difference in MT had greater importance than the absence or presence of a motor-evoked potential (MEP) in the affected hemisphere.  
Conclusion.
Machine learning methods may enable clinicians to accurately predict a chronic stroke patient’s postintervention UE-FMA. Interhemispheric difference in the MT is an important predictor of chronic stroke patients’ response to therapy and, therefore, could be included in prospective studies.

Machine Learning Methods Predict Individual Upper-Limb Motor Impairment Following Therapy in Chronic Stroke

What fucking stupidity. You think it is more important to predict recovery than actually provide recovery? You're fired. Survivors need to be in charge, we wouldn't allow this crapola.

Machine Learning Methods Predict Individual Upper-Limb Motor Impairment Following Therapy in Chronic Stroke

Show all authors

Ceren Tozlu, PhD, Dylan Edwards, PhD, Aaron Boes, MD, PhD, ...

First Published March 20, 2020 Research Article

https://doi.org/10.1177/1545968320909796

Article information

Abstract

Background.
Accurate prediction of clinical impairment in upper-extremity motor function following therapy in chronic stroke patients is a difficult task for clinicians but is key in prescribing appropriate therapeutic strategies.(Really? Have you talked to ANY stroke survivors?) Machine learning is a highly promising avenue with which to improve prediction accuracy in clinical practice.  

Objectives.
The objective was to evaluate the performance of 5 machine learning methods in predicting postintervention upper-extremity motor impairment in chronic stroke patients using demographic, clinical, neurophysiological, and imaging input variables.
Methods. A total of 102 patients (female: 31%, age 61 ± 11 years) were included. The upper-extremity Fugl-Meyer Assessment (UE-FMA) was used to assess motor impairment of the upper limb before and after intervention. Elastic net (EN), support vector machines, artificial neural networks, classification and regression trees, and random forest were used to predict postintervention UE-FMA. The performances of methods were compared using cross-validated R².
Results. EN performed significantly better than other methods in predicting postintervention UE-FMA using demographic and baseline clinical data (median R2EN=0.91,R2RF=0.88,R2ANN=0.83,R2SVM=0.79,R2CART=0.70;

P < .05). Preintervention UE-FMA and the difference in motor threshold (MT) between the affected and unaffected hemispheres were the strongest predictors. The difference in MT had greater importance than the absence or presence of a motor-evoked potential (MEP) in the affected hemisphere.  
Conclusion. Machine learning methods may enable clinicians to accurately predict a chronic stroke patient’s postintervention UE-FMA. Interhemispheric difference in the MT is an important predictor of chronic stroke patients’ response to therapy and, therefore, could be included in prospective studies.