Predicting Poor Outcome Before Endovascular Treatment in Patients With Acute Ischemic Stroke

Aren't you glad your doctor will tell you before your treatment that you will likely have a poor outcome, rather than saying; 'We have the protocols in place to get you 100% recovered?

Which version do you want to hear?

I would have everyone associated with this fired. It does nothing to get survivors a better recovery.

Predicting Poor Outcome Before Endovascular Treatment in Patients With Acute Ischemic Stroke

Lucas A. Ramos^1,2*, Manon Kappelhof³, Hendrikus J. A. van Os⁴, Vicky Chalos^5,6,7, Katinka Van Kranendonk³, Nyika D. Kruyt⁴, Yvo B. W. E. M. Roos⁸, Aad van der Lugt⁷, Wim H. van Zwam⁹, Irene C. van der Schaaf¹⁰, Aeilko H. Zwinderman², Gustav J. Strijkers^1,3, Marianne A. A. van Walderveen¹¹, Mariekke J. H. Wermer⁴, Silvia D. Olabarriaga², Charles B. L. M. Majoie³ and Henk A. Marquering^1,3

• ¹Department of Biomedical Engineering and Physics, University of Amsterdam, Amsterdam, Netherlands
• ²Department of Clinical Epidemiology and Biostatistics, University of Amsterdam, Amsterdam, Netherlands
• ³Department of Radiology and Nuclear Medicine, University of Amsterdam, Amsterdam, Netherlands
• ⁴Department of Neurology, Leiden University Medical Center, Leiden, Netherlands
• ⁵Department of Neurology, Erasmus MC - University Medical Center, Rotterdam, Netherlands
• ⁶Department of Public Health, Erasmus MC - University Medical Center, Rotterdam, Netherlands
• ⁷Department of Radiology and Nuclear Medicine, Erasmus MC - University Medical Center, Rotterdam, Netherlands
• ⁸Department of Neurology, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
• ⁹Department of Radiology, Cardiovascular Research Institute Maastricht, Maastricht University Medical Center, Maastricht, Netherlands
• ¹⁰Department of Radiology, University Medical Centre, Utrecht, Netherlands
• ¹¹Department of Radiology, Leiden University Medical Center, Leiden, Netherlands

Background: Although endovascular treatment (EVT) has greatly improved outcomes in acute ischemic stroke, still one third of patients die or remain severely disabled after stroke. If we could select patients with poor clinical outcome despite EVT, we could prevent futile treatment, avoid treatment complications, and further improve stroke care.(All these are forms of giving up on treating stroke patients, which is why I would have them fired.) We aimed to determine the accuracy of poor functional outcome prediction, defined as 90-day modified Rankin Scale (mRS) score ≥5, despite EVT treatment.

Methods: We included 1,526 patients from the MR CLEAN Registry, a prospective, observational, multicenter registry of ischemic stroke patients treated with EVT. We developed machine learning prediction models using all variables available at baseline before treatment. We optimized the models for both maximizing the area under the curve (AUC), reducing the number of false positives.

Results: From 1,526 patients included, 480 (31%) of patients showed poor outcome. The highest AUC was 0.81 for random forest. The highest area under the precision recall curve was 0.69 for the support vector machine. The highest achieved specificity was 95% with a sensitivity of 34% for neural networks, indicating that all models contained false positives in their predictions. From 921 mRS 0–4 patients, 27–61 (3–6%) were incorrectly classified as poor outcome. From 480 poor outcome patients in the registry, 99–163 (21–34%) were correctly identified by the models.

Conclusions: All prediction models showed a high AUC. The best-performing models correctly identified 34% of the poor outcome patients at a cost of misclassifying 4% of non-poor outcome patients. Further studies are necessary to determine whether these accuracies are reproducible before implementation in clinical practice.

Introduction

Over the past 4 years, endovascular thrombectomy (EVT) unquestionably proved its value in anterior circulation acute ischemic stroke (1, 14–20). Despite the encouraging results, however, still ~30% of patients die or remain dependent of daily nursing care after EVT, making their treatment benefit essentially minimal (17, 18).

If we could reliably select patients with poor outcome after stroke despite EVT, we could spare patients a futile treatment with a needless risk of complications and enable a more efficient use of resources (21). Unfortunately, so far, no studies have been able to definitively identify a subgroup of patients that should not be treated with EVT (21).

In patient selection, it could be useful to predict poor outcome. Many previous studies focused on predicting functional independence after EVT (22). However, the use of such models would raise an ethical question. If a model predicts a zero percent chance of functional independence with EVT for a patient, one might advise to not treat. Untreated, the patient likely has a worse outcome, possibly needing continuous care in a nursing home. Treated, the patient may be able to function with some assistance in daily activities. Should we not treat this patient? A more valuable argument could be a reliable prediction of death or complete dependence of continuous care, even after EVT.

Some studies, such as MR PREDICTS, used data from randomized trials to predict treatment benefit as a modified Rankin Scale (mRS) score shift, using ordinal logistic regression (13). Predicting treatment benefit can be useful: if a patient is predicted to benefit from EVT in addition to regular care, one would proceed with EVT. However, data from randomized trials are necessary for such a model because predicted outcomes need to be based on a sufficient number of patients who did or did not receive EVT without indication bias. The amount of available data from randomized trials on EVT is limited. No new data after the HERMES trials will be available to train and validate models (17). An outcome measure that can enable long-term model improvement such as poor functional outcome could be of added value to models predicting treatment benefit.

Only a few studies have used poor outcome as their outcome measure; however, they had a limited amount of data and focused on linear classifiers (23). Machine learning (ML) may be of added value in predicting outcome after EVT. The number of relevant prognostic factors in stroke patients is high, and their effects on outcome may be indirect, combined, or otherwise complicated. With the ability to identify relevant prognostic variables through linear and non-linear relationships, ML may have added value in poor outcome prediction.

ML belongs to the artificial intelligence domain, where algorithms are designed to automatically learn patterns from data. In the work by Van Os et al. (22), ML methods predicted functional independence after acute ischemic stroke in a large population (1,383 patients), with reasonable certainty [area under the curve (AUC) 0.79].

Since the addition of EVT to standard care, the amount of available outcome data has greatly increased, now allowing for more powerful and elaborate prediction modeling. In the current study, we aim to assess the accuracy of pre-procedural prediction of poor functional outcome after EVT using ML models in patients from the MR CLEAN Registry.