The Art and Science of Stroke Outcome Prognostication

There is literally no science involved, no objective damage diagnosis. All guesswork like this:

Mercury astronaut Scott Carpenter suffers stroke; full recovery expected

Oops! 

 Scott Carpenter - Obituary

 

 

The Art and Science of Stroke Outcome Prognostication

Meah MingYang Gao

,

Jeffrey Wang

,

Gustavo Saposnik

Originally published25 Mar 2020https://doi.org/10.1161/STROKEAHA.120.028980Stroke. 2020;51:1358–1360

• This article is a commentary on the following

See related article, p 1477

Tell me and I forget. Teach me and I remember. Involve me and I learn.

—Benjamin Franklin (1706–1790), statesman, scientist, and inventor¹

As stroke clinicians, we are routinely asked by patients and families regarding prognosis. The cognitive process underlying prognostication is complex, poorly understood, and thus can be subjected to cognitive biases colored by our personal experiences. Previous studies suggest that clinicians, even those with expertise in stroke, perform poorly in predicting clinical outcomes. For example, the JURaSSiC study (Clinician Judgment vs Risk Score to Predict Stroke Outcomes) reported that clinicians’ overall accuracy for predicting death or disability at discharge was a staggeringly low (16.9%), and none of the 111 participating clinicians with expertise in stroke care correctly predicted all 5 case outcomes.^1a Similarly, Ntaios et al² found that >50% of all estimates made by physicians with an interest in stroke care were inaccurate, and their predictions became even less precise in those patients who received thrombolytic therapy. Interestingly, meteorologists provide more accurate predictions for rain (r: 95%)³ and poker players for chances of winning a hand (r: 96%).⁴ Contrarily, patients and families rely on our lower rated predictions to make critical decisions. As a result, evidence-based prognostic models are necessary to better inform clinicians.

To address this need, several prognostic models have been developed to aid prognostication after ischemic stroke (Table). Comparative studies have shown that these prognostic models outperform clinician judgment in predicting stroke outcomes.^1,2 These prognostic scores were derived from rigorous mathematical modeling based on data from large cohorts of acute stroke patients. While there have been some external validation studies, they were largely conducted in a Western population and in patients recruited before the widespread utilization of reperfusion therapies.⁵ Furthermore, new advances are occurring at an unprecedented pace across the continuum of stroke care, from the development of neuroprotective agents, application of extended window reperfusion therapies, to modern stroke rehabilitation technologies. These prognostic models require regular validation and calibration using updated data to retain their usefulness.

Table. Comparison of 6 Stroke Prognostic Scores as Tested by Matsumoto et al⁶

Prognostic Score	Applicable Population	Variables	Outcomes	Patient-Centered Outcomes Included?
PLAN7	Patients treated with intravenous thrombolysis were excluded	Age, level of consciousness, arm weakness, leg weakness, neglect or aphasia, history of atrial fibrillation, history of congestive heart failure, cancer, prestroke functioning	mRS score of 5–6 and death at 30 d and 1 y	No
IScore8	Derived from an ischemic stroke cohort. Validated in NINDS tPA trials, VISTA	3 mo: age, sex, stroke severity (CNS), stroke subtype (TOAST), history of atrial fibrillation, history of congestive heart failure, cancer, renal dialysis, prestroke functioning, acute glucose	mRS score of 3–6 and death at 30 d and 1 y	No
		1 y: in addition, history of myocardial infarction, smoking status
ASTRAL9	Patients with prestroke dependency were excluded	Age, stroke severity (NIHSS), level of consciousness, presence of visual field defect, symptoms onset to treatment time, acute glucose	mRS score of 3–6 and death at 3 mo	No
HIAT10	Derived from patients treated with intra-arterial thrombolysis	Age, stroke severity (NIHSS), acute glucose	mRS score of 4–6 at discharge	No
THRIVE11	Derived from patients treated with endovascular therapy. Validated also in patients treated with intravenous thrombolysis or no acute treatment	Age, stroke severity (NIHSS), history of atrial fibrillation, history of hypertension, diabetes mellitus	Functional outcome (mRS, 0–2 vs 3–6) and mortality at 3 mo	No
SPAN-10012	Derived from patients treated with intravenous thrombolysis	Age, stroke severity (NIHSS)	Composite favorable outcome (mRS, 0–1; NIHSS, ≤1; Barthel index, ≥95; and Glasgow Outcome Scale score, 1), catastrophic outcome (mRS, 4–6) and death at 3 mo	No

ASTRAL indicates Acute Stroke Registry and Analysis of Lausanne; CNS, Canadian Neurological Scale; HIAT, Houston Intra-Arterial Recanalization Therapy; Iscore, Ischemic Stroke Predictive Risk Score; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; NINDS, National Institute of Neurological Disorders and Stroke; PLAN, Preadmission Comorbidities, Level of Consciousness, Age, and Neurological Deficit; SPAN, Stroke Prognostication Using Age and National Institutes of Health Stroke Scale; THRIVE, Totaled Health Risks in Vascular Events; TOAST, Trial of ORG 10172 in Acute Stroke Treatment; tPA, tissue-type plasminogen activator; and VISTA, Virtual International Stroke Trials Archive.

In the current issue, Matsumoto et al¹³ assessed the performance of 6 stroke prognostic scores in 4237 acute ischemic stroke patients hospitalized at a Japanese stroke center between 2012 and 2017. This study adds value to existing literature given its large sample size and relatively modern enrollment period allowing patients who received reperfusion therapies to be included in the cohort. Authors directly compared the performances of 6 point-based stroke prognostic scores against each other and showed that they all performed reasonably well in this real-world population; areas under the receiver operating characteristic curve ranged from 0.69 (Houston Intra-Arterial Recanalization Therapy [HIAT] score) to 0.92 (Preadmission Comorbidities, Level of Consciousness, Age, and Neurological Deficit [PLAN] score) in predicting poor functional outcomes and 0.87 (PLAN) to 0.88 (Ischemic Stroke Predictive Risk Score [Iscore] and Acute Stroke Registry and Analysis of Lausanne [ASTRAL] score) in predicting in-hospital mortality. This current study also explored the promising field of machine learning in the current era of big data. In all data-driven modeling methods, the National Institutes of Health Stroke Scale score, preadmission modified Rankin Scale (mRS), and age emerged as the top 3 factors predicting functional outcomes, validating what clinicians have long known intuitively to be important clinical prognostic indicators.

However, the findings by Matsumoto et al may not generalize readily to other stroke populations. The participants were all treated in a single center in Japan, and only 1.5% underwent endovascular therapy. Nonetheless, this study provides valuable external validation of several prognostic scores in an East Asian cohort. Furthermore, stroke patients who do not undergo reperfusion therapy remain the vast majority globally. Another important caveat to consider is that the functional outcomes in this study were recorded at the time of discharge, while original publications of the ASTRAL, Totaled Health Risks in Vascular Events (THRIVE), and Stroke Prognostication Using Age and National Institutes of Health Stroke Scale (SPAN)-100 scores used 90-day outcomes. This discrepancy may underlie the relatively lower performance of these three scores in this cohort as compared with PLAN and IScore, both of which were originally derived using functional data at discharge.

There is a growing body of literature examining the utility of machine learning for both diagnosis and prognosis in a multitude of medical subspecialties.¹⁴ In the present study, the authors assessed the utility of newer machine learning algorithms in prognostication for ischemic stroke. Unsurprisingly, these models performed well in predicting poor functional outcome at time of discharge. Specifically, the 2 ensemble decision tree models tested outperformed the more traditional logistic regression models for predicting mRS scores of 3 to 6 and 4 to 6, allowing improved precision at the midpoint of the functional outcome spectrum. However, decision tree models were limited by the problem of overfitting, which occurs with low-frequency outcomes (eg, in-hospital mortality) or smaller sample sizes. Furthermore, the 5 machine learning algorithms tested in this study offered at best a marginally higher degree of accuracy over the highest performing clinical prognostic scores. Overall, while stroke prognosis using machine learning is an exciting concept, more work is needed to fully realize its predictive potential and support clinical application.⁶

Another question remains: if prognostic scores for ischemic stroke have been externally validated, why are they not being used routinely in clinical practice? Are we still "eyeballing" when asked about the prognosis of our stroke patients? There are many potential reasons. Is it a matter of overconfidence in our ability to prognosticate based on clinical experience alone? Or are most frontline clinicians simply not aware of the existence of these scores? A higher number of variables not readily available without detailed chart review make routine use of a predictive model cumbersome. Lastly, perhaps an easy-to-measure outcome such as the mRS, though well suited for testing therapeutic interventions, falls short when it comes to the complex task of prognostication after stroke. Each individual patient we encounter on the stroke unit may have a different idea of what is important to his or her quality of life, and, therefore, prediction of specific patient-centered outcomes, such as swallowing or social participation, is likely more meaningful than dichotomized mRS ranges of 0 to 2 versus 3 to 6.¹⁵ Five of the 6 scores cited by Matsumoto et al were validated for prediction of a dichotomized mRS range or death, and the remaining score (SPAN-100) used a dichotomized composite of mRS, National Institutes of Health Stroke Scale, Barthel Index, and Glasgow Outcome Scale (Table).

Efforts should be made to survey stroke clinicians on why they are not routinely using externally validated prognostic models. Improving both the accuracy and meaningfulness of prognostication scores may foster better communication and lead to improved shared clinical decision-making. Ultimately, predictive models, even those derived from complex data-driven algorithms, are incorrect in some patients. They should not be the sole foundation for clinical decision-making in practice and certainly cannot replace a stroke physician’s comprehensive assessment and clinical acumen.¹⁶ After all, stroke is a complex condition occurring in heterogeneous populations with variety of mechanisms, each having a different prognosis. True stroke expertise lies in not only learning the science of evidence-based models but more importantly, mastering the art of how to apply the available evidence at bedside. Our mandate is to overcome those challenges and ameliorate the existing knowledge-to-action gaps in acute stroke care.

Sources of Funding

Dr Saposnik is supported by the Heart and Stroke Foundation of Canada Career Award following an open peer-review competition.

Disclosures

Dr Saposnik is the Associate Editor for the Emerging Therapies Section at Stroke journal. He is supported by the Heart and Stroke Foundation of Canada Career Award following an open peer-reviewed competition and received research grants from Roche and speaker honoraria from Servier and Celgene. The other authors report no conflicts.

Footnotes

*Drs Gao and Wang contributed equally.