You BLITHERING IDIOTS are predicting failure to recover. In what multiverse do you live where that helps survivors recover? I'm sure your definition of 'good outcome' is vastly different that a survivors. 100% recovery is a good outcome to a survivor. NOTHING LESS!
Alexander C Flinthttps://orcid.org/0000-0002-3721-26941, Sheila L Chan1, Nancy J Edwardshttps://orcid.org/0000-0003-4440-59341, Vivek A Rao1, Jeffrey G Klingman2, Mai N Nguyen-Huynh2, Bernard Yan3, Peter J Mitchell4, Stephen M. Davis3, Bruce CV Campbellhttps://orcid.org/0000-0003-3632-94333, Diederik W Dippelhttps://orcid.org/0000-0002-9234-35155, Yvo BWEM Roos6, Wim H van Zwam7, Jeffrey L Saver8, Chelsea S Kidwell9, Michael D Hillhttps://orcid.org/0000-0002-6269-154310, Mayank Goyal10, Andrew M Demchuk10, Serge Bracard11, Martin Bendszus12, Geoffrey A Donnan3, and on behalf of the VISTA-Endovascular Collaboration*
Introduction:
The THRIVE score and the THRIVE-c calculation are validated ischemic stroke outcome prediction tools based on patient variables that are readily available at initial presentation. Randomized controlled trials (RCTs) have demonstrated the benefit of endovascular treatment (EVT) for many patients with large vessel occlusion (LVO), and pooled data from these trials allow for adaptation of the THRIVE-c calculation for use in shared clinical decision making regarding EVT.
Methods:
To extend THRIVE-c for use in the context of EVT, we extracted data from the Virtual International Stroke Trials Archive (VISTA) from 7 RCTs of EVT. Models were built in a randomly selected development cohort using logistic regression that included the predictors from THRIVE-c: age, NIH Stroke Scale (NIHSS) score, presence of hypertension, diabetes mellitus, and/or atrial fibrillation, as well as randomization to EVT and, where available, the Alberta Stroke Program Early CT Score (ASPECTS).
Results:
Good outcome (Your good outcome is not valid if the survivor doesn't think it was a good outcome. Survivors take precedence.)was achieved in 366/787 (46.5%) of subjects randomized to EVT and in 236/795 (29.7%) of subjects randomized to control (P < 0.001), and the improvement in outcome with EVT was seen across age, NIHSS, and THRIVE-c good outcome prediction. Models to predict outcome using THRIVE elements (age, NIHSS, and comorbidities) together with EVT, with or without ASPECTS, had similar performance by ROC analysis in the development and validation cohorts (THRIVE-EVT ROC area under the curve (AUC) = 0.716 in development, 0.727 in validation, P = 0.30; THRIVE-EVT + ASPECTS ROC AUC = 0.718 in development, 0.735 in validation, P = 0.12).
Conclusion:
THRIVE-EVT may be used alongside the original THRIVE-c calculation to improve outcome probability estimation for patients with acute ischemic stroke, including patients with or without LVO, and to model the potential improvement in outcomes with EVT for an individual patient based on variables that are available at initial presentation. Online calculators for THRIVE-c estimation are available at www.thrivescore.org and www.mdcalc.com/thrive-score-for-stroke-outcome.
Keywords
Acute stroke therapy, outcome prediction, THRIVE score, ASPECTS, endovascular therapy, shared decision-making
1Division of Research and Department of Neuroscience, Kaiser Permanente, Redwood City, CA, USA
2Department of Neurology, Kaiser Permanente, Walnut Creek, CA, USA
3Melbourne Brain Centre at Royal Melbourne Hospital, The University of Melbourne, Parkville, VIC, Australia
4Department of Radiology, The University of Melbourne, The Royal Melbourne Hospital, Parkville, VIC, Australia
5Department of Neurology, Erasmus Medical Center, Rotterdam, The Netherlands
6Department of Neurology, Amsterdam University Medical Center, Amsterdam, The Netherlands
7Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, Maastricht, The Netherlands
8Department of Neurology and Comprehensive Stroke Center, University of California, Los Angeles, Los Angeles, CA, USA
9Department of Neurology, The University of Arizona, Tucson, AZ, USA
10Department of Clinical Neurosciences, University of Calgary, Calgary, AB, Canada
11Department of Neuroradiology, University of Lorraine, Nancy, France
12Department of Neuroradiology, University of Heidelberg, Heidelberg, Germany
Corresponding author(s):
Alexander C Flint, Division of Research and Department of Neuroscience, Kaiser Permanente, 1150 Veterans Boulevard, Redwood City, CA 94025, USA. Email: alexander.c.flint@kp.org; @neuroicudoc
*
VISTA-Endovascular Steering Committee listed in Appendix A.
Introduction
The THRIVE score is an extensively validated ischemic stroke outcome prediction tool based on clinical variables easily obtained on acute stroke presentation.1,2 While the THRIVE score uses trichotomized NIHSS and patient age data in combination with the chronic disease scale to compute a score that corresponds to broad outcome categories, the THRIVE-c calculation uses continuous age and NIHSS in the logistic equation to estimate outcome more precisely.2 Among patients with LVO, the extent to which THRIVE-c outcome prediction would be impacted by EVT status remains unknown.
RCTs of second generation mechanical thrombectomy devices such as stent retrievers have demonstrated the overall efficacy of EVT3,4 but many patients do poorly despite receiving intervention.3,5 Outcome prediction models developed in the subsequent era of widespread EVT with variables available prior to treatment are based on small and often single-center cohorts6,7 or require a large number of variable inputs into a machine learning model that limits utility in acute decision making.6,8
Here, we use data from VISTA from 7 RCTs of EVT to develop tools for outcome prediction with or without EVT: the THRIVE-EVT calculations. The resulting tools are based on straightforward patient variables easily determined at initial presentation, allowing outcome prediction results to be used in shared clinical decision making about EVT.
Methods
Data source, subjects, and measurements
Data were obtained in anonymized form from VISTA http://www.virtualtrialsarchives.org/vista-endovascular/, pooled from 7 RCTs of EVT: MR CLEAN,9 SWIFT-PRIME,10 ESCAPE,11 EXTEND IA,12 MR RESCUE,13 THRILL,14 and THRACE.15 Full trial names are listed in Appendix B. Per VISTA policy, source RCT was deidentified. Data were available for all subjects on age, sex, initial NIHSS score, medical comorbidities and history as shown in Table 1, intravenous Alteplase administration, randomization to EVT or control, and clinical outcome on the mRS at 90 days post stroke. ASPECTS on the initial noncontrast head computed tomogram (CT) was available for 1484/1582 subjects (93.8%).
The THRIVE score is calculated from trichotomized age and NIHSS, and presence of HTN, DM, or AF.1 THRIVE-c is based on a logistic equation including age, NIHSS, and dummy variables encoding the comorbidities HTN, DM, and AF.2
For development and validation of two new logistic equations in patients who underwent randomization to EVT or control, we fit two multivariable logistic regression models. The first model included continuous age, continuous NIHSS, presence of HTN, DM, and AF (dummy variables with natural coding for chronic disease scale levels of 1, 2, and 3), and randomization to EVT or control. The second model included these same multivariable predictors and also included ASPECTS; this second model was fit in the subset of the cohort for whom ASPECTS was available (93.8% of the total). ASPECTS was adjudicated by the radiology core labs of the RCTs.
Analysis
Multivariable logistic regression was performed using standard techniques with direct entry of all predictors. For the THRIVE-EVT calculations, model-predicted probabilities were estimated using the logistic equation. Receiver-Operator Characteristics curve (ROC) analysis of the two new THRIVE-c models was performed using standard post-estimation techniques. Statistical comparisons of ROC curve area under the curve (AUC, C-statistic) were performed using a two-tailed Chi-square test as previously described.2 Development of the predictive models was performed in a randomly selected subset (n = 1107) representing approximately 70% of the total cohort, and separate validation was performed in the remaining subset (n = 475) representing approximately 30% of the total cohort. Model improvement by addition of EST and ASPECTS was assessed using integrated discrimination improvement (IDI).16 Bivariate analyses comparing subjects in two groups were performed with the Fisher’s exact test for categorical variables and the nonparametric Kruskal–Wallis equality-of-populations rank test for continuous data. All statistical analyses were performed using Stata MP version 16.1 (Stata Corp., College Station, TX).
Results
Patient characteristics for the total cohort of 1582 subjects from 7 RCTs of EVT are shown in Table 1, broken down according to our randomly selected development (n = 1107) and validation (n = 475) cohorts. No statistically significant differences in patient characteristics were found between the development and validation cohorts (Table 1).
Randomization to EVT was, as expected from the positive results of the underlying RCTs, associated with improved clinical outcomes. In the overall cohort, good outcome (mRS 0-2 at 90 days) was achieved in 366/787 (46.5%) of subjects randomized to EVT and in 236/795 (29.7%) of subjects randomized to control (P < 0.001). In multivariable logistic regression of good outcome in the overall cohort with the predictors age, NIHSS, chronic disease scale (dummy-encoded), and randomization to EVT, the odds ratio for EVT randomization prediction of good outcome was 2.24 (95% CI: 1.80-2.81, P < 0.001).
The relative impact of randomization to EVT was not selective for particular ranges of age, NIHSS, or the overall estimation of outcome by the parent THRIVE-c calculation. Using margin estimation of outcome from the above multivariable logistic model including the predictors age, NIHSS, chronic disease scale, and EVT, randomization to EVT improved the chance of good outcome across the range of ages (Figure 1(a)) and across the range of NIHSS scores (Figure 1(b)) encountered in the overall cohort. Using a multivariable logistic model of calculated THRIVE-c probability and EVT in the overall cohort, randomization to EVT similarly improved good outcome probability across the range of THRIVE-c in margin estimation (Figure 1(c)). In logistic regression modeling good outcome, there was no evidence for an interaction between randomization to EVT and THRIVE Score (Supplemental Table 1). Similarly, addition of IV Alteplase administration prior to EVT did not alter the relationship between EVT and outcome or THRIVE Score and outcome (Supplemental Table 1).
Figure 1. Effects of varying age, NIHSS, and THRIVE-c prediction on good outcome, according to randomization to endovascular stroke treatment (EVT) or control (No EVT): (a) Impact of varying age on good outcome (mRS 0-2 at 90 days), according to randomization to EVT or control (No EVT). Solid curves mark the margins estimates for age from a logistic model of good outcome using the predictors age, NIHSS, chronic disease scale, and EVT randomization. Thin dotted lines mark the 95% confidence intervals for the estimates. (b) Impact of varying NIHSS on good outcome, according to EVT randomization. The margins estimates for NIHSS are from the same logistic model used in (a). Relationship between THRIVE-c good outcome probability, calculated using the original THRIVE-c equation, and good outcome, according to EVT randomization. The margins estimates for THRIVE-c are from a logistic model with the predictors THRIVE-c probability and EVT randomization.
We built two outcome prediction equations in the development cohort, with separate validation by ROC curve comparison in the validation cohort. For the first equation, we combined the original THRIVE-c elements of age, NIHSS, and chronic disease scale with randomization to EVT (0/1) (the THRIVE-EVT model). ROC curve comparison showed similar model performance in the development cohort (ROC area under the curve (AUC) = 0.716) and validation cohort (ROC AUC = 0.727), and there was no significant difference between the ROC curves (Chi-square P = 0.30) (Figure 2(a)). There was good model calibration in both cohorts (Hosmer-Lemeshow goodness of fit test: P = 0.60 for development cohort, P = 0.51 for validation cohort). For the second equation, we used the same predictive elements of the basic THRIVE-EVT model (age, NIHSS, chronic disease scale, and EVT) and also included ASPECTS (0-10), developing and validating this second model in the subset of subjects for whom ASPECTS was recorded (93.8% of the total cohort). ROC curve comparison showed similar model performance in the development cohort (ROC area under the curve (AUC) = 0.718) and validation cohort (ROC AUC = 0.735), and there was no significant difference between the ROC curves (Chi-square P = 0.12) (Figure 2(b)). There was good model calibration in both cohorts (Hosmer-Lemeshow goodness of fit test: P = 0.49 for development cohort, P = 0.70 for validation cohort). There was significant improvement in THRIVE model classification on the integrated discrimination improvement (IDI) statistic by the addition of either EVT alone or by the addition of both EVT and ASPECTS (Supplemental Table 2).
Figure 2. Calibration of the THRIVE-EVT and THRIVE-EVT (+ ASPECTS) calculations in the development and validation cohorts: (a) Receiver-Operator Characteristics (ROC) curves for the performance of the THRIVE-EVT multivariable logistic regression model in the development cohort (n = 1107) and the validation cohort (n = 475). Area under the ROC curve (AUC) in the development cohort (0.716) was not significantly different from the AUC in the validation cohort (0.727) (P = 0.30). (b) ROC curves for the performance of the THRIVE-EVT (+ ASPECTS) multivariable logistic regression model in the development cohort (n = 1036) and the validation cohort (n = 448). ROC curve AUC in the development cohort (0.718) was not significantly different from the AUC in the validation cohort (0.735) (P = 0.12). P values for each comparison are from the Chi-square test.
From the multivariable logistic regression models fit using data from the development cohort, the two predictive calculations were determined as shown in Figure 3(a). Figure 3(b) shows a worked example of the THRIVE-EVT calculation without ASPECTS for an 82 year-old LVO patient with a NIHSS score of 16 and a history of hypertension and diabetes mellitus but no history of atrial fibrillation, including outcome estimation based on whether EVT is performed or not.
Figure 3. Logistic equation estimation of outcome probability: (a) Standard form of the logistic equation, with detailed variables and coefficients for x shown for the THRIVE-EVT calculation and the THRIVE-EVT calculation with ASPECTS. CDS1-3 represent dummy variables encoding the state of the sum of the presence of HTN, DM, or AF. (b). Worked example of the THRIVE-EVT calculation (without ASPECTS) for a 82- year-old patient with NIHSS of 16, HTN, DM, but no AF. Estimated outcome probabilities are shown for this patient with or without EVT.
Using these two new calculations alongside the original THRIVE-c calculation, probability of good outcome may be estimated for patients with acute ischemic stroke, including patients without LVO (using the THRIVE-c calculation) and patients with LVO (using the THRIVE-EVT calculation, with or without ASPECTS). Online calculators for THRIVE-c and THRIVE-EVT estimation are available at www.thrivescore.org and www.mdcalc.com/thrive-score-for-stroke-outcome.
Discussion
We have developed and validated extensions to the THRIVE-c calculation to serve as a tool to estimate the potential benefit of EVT in individuals with large vessel occlusion, using patient data that are readily available at initial presentation.
In this study, we developed the THRIVE-EVT models using contemporary data from endovascular trials. These models improve outcome prediction in patients with LVO and allow for a quantitative estimation of the impact of EVT in an individualized clinical context. The ROC curve AUCs for THRIVE-EVT (with or without ASPECTS) are comparable to those of previously reported outcome prediction models in acute ischemic stroke, including complex models generated via machine learning.6–8,17–22 The addition of ASPECTS, when available, appears to increase predictive accuracy as evidenced by a higher AUC.
The outcome models generated here have several strengths. Our derivation cohort (1107 patients) is one of the largest cohorts studied in this context. In their machine learning prediction models, Ramos et al included 1526 patients from the MR CLEAN registry, but the MR CLEAN registry is of patients exclusively treated with EVT in the Netherlands, without control subjects.8 The MR PREDICT model was derived from the MR CLEAN trial,21 and validated in the HERMES collaboration and MR CLEAN registry.23 While MR PREDICT is a robust and well-validated prediction tool in this context, it does require 11 inputs to calculate, including some which may not be readily available or known at the time of clinical decision making.21,23 Many other models examining outcome in the setting of EVT did so with the use of treated patients only, and thus could not show the relative benefit of intervention in individual patients.7,18,22 Our cohort, derived from VISTA-Endovascular, is of substantial diversity, includes data from both EVT-treated and untreated patients, and can be determined from a small number of inputs that are known at the time of initial presentation.
Our results show that treatment of eligible patients with EVT results in improved outcome independent of age, NIHSS, and original THRIVE calculation. We do not identify a particular subgroup where endovascular therapy would be definitively futile, with a zero probability of improved outcome. The THRIVE-EVT calculations are not intended to replace clinician judgment, nor should they be used by clinicians to unilaterally exclude patients who are otherwise candidates for EVT based on inclusion / exclusion criteria from the original RCTs. Instead, THRIVE-EVT should be used as aid to shared clinical decision-making. Whether or not to pursue EVT for a particular individual can be a complex decision made under emergent conditions, often with input from patient surrogates rather than the patient themselves, and can require emergency transfer of a patient to a specialized center. Our THRIVE-EVT calculations may help practitioners set reasonable expectations for patients or their families, particularly when EVT is pursued in patients predicted to have a low probability of good outcome.
Certain limitations of our study should be addressed. First are the challenges of using models to predict outcomes at the individual patient level. Though the THRIVE-EVT calculations perform similarly to other predictive models frequently used in clinical practice (such as CHA2DS2-VASc24 and the previously described models predicting outcome specifically in LVO patients), an AUC in the 0.7 to 0.8 range represents moderate discriminative ability. Although our models include numerous baseline patient variables—and the extensions constructed here add additional variables—all of these variables are nonmodifiable. We did not evaluate potentially modifiable variables such as transport time or time to groin puncture. Factors such as collateral status and operator factors were not recorded in the RCTs on which the present study is based, and thus these data are not available for our analysis. The ASPECTS in the subjects included in the RCTs was toward the higher end of the range (interquartile range 7-10), with a lesser degree of early ischemic change, and thus the THRIVE-EVT including ASPECTS may have less certain predictive power in patients with very low ASPECTS. We used the standard good outcome definition of mRS 0-2, and for some patients and families, mRS 3 might be considered a favorable prognosis. Finally, addressing the utility of predictive tools in subsets of patients with pre-existing conditions, use of oral anticoagulation, and stratifying such tools according to EVT at the primary center vs transport to a secondary center will require future research.
In conclusion, the THRIVE-EVT calculations are validated tools to assist clinicians, patients, and families in shared clinical decision-making about EVT in patients with LVO.
