Introduction
Endovascular treatment (EVT) became the standard therapy
for anterior circulation large vessel occlusion stroke (LVOS) after
landmark clinical trials were published in 2015 (1–5).
Previous studies have indicated that a higher procedure volume is
associated with a higher proportion of reperfusion and favorable
clinical outcomes (6–8).
However, in the real world, it is difficult for high-volume hospitals
to provide access to this time-dependent therapy to every eligible
patient because of the geographic maldistribution of
neurointerventionalists. An increasing prevalence of EVT performed by
less-experienced operators has been seen in hospitals in recent years (9).
To ensure adequate procedural quality, describing the learning curve
for EVT and establishing a caseload threshold for operators are urgently
needed. Therefore, based on a prospective database, we investigated the
relationship between operator experience and the outcome of EVT and
further identified the specific number of cases needed to acquire
technical success.
Materials and Methods
Patient Selection
We retrospectively analyzed the clinical data of all
consecutive patients with LVOS who underwent EVT by seven neurologists
at a single center from January 2016 to September 2019. Each neurologist
was accredited to perform carotid and intracranial stenting by the
national accrediting institution before 2013. We performed the first EVT
case in the year 2016 in our center; however, before that time, all
seven neurologists had completed a 1-year additional fellowship of
endovascular mechanical thrombectomy. In general, patients with acute
ischemic stroke were enrolled in this study if they fulfilled the
following criteria: (1) age ≥18 years, (2) anterior circulation vessel
occlusion confirmed by radiographic imaging, (3) baseline National
Institutes of Health Stroke Scale (NIHSS) score ≥6, (4) premorbid
modified Rankin Scale (mRS) score of <2, and (5) time from symptom
onset to puncture time <24 h. The study excluded patients treated
with posterior circulation LVOS.
Procedure for Endovascular Treatment
The EVT procedures included the use of stent retrievers,
aspiration, angioplasty, stenting, or combinations of these approaches.
If reocclusion occurred after thrombectomy, the rescue therapy,
including angioplasty, stenting, intra-arterial thrombolysis, and
glycoprotein IIb/IIIa inhibitor infusion, were allowed. The choice of
treatment option, material used for EVT, and type of anesthesia was left
to the discretion of the operators. In our institution, the Solitaire
stent (Covidien), was the only stent retriever available. Furthermore,
balloon-guide catheters were unavailable. The blood pressure after the
procedurals was kept <130/80 mmHg if recanalization was achieved;
otherwise, the blood pressure was controlled under 180/100 mmHg. All
patients were monitored over at least a 24-h period after the
procedures.
All procedures were performed during and off hours with
the same staff (i.e., including a primary operator, an assistant at the
operation table, an experienced nurse, and a radiology technician).
Data Collection and Outcome Evaluation
Fourteen patient-related, seven treatment-related, and
two operator-related. Patient-related variables included age, sex, NIHSS
score, hypertension, diabetes mellitus, dyslipidemia, atrial
fibrillation, smoking, stroke cause (large artery atherosclerosis,
cardioembolism, and dissection/undetermined cause), baseline Alberta
Stroke Program Early CT score (ASPECTS) (10)
affected hemisphere, occlusion site, including intracranial internal
carotid artery (ICA) or M1/proximal M2 segment of the middle cerebral
artery (MCA) or anterior cerebral artery occlusion, tandem lesion
(defined as carotid stenosis ≥90% or complete occlusion confirmed by
radiographic imaging) and collateral score with a 4-point scale (0, no
filling of the occluded territory; 1, >0% and ≤ 50% filling of the
occluded territory; 2, >50% and <100% filling of the occluded
territory; 3, 100% filling of the occluded territory) (11) and was dichotomized as poor (0–1) or good collaterals (2–3).
Treatment-related variables included onset-to-puncture
time (OPT), door-to-puncture time (DPT), intravenous thrombolysis,
intracranial occlusion treatment option (stent retrieval,
angioplasty/intracranial stent, aspiration), type of anesthesia,
cervical carotid artery stenting, and additional rescue therapy.
Operator-related variables included operators and
procedural experience. Procedural experience was defined as the number
of cases performed by each individual operator (i.e., operator's 1st,
2nd, 3rd, … or nth case). In the case of two operators with different
levels of case experience performing a procedure together, the case was
categorized based on the number of cases of the more experienced
operator. Procedural experience was graded as three levels (1–20 cases,
21–40 cases, and >40 cases) for univariate analysis.
The EVT prognostic outcome was functional independence (defined as mRS score of 0–2 at 90 days) and mortality within 90 days.
The primary EVT technical outcome was successful
reperfusion (SR), which was defined as the achievement of a modified
Thrombolysis in Cerebral Infarction score (mTICI) of 2b-3, as determined
according to the final angiogram after the procedure. The secondary
technical outcome was puncture-to-reperfusion time (PRT), which was
defined as the time interval from puncture to final SR or to abortion of
the procedure if SR was not achieved.
The safety outcome was symptomatic intracerebral
hemorrhage (SICH), which was defined as any parenchymal hematoma,
subarachnoid hemorrhage, or intraventricular hemorrhage associated with a
worsening of the NIHSS score by ≥4 points within 36 h after the
procedure in this study.
All radiological imaging data, including occlusion sites,
ASPECTS, type of hemorrhagic transformation, collateral score, and
mTICI post-procedure were sent to the core laboratory in Jingling
Hospital and were evaluated in a blind manner by two physicians (LL, X
and P, W). Any disagreement was resolved by consensus.
Statistical Analysis
Continuous variables are presented as the median
(interquartile range, IQR), and categorical variables are presented as
the frequency (percentage). Univariate analysis was compared with
Kruskal-Wallis test or Mann-Whitney U test for continuous
variables and chi-square or Fisher's exact test for categorical
variables. Covariates associated with PRT were analyzed using bivariate
correlation analysis. Multivariable backward regression analyses were
used to investigate the association between procedural experience (as a
continuous variable) and functional independence (logistic), 90-days
mortality (logistic), SR (logistic), and PRT (linear) adjusting
covariates with P < 0.10 in univariate analyses. Relationships
between procedural experience and outcomes were plotted as curves based
on multivariable backward regression models with the mean values of
other significant covariates.
To identify the specific number of cases needed to
acquire SR, the risk-adjusted cumulative sum chart (RA-CUSUM) was
plotted; another logistic backward model was constructed for SR itself,
regardless of procedural experience, only using covariates with P < 0.10 in univariate analyses. The details of the RA-CUSUM approach are provided in the online-only Supplemental Material, and this approach has been described in a previous study (12).
In brief, the RA-CUSUM chart demonstrated the difference between
cumulative predicted and observed events, with the X-axis representing
the number of cases the operator had performed. In the inexperienced
phase, the cumulative predicted SR was greater than the observed SR, as
demonstrated by the ascending graph. In the plateau phase, the
cumulative predicted SR was comparable to the observed SR, as shown by
the leveling off. After achieving enough experience, the cumulative
predicted SR was less than the observed SR, as displayed by the
descending graph. Therefore, there ought to be a turning point, which
could represent the specific number of cases needed to require SR.
Missing data occurred at a rate of <5% for all
covariates, and it was imputed with the median for continuous variables
and with the mode for categorical variables. For all analyses, a
two-tailed value of P ≤ 0.05 was considered significant. All
statistical analyses were performed using SAS version 9.4 (SAS Institute
Inc., Cary, North Carolina).
Ethics Approval
This study was approved by the Institutional Review
Boards of our center. Patient consent was waived due to the
retrospective nature of the study.
Results
A total of 470 consecutive patients who underwent EVT by
seven operators were enrolled from January 2016 to September 2019. Of
these, 36 patients were excluded because of posterior circulation
occlusion. The remaining 434 patients all underwent 90-days follow-up
and were entered for the data analysis. Of these, 150 (34.6%) patients
were female, 82 (18.9%) patients were diagnosed with tandem lesions, and
377 (86.9%) patients were treated by stent retrieval. The median age,
baseline NIHSS score, ASPECTS, and OPT were 67 years (IQR, 57–74), 17
(IQR, 13–21), 9 (IQR, 8–10), and 285 min (IQR, 225 min−363 min),
respectively. The numbers of procedures performed by each of the seven
operators were 61, 54, 56, 54, 58, 79, and 72.
Of the enrolled patients, 348 (80.2%) patients achieved
SR, 51 (11.8%) had SICH, and in 10 (2.3%), the guidewire failed to cross
the occlusion site. There were eight patients with missing PRT data,
and the median PRT in the remaining patients was 73 min (IQR, 47 min-105
min). There were 193 (44.5%) patients who achieved 90-days functional
independence, and 100 (23.0%) patients died.
Patient- and treatment-related variables stratified according to the three levels of procedural experience are compared in Table 1.
There was no significant difference in the other variables, except for
baseline NIHSS, dyslipidemia, OPT, and type of anesthesia among the
three levels.
For the outcome variables, the rate of
SR and functional independence gradually increased across the three
levels of procedural experience. The median of PRT, mRS at 90 days, and
the rate of mortality gradually decreased. The distribution of mRS is
shown in Figure 1. However, the proportion of SICH was not significantly different with the accumulation of experience (Table 2).
Moreover, the rates of SR,
functional independence, mortality, SICH, and the median of PRT and mRS
were not significantly different among the seven operators (see Supplementary Table 1).
The Relationship Between Procedural Experience and Functional Independence and 90-Days Mortality
After adjusting for covariates (see Supplementary Table 2),
multivariable logistic backward regression showed that procedural
experience was independently associated with functional independence and
90-days mortality reduction (per 10-case increment: OR 1.219, 95% CI:
1.079–1.383, P < 0.001 and OR 0.847, 95% CI: 0.738–0.968, P = 0.016, both see Supplementary Table 4).
The adjusted predicted probability for functional independence and
90-days mortality with procedural experience are depicted in Figures 2A,B, respectively.
The Association Between Procedural Experience and PRT and SR
Similarly, adjusting for covariates (see Supplementary Table 3),
procedural experience was independently associated with a reduction in
PRT (per 10-case increment: β 8.087 min, 95% CI: 6.184–9.991, P < 0.001, see Supplementary Table 6). The predicted PRT with procedural experience is depicted in Figure 2C.
Multivariable logistic backward regression-adjusted covariates (see Supplementary Table 2)
demonstrated that procedural experience was independently associated
with SR (per 10-case increment: OR 1.553, 95% CI: 1.332–1.830, P < 0.001, see Supplementary Table 4). The predicted probability for SR with procedural experience is depicted in Figure 2D. We could visually observe that there was a clear ceiling effect for SR.
The Relationship Between Procedural Experience and SICH
Procedural experience showed no significant correlation with SICH (r = −0.072, P = 0.134). In the multivariable model, current smoking and collateral circulation were independently associated with SICH (see Supplementary Table 5).
Risk-Adjusted Cumulative Sum Chart
To accurately measure the threshold of EVT, the predicted
probability of SR itself, regardless of procedural experience, was
calculated with another logistic backward model for the RA-CUSUM chart
(see Supplementary Table 6).
The RA-CUSUM chart demonstrated that the predicted SR was greater than
the observed SR before the 19th consecutive case (ascending graph) and
that the predicted SR was comparable to the observed SR from the 19th to
the 29th consecutive case (leveling off). However, the predicted SR was
less than the observed SR after the 29th case (descending graph), as
shown in Figure 3. This finding suggests that the threshold of EVT required to achieve SR is 29 cases.
After experience with 29 cases, the
comparison of outcomes between our study and the published data from
Highly Effective Reperfusion evaluated in Multiple Endovascular Stroke
Trials (HERMES) collaboration (13), is shown in Table 3.
The rates of SR, functional independence, and SICH in our study were
significantly higher than those of the HERMES collaboration. However,
the proportion of 90-days mortality was not significantly different.
In addition, we found that patients
with tandem lesions had a lower proportion of SR than those with
isolated intracranial artery occlusion (OR 0.320, 95% CI: 0.177–0.578, P < 0.001, see Supplementary Table 4). The average predicted probability of SR in the 29th case was 83% according to Figure 2D.
The operators needed to complete at least 24 cases without tandem
lesions to obtain an equivalent predicted probability of SR; however,
for patients with tandem lesions, the number of procedures required
substantially increased to 50, as shown in Figure 4.
Discussion
Our study demonstrated that operator experience
significantly increased the likelihood of favorable clinical outcomes
and reduced the likelihood of mortality. This finding is supported by a
recent large-sample study (8).
This phenomenon is mainly attributed to the shorter procedural time
needed and the greater SR achieved with accumulated experience.
It is well-known that the benefit of EVT is highly time-dependent (14).
Recently, a multicenter study indicated that shorter procedure time led
to higher rates of functional independence and lower rates of 90-days
mortality (15).
Our study showed a dose-response relationship between the PRT and the
operator's experience although the magnitude of PRT reduction was small,
an 8 min reduction per 10-case increment. However, the effect was
obvious with the accumulation of experience; the median PRT reduced from
92 min at the 1–20 cases category to 52 min at the >40 cases
category.
Previous studies reported that higher procedure volume is associated with a higher proportion of SR (6–8),
and our study found a similar phenomenon. In addition, we observed that
the predicted rate of SR quickly reached a plateau. The final status of
reperfusion represent the strongest predictors of prognostic outcome in
patients who undergo EVT (16).
Unlike the ceiling effect observed with SR, the predicted rate of
functional independence continuously improved as experience increased.
This suggested that the goal of SR was easily achieved with procedural
experience. However, functional independence might rely more on patient
characteristics (e.g., baseline NIHSS score was significantly lower at
the >40 cases category in our study), not just SR.
Because of a ceiling effect on SR, there should be a
threshold for procedural experience. In addition to operator skill, the
learning curve depends on other factors, including procedure complexity
and patient characteristics. Therefore, describing the learning curve
should adjust for case mixing. The RA-CUSUM chart exactly solves this
problem. We plotted the RA-CUSUM chart to identify operators who needed
29 cases to overcome the learning curve effect. In our study, after 29
cases, the rates of SR, functional independence, and 90-days mortality
were 90.5, 57.3, and 18.6%, respectively. The results are numerically
not inferior to those of the HERMES collaboration (13).
As we all know, these trials were exclusively conducted by experienced
operators in high-volume centers. Therefore, we suggest that operators
can obtain beneficial effects after experience with 29 cumulative cases
of EVT based on the pooled results of five trials.
It is well-known that SICH occurred in 11.8% of patients
in our study, which is significantly more than that reported by the
HERMES collaboration (4.4%) (13).
However, it is consistent with previously published real-world data
from China. The Endovascular Treatment for Acute Anterior Circulation
Ischemic Stroke Registry reported 16.0% incidence of SICH after EVT (17).
Our study may reflect real-world practice, as we enrolled more patients
with disadvantageous treatment conditions. For example, compared to 15%
of patients in the HERMES collaboration, which used the same method to
assess collaterals, 46.5% (202/434) of the patients enrolled in our
study had poor collaterals compared to 15% patient in HERMES
collaboration (18).
Moreover, the rate of SICH did not significantly decrease as operator
experience increased in our study. This phenomenon may partly be
attributable to the insufficient case volume. According to a recent
study, when the operator case volume was limited within 150 cases, no
significant correlation between case volume and SICH could be found (r = −0.05, P = 0.178) (8).
In addition, in our present study, it is not surprising
that patients with tandem occlusions had a lower SR rate than those with
isolated intracranial occlusion because the additional complicated
manipulation involved in tandem occlusion treatment inevitably results
in a longer learning process. This phenomenon was also noticed in a
recently published study (19).
However, in contrast to our expectations, the number of EVT procedures
needed to treat, for patients with tandem lesions, was more than twice
as much as the number needed for patients with isolated intracranial
occlusions (50 vs. 24) in order to reach the target level of SR. There
have been no relevant data to date that support our results. Therefore,
further confirmation is needed.
Our study has some limitations. First, the learning curve
in the present study is representative of only the classic approaches
and devices, not the new ones (e.g., balloon guide catheter) that have
been developed in recent years (20, 21).
Second, we excluded patients with vertebral-basilar LVOS because the
methods for evaluating ASPECTS and collateral score in posterior
circulation are different from anterior circulation (10, 11, 22, 23).
Despite the proportion of posterior circulation-only accounting for
7.7% (36/470) of all LVOS treated by EVT in our study, this may lead to
underestimating the pitfalls of the learning curve because of the
similarity of EVT for anterior and posterior circulation. Third, our
results describe the performance of seven operators in a single center;
further validation is needed from multicenter data.
Conclusions
Our data suggested that there was a dose-response
relationship between operator case volume and clinical outcome,
procedure time reduction, and SR. Moreover, the experience level needed
to perform EVT successfully was at least 29 cases in this study.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed
and approved by the First Affiliated Hospital of University of Science
and Technology of China. Written informed consent for participation was
not required for this study in accordance with the national legislation
and the institutional requirements.
Author Contributions
WS and GW conceived, designed, and supervised the study.
XH, LX, MG, CZ, WH, and JC acquired the data. QC and YZ analyzed and
interpreted the data, provided statistical analysis, responsible for the
integrity of the data, accuracy of the data analysis, and drafted the
manuscript. WS, GW, and WH critically revised the manuscript for
important intellectual content. All authors contributed to the article
and approved the submitted version.
Funding
This work was supported by the National Natural Science
Foundation of China (Grant Numbers 81501193 and 81701184) and
Fundamental Research Funds for Central Universities (Grant Number
WK9110000056).
Conflict of Interest
The authors declare that the research was conducted in
the absence of any commercial or financial relationships that could be
construed as a potential conflict of interest.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2020.587409/full#supplementary-material
Qiankun Cai1, 
Xianjun Huang
Chao Zhang
Wei Hu
Wen Sun
