Impact of selective reporting bias on stroke trials: potential compromise in evidence synthesis - A cross-sectional study

 Shouldn't our non-existent stroke leadership be all over this and cracking down? But since there is NO leadership in stroke, NOTHING WILL OCCUR!

Impact of selective reporting bias on stroke trials: potential compromise in evidence synthesis - A cross-sectional study

• Xinyao Wang,
• Youlin Long,
• Na Zhang,
• Xinyi Wang,
• Qiong Guo,
• Ya Deng,
• Jin Huang &
• Liang Du 

BMC Medical Research Methodology volume 24, Article number: 255 (2024) Cite this article

• Metrics details

Abstract

Background

Accurate reporting of outcomes is crucial for interpreting the results of randomized controlled trials (RCTs). However, selectively reporting outcomes in publications to achieve researchers’ anticipated results still occurs frequently. This study aims to investigate the prevalence of selective reporting of outcomes in RCTs on treating acute ischemic stroke (AIS), identify factors contributing to this issue, and assess its potential impact on the degree and direction of intervention effect.

Methods

A search was conducted in MEDLINE, Embase, and the Cochrane Library to collect interventional RCTs on AIS published from 2020 to 2022. Full texts of RCTs were reviewed, and only those reporting International Clinical Trials Registry Platform primary registry numbers were included. Registration information of the RCTs was extracted from the registry platforms and compared with the publications’ details to assess the selective reporting of outcomes. Bayesian multilevel logistic regression was used to analyze the reasons behind selective reporting.

Results

Among the total of 159 AIS RCTs identified, 82 (51.6%) were ultimately included, as they reported registration numbers, which encompassed 819 outcomes. Among them, 72 RCTs (87.8%) and 497 outcomes (60.7%) exhibited selective reporting. Omission-type selective reporting (downgrading, omitting, or ambiguously reporting) accounted for 36.4%, while addition-type selective reporting (upgrading, adding, or altering the measurement scope of outcomes) comprised 63.6%. Omission-type selective reporting correlated with negative results (OR: 7.39; 95% CI: 4.08—13.44), whereas addition-type selective reporting correlated with positive results (OR: 2.07; 95% CI: 1.34—3.26) and publication in journals that are not in the top quartile of the Journal Citation Reports (OR: 2.48; 95% CI: 1.15—5.38).

Conclusions

Registered interventional AIS RCTs still face significant issues regarding selective reporting of outcomes. Therefore, it is necessary to further evaluate the influence of selective reporting bias on the positive results obtained from individual AIS RCTs and the systematic reviews based on these RCTs.

Peer Review reports

Background

Globally, stroke ranks as the second leading cause of death and the third leading cause of disability [1]. Acute Ischemic Stroke (AIS), characterized by acute focal neurological deficits due to vascular occlusive lesions, garners considerable attention among stroke types owing to its prevalent occurrence, notable disability rate, elevated mortality risk, and substantial economic burden [2,3,4,5,6,7]. Effective and timely treatments, such as intravenous thrombolysis and mechanical thrombectomy, are crucial in reducing mortality and disability rates, as well as in mitigating the economic burden of AIS [4, 8]. In the field of AIS treatment research, a substantial body of Randomized Controlled Trials (RCTs) and Systematic Reviews (SRs) provides a solid theoretical foundation for the formulation of treatment protocols [9,10,11,12,13,14,15,16]. These studies not only disclose the preferred treatment options under various circumstances but also highlight the optimal timing for intervention, offering crucial guidance for clinical practice.

However, the quality of RCTs may be compromised by the presence of potential biases such as measurement bias, attrition bias, and selective reporting bias, which ultimately could impair the quality of evidence produced [17]. Among these, selective reporting bias is a common source of bias at the time of publication, leading to inaccurate estimations of intervention effects and the production of biased evidence [18]. To enhance the transparency of clinical research, organizations such as the World Health Organization (WHO) and The International Committee of Medical Journal Editors (ICMJE) have been advocating for the registration of clinical trials since 2004 [19]. Clinical trial registration is regarded as a tool for detecting selective reporting. Existing studies indicate that discrepancies are common between publications in fields such as dentistry, dermatology, and psychology and their registered information [20,21,22,23,24].

Prior research has predominantly investigated the occurrence of selective outcome reporting in specific fields [20,21,22,23,24,25]. However, there is a lack of in-depth research analyzing the factors influencing different types of selective reporting. Therefore, this study aims to investigate the occurrence of selective reporting of outcomes in recently published RCTs on AIS treatment, analyze the factors influencing different types of selective reporting, and assess the potential impact on the degree and direction of outcome effects. This research endeavors to provide insights for the evaluation and utilization of evidence in AIS therapeutic studies.