More than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments.

If there is not enough information to reproduce research there surely is not enough information to create a stroke protocol out of it. We're screwed, that great stroke association president should be reading the riot act to stroke researchers. 

More than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments.

More than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments. Those are some of the telling figures that emerged from Nature's survey of 1,576 researchers who took a brief online questionnaire on reproducibility in research.
The data reveal sometimes-contradictory attitudes towards reproducibility. Although 52% of those surveyed agree that there is a significant 'crisis' of reproducibility, less than 31% think that failure to reproduce published results means that the result is probably wrong, and most say that they still trust the published literature.

Data on how much of the scientific literature is reproducible are rare and generally bleak. The best-known analyses, from psychology¹ and cancer biology², found rates of around 40% and 10%, respectively. Our survey respondents were more optimistic: 73% said that they think that at least half of the papers in their field can be trusted, with physicists and chemists generally showing the most confidence.
The results capture a confusing snapshot of attitudes around these issues, says Arturo Casadevall, a microbiologist at the Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland. “At the current time there is no consensus on what reproducibility is or should be.” But just recognizing that is a step forward, he says. “The next step may be identifying what is the problem and to get a consensus.”

Failing to reproduce results is a rite of passage, says Marcus Munafo, a biological psychologist at the University of Bristol, UK, who has a long-standing interest in scientific reproducibility. When he was a student, he says, “I tried to replicate what looked simple from the literature, and wasn't able to. Then I had a crisis of confidence, and then I learned that my experience wasn't uncommon.”

The P value - not all it's cracked up to be

The challenge is not to eliminate problems with reproducibility in published work. Being at the cutting edge of science means that sometimes results will not be robust, says Munafo. “We want to be discovering new things but not generating too many false leads.”

The scale of reproducibility

But sorting discoveries from false leads can be discomfiting. Although the vast majority of researchers in our survey had failed to reproduce an experiment, less than 20% of respondents said that they had ever been contacted by another researcher unable to reproduce their work. Our results are strikingly similar to another online survey of nearly 900 members of the American Society for Cell Biology (see go.nature.com/kbzs2b). That may be because such conversations are difficult. If experimenters reach out to the original researchers for help, they risk appearing incompetent or accusatory, or revealing too much about their own projects.

How scientists fool themselves – and how they can stop

A minority of respondents reported ever having tried to publish a replication study. When work does not reproduce, researchers often assume there is a perfectly valid (and probably boring) reason. What's more, incentives to publish positive replications are low and journals can be reluctant to publish negative findings. In fact, several respondents who had published a failed replication said that editors and reviewers demanded that they play down comparisons with the original study.
Nevertheless, 24% said that they had been able to publish a successful replication and 13% had published a failed replication. Acceptance was more common than persistent rejection: only 12% reported being unable to publish successful attempts to reproduce others' work; 10% reported being unable to publish unsuccessful attempts.

Survey respondent Abraham Al-Ahmad at the Texas Tech University Health Sciences Center in Amarillo expected a “cold and dry rejection” when he submitted a manuscript explaining why a stem-cell technique had stopped working in his hands. He was pleasantly surprised when the paper was accepted³. The reason, he thinks, is because it offered a workaround for the problem.

Robust research: Institutions must do their part for reproducibility

Others place the ability to publish replication attempts down to a combination of luck, persistence and editors' inclinations. Survey respondent Michael Adams, a drug-development consultant, says that work showing severe flaws in an animal model of diabetes has been rejected six times, in part because it does not reveal a new drug target. By contrast, he says, work refuting the efficacy of a compound to treat Chagas disease was quickly accepted⁴.

The corrective measures

One-third of respondents said that their labs had taken concrete steps to improve reproducibility within the past five years. Rates ranged from a high of 41% in medicine to a low of 24% in physics and engineering. Free-text responses suggested that redoing the work or asking someone else within a lab to repeat the work is the most common practice. Also common are efforts to beef up the documentation and standardization of experimental methods.
Any of these can be a major undertaking. A biochemistry graduate student in the United Kingdom, who asked not to be named, says that efforts to reproduce work for her lab's projects doubles the time and materials used — in addition to the time taken to troubleshoot when some things invariably don't work. Although replication does boost confidence in results, she says, the costs mean that she performs checks only for innovative projects or unexpected results.

Consolidating methods is a project unto itself, says Laura Shankman, a postdoc studying smooth muscle cells at the University of Virginia, Charlottesville. After several postdocs and graduate students left her lab within a short time, remaining members had trouble getting consistent results in their experiments. The lab decided to take some time off from new questions to repeat published work, and this revealed that lab protocols had gradually diverged. She thinks that the lab saved money overall by getting synchronized instead of troubleshooting failed experiments piecemeal, but that it was a long-term investment.

How quality control could save your science

Irakli Loladze, a mathematical biologist at Bryan College of Health Sciences in Lincoln, Nebraska, estimates that efforts to ensure reproducibility can increase the time spent on a project by 30%, even for his theoretical work. He checks that all steps from raw data to the final figure can be retraced. But those tasks quickly become just part of the job. “Reproducibility is like brushing your teeth,” he says. “It is good for you, but it takes time and effort. Once you learn it, it becomes a habit.”
One of the best-publicized approaches to boosting reproducibility is pre-registration, where scientists submit hypotheses and plans for data analysis to a third party before performing experiments, to prevent cherry-picking statistically significant results later. Fewer than a dozen people mentioned this strategy. One who did was Hanne Watkins, a graduate student studying moral decision-making at the University of Melbourne in Australia. Going back to her original questions after collecting data, she says, kept her from going down a rabbit hole. And the process, although time consuming, was no more arduous than getting ethical approval or formatting survey questions. “If it's built in right from the start,” she says, “it's just part of the routine of doing a study.”

The cause

The survey asked scientists what led to problems in reproducibility. More than 60% of respondents said that each of two factors — pressure to publish and selective reporting — always or often contributed. More than half pointed to insufficient replication in the lab, poor oversight or low statistical power. A smaller proportion pointed to obstacles such as variability in reagents or the use of specialized techniques that are difficult to repeat.
But all these factors are exacerbated by common forces, says Judith Kimble, a developmental biologist at the University of Wisconsin–Madison: competition for grants and positions, and a growing burden of bureaucracy that takes away from time spent doing and designing research. “Everyone is stretched thinner these days,” she says. And the cost extends beyond any particular research project. If graduate students train in labs where senior members have little time for their juniors, they may go on to establish their own labs without having a model of how training and mentoring should work. “They will go off and make it worse,” Kimble says.

What can be done?

Respondents were asked to rate 11 different approaches to improving reproducibility in science, and all got ringing endorsements. Nearly 90% — more than 1,000 people — ticked “More robust experimental design” “better statistics” and “better mentorship”. Those ranked higher than the option of providing incentives (such as funding or credit towards tenure) for reproducibility-enhancing practices. But even the lowest-ranked item — journal checklists — won a whopping 69% endorsement.

The survey — which was e-mailed to Nature readers and advertised on affiliated websites and social-media outlets as being 'about reproducibility' — probably selected for respondents who are more receptive to and aware of concerns about reproducibility. Nevertheless, the results suggest that journals, funders and research institutions that advance policies to address the issue would probably find cooperation, says John Ioannidis, who studies scientific robustness at Stanford University in California. “People would probably welcome such initiatives.” About 80% of respondents thought that funders and publishers should do more to improve reproducibility.
“It's healthy that people are aware of the issues and open to a range of straightforward ways to improve them,” says Munafo. And given that these ideas are being widely discussed, even in mainstream media, tackling the initiative now may be crucial. “If we don't act on this, then the moment will pass, and people will get tired of being told that they need to do something.”
Download the full questionnaire used in the survey and the raw data in a spreadsheet (the data are also available as a tab-delimited file at Figshare).

Nature

533,

452–454

(26 May 2016)

doi:10.1038/533452a

Brain‐computer interfaces for post‐stroke motor rehabilitation: a meta‐analysis

Without any references to protocols for use with BCI this is totally fucking useless without a lot of followup work. The mentors and senior researchers involved need to be read the riot act. The whole fucking point of stroke research is to help survivors recover. This does nothing of the sort as it is written.  I don't care how hard that is to do. You try recovering from a stroke with NO useful medical help. 

Brain‐computer interfaces for post‐stroke motor rehabilitation: a meta‐analysis

María A. Cervera

Surjo R. Soekadar

Junichi Ushiba

José del R. Millán

Meigen Liu

Niels Birbaumer

Gangadhar Garipelli

First published: 25 March 2018

https://doi.org/10.1002/acn3.544

Read the full text

Abstract

Brain‐computer interfaces (BCIs) can provide sensory feedback of ongoing brain oscillations, enabling stroke survivors to modulate their sensorimotor rhythms purposefully. A number of recent clinical studies indicate that repeated use of such BCIs might trigger neurological recovery and hence improvement in motor function. Here, we provide a first meta‐analysis evaluating the clinical effectiveness of BCI‐based post‐stroke motor rehabilitation. Trials were identified using MEDLINE, CENTRAL, PEDro and by inspection of references in several review articles. We selected randomized controlled trials that used BCIs for post‐stroke motor rehabilitation and provided motor impairment scores before and after the intervention. A random‐effects inverse variance method was used to calculate the summary effect size. We initially identified 524 articles and, after removing duplicates, we screened titles and abstracts of 473 articles. We found 26 articles corresponding to BCI clinical trials, of these, there were nine studies that involved a total of 235 post‐stroke survivors that fulfilled the inclusion criterion (randomized controlled trials that examined motor performance as an outcome measure) for the meta‐analysis. Motor improvements, mostly quantified by the upper limb Fugl‐Meyer Assessment (FMA‐UE), exceeded the minimal clinically important difference (MCID=5.25) in six BCI studies, while such improvement was reached only in three control groups. Overall, the BCI training was associated with a standardized mean difference of 0.79 (95% CI: 0.37 to 1.20) in FMA‐UE compared to control conditions, which is in the range of medium to large summary effect size. In addition, several studies indicated BCI‐induced functional and structural neuroplasticity at a subclinical level. This suggests that BCI technology could be(NOT GOOD ENOUGH!, will be should be the only answer) an effective intervention for post‐stroke upper limb rehabilitation. However, more studies with larger sample size are required to increase the reliability of these results.

Sonification of Arm Movements in Stroke Rehabilitation – A Novel Approach in Neurologic Music Therapy

This has been out there for years. When will the presidents of our fucking failures of stroke associations read the riot act to researchers that duplicate previous research and don't write up stroke protocols?

Interactive Sonification of Human Movements for Stroke Rehabilitation  May, 2012 

Approaching a new stroke rehabilitation therapy with a SonicPainter  Nov. 2013

A mobile sonification system for stroke rehabilitation  July, 2014 

Sonification as a possible stroke rehabilitation strategy  Oct. 2014 

Moving with music for stroke rehabilitation: a sonification feasibility study  March 2015 

The latest here:

Sonification of Arm Movements in Stroke Rehabilitation – A Novel Approach in Neurologic Music Therapy

Daniel S. Scholz¹, Sönke Rohde¹, Nikou Nikmaram¹, Hans-Peter Brückner¹, Michael Großbach¹, Jens D. Rollnik² and Eckart O. Altenmüller¹*

• ¹Institute of Music Physiology and Musicians’ Medicine, University of Music, Drama and Media, Hannover, Germany
• ²Institute for Neurorehabilitational Research (InFo), BDH-Clinic Hessisch Oldendorf, Teaching Hospital of Hannover Medical School (MHH), Hessisch Oldendorf, Germany

Gross motor impairments are common after stroke, but efficient and motivating therapies for these impairments are scarce. We present an innovative musical sonification therapy, especially designed to retrain patients’ gross motor functions. Sonification should motivate patients and provide additional sensory input informing about relative limb position. Twenty-five stroke patients were included in a clinical pre–post study and took part in the sonification training. The patients’ upper extremity functions, their psychological states, and their arm movement smoothness were assessed pre and post training. Patients were randomly assigned to either of two groups. Both groups received an average of 10 days (M = 9.88; SD = 2.03; 30 min/day) of musical sonification therapy [music group (MG)] or a sham sonification movement training [control group (CG)], respectively. The only difference between the two protocols was that in the CG no sound was played back during training. In the beginning, patients explored the acoustic effects of their arm movements in space. At the end of the training, the patients played simple melodies by coordinated arm movements. The 15 patients in the MG showed significantly reduced joint pain (F = 19.96, p < 0.001) in the Fugl–Meyer assessment after training. They also reported a trend to have improved hand function in the stroke impact scale as compared to the CG. Movement smoothness at day 1, day 5, and the last day of the intervention was compared in MG patients and found to be significantly better after the therapy. Taken together, musical sonification may be a promising therapy for motor impairments after stroke, but further research is required since estimated effect sizes point to moderate treatment outcomes.

Introduction

Stroke is a major cause of mortality and morbidity in both the developed and developing world (1). In Germany, stroke is one of the most common disorders with an estimated 200,000 first events and 66,000 recurrent events in 2008 (2). The World Health Organization stresses the need to collect high quality longitudinal data on rehabilitation and to improve the comparability between studies (3).

The rehabilitation of stroke patients remains a challenge, although there are currently several new training programs under development that aim at improved efficiency and sustainability of stroke rehabilitation (4). Some of the traditional rehabilitation programs lack general acceptance by patients, due to the required endurance and high demands on the patients’ cooperation, which sometimes is perceived as a frustrating experience (5). Yet, even the well-established standard physiotherapies do not unambiguously provide evidence of efficacy when it comes to improvement of skilled motor behavior (6–8). Therefore, there is an urgent need for innovative, motivating, and goal-directed training protocols in stroke rehabilitation.

In this article, we present an innovative approach to rehabilitation by retraining the gross motor functions of the affected upper limbs using musical sonification. In an earlier clinical feasibility study (9), we showed how a musical sonification therapy could be applied. The data presented herein were obtained with this method from a larger number of patients. Sonification stands for the usage of non-speech sound representing otherwise not audible information (10). One of the first sonification devices was the Geiger–Müller counter, which detects electromagnetic radiation and communicates a decay by a click sound. In the present study, arm movements were translated into sound. In two earlier studies, we demonstrated the efficacy of a music-supported stroke rehabilitation training utilizing a MIDI drumset and a MIDI piano (11, 12). Stroke patients with some residual abilities to move the arm and the fingers were instructed to play simple tunes (nursery rhymes or folk songs) on either instrument. We could show that auditory sensorimotor circuits established via this form of music-supported therapy (MST) promotes beneficial neuroplasticity in stroke patients (13, 14). One of the few constraints of MST was that it was mainly designed to retrain fine-motor skills on MIDI instruments. And it did not provide continuous real-time feedback for the gross motor functions of the arm, which are more frequently impaired in early rehabilitation stages. A real-time movement feedback may be beneficial since it informs the patients about the way they move, not only whether they hit the target or not. With the musical sonification therapy presented here, patients repeatedly train movements with their affected arm in a predefined space. They form associations of their relative armposition in space and the corresponding sound at this specific position. At the end, they play familiar melodies by moving their arm. This musical sonification therapy, therefore, broadens the scope to train stroke patients from an earlier stage on, when still suffering from gross motor dysfunction. Musical sonification will not only contribute to the motivation of the patients due to its playful and positive emotional character, but may also improve motor control, since auditory real-time feedback of the patient’s arm movements can be substituted for potentially lost proprioception. There are several preliminary studies with healthy participants that apply non-musical sonification in motor control and the perception of movements (15–17). Schmitz et al. found that sonifying breast stroke movements led to more precise perceptual judgments of movement velocity. They showed that sonification of movements amplifies the human action observation system as indicated by more pronounced fMRI connectivity patterns between the activation peaks of the left superior and medial posterior temporal regions with the basal ganglia, the thalamus, and frontal regions for movement congruent sonification stimuli. Thus, sonification may be an important method to enhance training and therapy effects in neurological rehabilitation. Chen et al. developed a real-time, multimodal feedback system for stroke rehabilitation (18). This sonification system was tested with stroke patients and showed promising results (19). However, in their design, music was only a passive byproduct of arm movements. That means participants did not play with the sonification sound intentionally. They moved their arms and harmonic music progressions were played back to them. In contrast to that, we developed a musical sonification therapy to train stroke patients to explicitly and consciously play music through intended movements of their affected upper extremity. Thus, we hoped to be able to use the beneficial effects of music on neuroplasticity to facilitate the recovery after stroke (13). Since in other studies repetitive exercise has been shown to be effective (8, 20), our training is of a repetitive nature too. We hypothesize that the auditory cues provided by the sonification may make multimodal associative learning possible where otherwise mere visual and motor learning would have taken place. We assume that patients will benefit in their rehabilitation process from guided attention, necessary concentration, and long-term motivation to play music. Rohrer et al. (21) (see also references therein) describe an increase of several movement smoothness indices in both acute and chronic stroke patients during movement therapy. Hence, the present study additionally investigated changes in movement smoothness over the course of the therapy. After having evaluated an optimal two-dimensional sonification mapping (22), we now present a more detailed analysis of our three-dimensional musical sonification therapy with a larger sample (9).

Methodology of the Stroke Self-Management Rehabilitation Trial: An International, Multisite Pilot Trial

Well, well, look at that, stroke researchers giving up on solving how to get better rehabilitation results. They want to throw all the responsibility for your rehabilitation on you. As Dr. Steven Wolf writes, a rehabilitation stroke expert and professor at Emory University School of Medicine in Atlanta.  "Stroke patients need to rely more on their own problem solving to regain mobility".
If we had anything resembling a decent stroke association, that president would be reading the riot act to anyone who throws up their hands like this and gives up on doing their job because it is too hard. F*cking hey, try rehabbing from a stroke with NO help from your doctor.

What a bunch of bullcrap, 'You've had this major brain attack and we know nothing about how to get you better.  So deal with this yourself, you will recover just as well without our help.'
But it will save a lot of money and our doctors will never need to read research again, washing their hands of their responsibility to help us get better. 


http://www.strokejournal.org/article/S1052-3057%2814%2900433-9/abstract 

Kelly M. Jones, PhD

,

Rohit Bhattacharjee, MPh

,

Rita Krishnamurthi, PhD

,

Sarah Blanton, PhD

,

Alice Theadom, PhD

,

Suzanne Barker-Collo, PhD

,

Amanda Thrift, PhD

,

Priya Parmar, MSc

,

Annick Maujean, PhD

,

Annemarei Ranta, FRACP, MD, PG Cert Sci (Public Health)

,

Emmanuel Sanya, FWACP, MSc, PhD

,

Valery L. Feigin, MD, PhD

on behalf of the

SMART Study Group

¹SMART Study Group members (additional to the article coauthors): Yogini Ratnasabapathy, Denise Taylor, Elizabeth Kendall, Carolyn Ehrlich, Steven Wolf, Dominique Cadilhac, Marilyn MacKay-Lyons, Man Mohan Mehndiratta, Jeyaraj Durai Pandian, Deepti Arora, Peter Langhorn, Gustavo Saposnik, Narayanaswamy Venketasubramanian, Bo Norrving, Akshay Anand, Dheeraj Kurana, Michael Brainin, Natan Bornstein, Richard Lindley, Foad Abd-Allah, Reginald Obiako, Emmanuel Sanya, Maree O'Connor, Rene Stolwyk, and Peter New.

Received: May 29, 2014; Received in revised form: August 6, 2014; Accepted: August 25, 2014; Published Online: December 09, 2014

DOI: http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2014.08.029

Publication stage: In Press Corrected Proof

• Abstract
• Full Text
• Images
• References

Rationale

Stroke is a major cause of long-term adult disability with many survivors living in the community relying on family members for on-going support. However, reports of inadequate understanding of rehabilitation techniques are common. A self-management DVD-based observational learning tool may help improve functional outcomes for survivors of stroke and reduce caregivers' burden.

Aims

This article describes the methodology of the stroke self-management rehabilitation trial. The overall aim of this pilot trial is to assess the feasibility and preliminary efficacy of a DVD-based intervention for improving functional outcomes of survivors of stroke 2 months postrandomization to inform the design of a full-scale randomized clinical trial.

Design

Recruitment of a minimum of 20 survivors of stroke and their informal caregivers (where available) in each of the participating centers will occur across multiple international sites. After baseline assessments, participants will be randomly assigned to an intervention or standard care group. The intervention comprises a structured DVD observation and practice schedule over 8 weeks. All participants will complete follow-up assessments.

Study outcomes

The outcome measures will include a global shift in the Rankin Scale scores and dichotomized scores, changes in quality of life, general health, depression, and caregiver burden at 2 months postrandomization. A qualitative analysis of the effects of the intervention will also be undertaken.

Discussion

The results of the pilot study will provide knowledge of whether observational learning techniques delivered via DVD can effectively improve recovery after stroke and reduce caregiver burden.