40% of Scientists Admit Fraud “Always or Often” Contributes to Irreproducible Research

TruthFact
Source: Nature.com
Monya Baker
May 25, 2016

More than 70% of researchers have tried and failed to reproduce another scientist’s experiments, and more than half have failed to reproduce their own experiments. Those are some of the telling figures that emerged from Nature‘s survey of 1,576 researchers who took a brief online questionnaire on reproducibility in research.

The data reveal sometimes-contradictory attitudes towards reproducibility. Although 52% of those surveyed agree that there is a significant ‘crisis’ of reproducibility, less than 31% think that failure to reproduce published results means that the result is probably wrong, and most say that they still trust the published literature.

Data on how much of the scientific literature is reproducible are rare and generally bleak. The best-known analyses, from psychology1 and cancer biology2, found rates of around 40% and 10%, respectively. Our survey respondents were more optimistic: 73% said that they think that at least half of the papers in their field can be trusted, with physicists and chemists generally showing the most confidence.

The results capture a confusing snapshot of attitudes around these issues, says Arturo Casadevall, a microbiologist at the Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland. “At the current time there is no consensus on what reproducibility is or should be.” But just recognizing that is a step forward, he says. “The next step may be identifying what is the problem and to get a consensus.”

Failing to reproduce results is a rite of passage, says Marcus Munafo, a biological psychologist at the University of Bristol, UK, who has a long-standing interest in scientific reproducibility. When he was a student, he says, “I tried to replicate what looked simple from the literature, and wasn’t able to. Then I had a crisis of confidence, and then I learned that my experience wasn’t uncommon.”

The challenge is not to eliminate problems with reproducibility in published work. Being at the cutting edge of science means that sometimes results will not be robust, says Munafo. “We want to be discovering new things but not generating too many false leads.”

The scale of reproducibility

But sorting discoveries from false leads can be discomfiting. Although the vast majority of researchers in our survey had failed to reproduce an experiment, less than 20% of respondents said that they had ever been contacted by another researcher unable to reproduce their work. Our results are strikingly similar to another online survey of nearly 900 members of the American Society for Cell Biology (see go.nature.com/kbzs2b). That may be because such conversations are difficult. If experimenters reach out to the original researchers for help, they risk appearing incompetent or accusatory, or revealing too much about their own projects.

A minority of respondents reported ever having tried to publish a replication study. When work does not reproduce, researchers often assume there is a perfectly valid (and probably boring) reason. What’s more, incentives to publish positive replications are low and journals can be reluctant to publish negative findings. In fact, several respondents who had published a failed replication said that editors and reviewers demanded that they play down comparisons with the original study.

Nevertheless, 24% said that they had been able to publish a successful replication and 13% had published a failed replication. Acceptance was more common than persistent rejection: only 12% reported being unable to publish successful attempts to reproduce others’ work; 10% reported being unable to publish unsuccessful attempts.

Survey respondent Abraham Al-Ahmad at the Texas Tech University Health Sciences Center in Amarillo expected a “cold and dry rejection” when he submitted a manuscript explaining why a stem-cell technique had stopped working in his hands. He was pleasantly surprised when the paper was accepted3. The reason, he thinks, is because it offered a workaround for the problem.

Others place the ability to publish replication attempts down to a combination of luck, persistence and editors’ inclinations. Survey respondent Michael Adams, a drug-development consultant, says that work showing severe flaws in an animal model of diabetes has been rejected six times, in part because it does not reveal a new drug target. By contrast, he says, work refuting the efficacy of a compound to treat Chagas disease was quickly accepted4.

The corrective measures

One-third of respondents said that their labs had taken concrete steps to improve reproducibility within the past five years. Rates ranged from a high of 41% in medicine to a low of 24% in physics and engineering. Free-text responses suggested that redoing the work or asking someone else within a lab to repeat the work is the most common practice. Also common are efforts to beef up the documentation and standardization of experimental methods.

Any of these can be a major undertaking. A biochemistry graduate student in the United Kingdom, who asked not to be named, says that efforts to reproduce work for her lab’s projects doubles the time and materials used — in addition to the time taken to troubleshoot when some things invariably don’t work. Although replication does boost confidence in results, she says, the costs mean that she performs checks only for innovative projects or unexpected results.

Consolidating methods is a project unto itself, says Laura Shankman, a postdoc studying smooth muscle cells at the University of Virginia, Charlottesville. After several postdocs and graduate students left her lab within a short time, remaining members had trouble getting consistent results in their experiments. The lab decided to take some time off from new questions to repeat published work, and this revealed that lab protocols had gradually diverged. She thinks that the lab saved money overall by getting synchronized instead of troubleshooting failed experiments piecemeal, but that it was a long-term investment.

Irakli Loladze, a mathematical biologist at Bryan College of Health Sciences in Lincoln, Nebraska, estimates that efforts to ensure reproducibility can increase the time spent on a project by 30%, even for his theoretical work. He checks that all steps from raw data to the final figure can be retraced. But those tasks quickly become just part of the job. “Reproducibility is like brushing your teeth,” he says. “It is good for you, but it takes time and effort. Once you learn it, it becomes a habit.”

One of the best-publicized approaches to boosting reproducibility is pre-registration, where scientists submit hypotheses and plans for data analysis to a third party before performing experiments, to prevent cherry-picking statistically significant results later. Fewer than a dozen people mentioned this strategy. One who did was Hanne Watkins, a graduate student studying moral decision-making at the University of Melbourne in Australia. Going back to her original questions after collecting data, she says, kept her from going down a rabbit hole. And the process, although time consuming, was no more arduous than getting ethical approval or formatting survey questions. “If it’s built in right from the start,” she says, “it’s just part of the routine of doing a study.”

The cause

The survey asked scientists what led to problems in reproducibility. More than 60% of respondents said that each of two factors — pressure to publish and selective reporting — always or often contributed. More than half pointed to insufficient replication in the lab, poor oversight or low statistical power. A smaller proportion pointed to obstacles such as variability in reagents or the use of specialized techniques that are difficult to repeat.

But all these factors are exacerbated by common forces, says Judith Kimble, a developmental biologist at the University of Wisconsin–Madison: competition for grants and positions, and a growing burden of bureaucracy that takes away from time spent doing and designing research. “Everyone is stretched thinner these days,” she says. And the cost extends beyond any particular research project. If graduate students train in labs where senior members have little time for their juniors, they may go on to establish their own labs without having a model of how training and mentoring should work. “They will go off and make it worse,” Kimble says.

What can be done?

Respondents were asked to rate 11 different approaches to improving reproducibility in science, and all got ringing endorsements. Nearly 90% — more than 1,000 people — ticked “More robust experimental design” “better statistics” and “better mentorship”. Those ranked higher than the option of providing incentives (such as funding or credit towards tenure) for reproducibility-enhancing practices. But even the lowest-ranked item — journal checklists — won a whopping 69% endorsement.

The survey — which was e-mailed to Nature readers and advertised on affiliated websites and social-media outlets as being ‘about reproducibility’ — probably selected for respondents who are more receptive to and aware of concerns about reproducibility. Nevertheless, the results suggest that journals, funders and research institutions that advance policies to address the issue would probably find cooperation, says John Ioannidis, who studies scientific robustness at Stanford University in California. “People would probably welcome such initiatives.” About 80% of respondents thought that funders and publishers should do more to improve reproducibility.

“It’s healthy that people are aware of the issues and open to a range of straightforward ways to improve them,” says Munafo. And given that these ideas are being widely discussed, even in mainstream media, tackling the initiative now may be crucial. “If we don’t act on this, then the moment will pass, and people will get tired of being told that they need to do something.”

Read More At: Nature.com

Advertisements

The Illusion Called Medical Journalism: The Deep Secret

Corruption
Source: NoMoreFakeNews.com
Jon Rappoport
November 17, 2016

—Some of the greatest illusions are sitting out in the open. They are bypassed for two reasons. People refuse to believe they are illusions, despite the abundant evidence; and the professionals dedicated to upholding the illusions continue their work as if nothing at all has been exposed.

Medical journalists in the mainstream rely completely on studies published in prestigious journals.

This the rock. This is the science.

This is also the source of doctors’ authoritarian and arrogant advice to patients.

“Studies show…”

Well, that wraps it up. Nothing else to prove. The studies in the journals are the final word.

Medical reporters base their entire careers on these published reports.

But what if higher authorities contradicted all these studies? What if they scrutinized more studies than any reporter or doctor possibly could…and came to a shocking and opposite conclusion?

This very thing has happened. And the conclusions have been published. But medical reporters ignore them and go their merry way, as if a vast pillar of modern medicine is still intact…when it isn’t, when it has been decimated.

Buckle up.

Let us begin with a statement made by Dr. Marcia Angell, the former editor of The New England Journal of Medicine, perhaps the most prestigious medical journal in the world—a journal that routinely vets and prints thousands of medical studies:

“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.” —Marcia Angell, MD, The New York Review of Books, January 15, 2009

You might want to read that statement several times, to savor its full impact. Then proceed to this next one, penned by the editor of The Lancet, another elite and time-honored medical journal that publishes medical studies:

Richard Horton, editor-in-chief, The Lancet, in The Lancet, 11 April, 2015, Vol 385, “Offline: What is medicine’s 5 sigma?”

“The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness…

“The apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world. Or they retrofit hypotheses to fit their data. Journal editors deserve their fair share of criticism too. We aid and abet the worst behaviours. Our acquiescence to the impact factor fuels an unhealthy competition to win a place in a select few journals. Our love of ‘significance’ pollutes the literature with many a statistical fairy-tale…Journals are not the only miscreants. Universities are in a perpetual struggle for money and talent…”

Still standing? Here are several more statements. They are devastating.

The NY Review of Books (May 12, 2011), Helen Epstein, “Flu Warning: Beware the Drug Companies”:

“Six years ago, John Ioannidis, a professor of epidemiology at the University of Ioannina School of Medicine in Greece, found that nearly half of published articles in scientific journals contained findings that were false, in the sense that independent researchers couldn’t replicate them. The problem is particularly widespread in medical research, where peer-reviewed articles in medical journals can be crucial in influencing multimillion- and sometimes multibillion-dollar spending decisions. It would be surprising if conflicts of interest did not sometimes compromise editorial neutrality, and in the case of medical research, the sources of bias are obvious. Most medical journals receive half or more of their income from pharmaceutical company advertising and reprint orders, and dozens of others [journals] are owned by companies like Wolters Kluwer, a medical publisher that also provides marketing services to the pharmaceutical industry.”

Here’s another quote from the same article:

“The FDA also relies increasingly upon fees and other payments from the pharmaceutical companies whose products the agency is supposed to regulate. This could contribute to the growing number of scandals in which the dangers of widely prescribed drugs have been discovered too late. Last year, GlaxoSmithKline’s diabetes drug Avandia was linked to thousands of heart attacks, and earlier in the decade, the company’s antidepressant Paxil was discovered to exacerbate the risk of suicide in young people. Merck’s painkiller Vioxx was also linked to thousands of heart disease deaths. In each case, the scientific literature gave little hint of these dangers. The companies have agreed to pay settlements in class action lawsuits amounting to far less than the profits the drugs earned on the market. These precedents could be creating incentives for reduced vigilance concerning the side effects of prescription drugs in general.”

Also from the NY Review of Books, here are two more quotes from Marcia Angell, former editor-in-chief of The New England Journal of Medicine (“Drug Companies and Doctors: A Story of Corruption”):

“Consider the clinical trials by which drugs are tested in human subjects. Before a new drug can enter the market, its manufacturer must sponsor clinical trials to show the Food and Drug Administration that the drug is safe and effective, usually as compared with a placebo or dummy pill. The results of all the (there may be many) are submitted to the FDA, and if one or two trials are positive—that is, they show effectiveness without serious risk—the drug is usually approved, even if all the other trials are negative.”

Here is another Angell statement:

Continue Reading At: JonRappoport.wordpress.com
______________________________________________________________
Jon Rappoport

The author of three explosive collections, THE MATRIX REVEALED, EXIT FROM THE MATRIX, and POWER OUTSIDE THE MATRIX, Jon was a candidate for a US Congressional seat in the 29th District of California. He maintains a consulting practice for private clients, the purpose of which is the expansion of personal creative power. Nominated for a Pulitzer Prize, he has worked as an investigative reporter for 30 years, writing articles on politics, medicine, and health for CBS Healthwatch, LA Weekly, Spin Magazine, Stern, and other newspapers and magazines in the US and Europe. Jon has delivered lectures and seminars on global politics, health, logic, and creative power to audiences around the world. You can sign up for his free NoMoreFakeNews emails here or his free OutsideTheRealityMachine emails here.