Encouraging See-Through Science – Maya B.

Chapter seven is dedicated to the problems with transparency in biomedical research and how the contribute to irreproducibility.

It starts with Brian Nosek a psychology professor at the University of Virginia who aims to reduce irreproducibility problems by having more transparency, especially with keeping track of overreaching ideas of studies. According to Nosek, the main problem is that scientists don’t clearly distinguish between exploratory and confirmatory research. This, in turn, causes the data to be analyzed incorrectly.

To deal with this he offers to let researchers register their hypotheses with an organization he started called the Center for Open Science to prove the studies origins. The Center for Open Science was formed to check studies reproducibility. In clinical trials, the hypothesis-submitting idea has actually already been put into law; requiring scientists to pre-register what their hypotheses in advance.

This turns out to have a big effect on reproducibility. Robert Kaplan and Veronica Irvin of the NIH tested major studies about drugs or dietary supplements done by the National Heart, lung and Blood Institute between 1970 and 2012 to see what effect the law had. “Of the thirty big studies done before the law took effect, 57 percent showed that the drug or supplement being tested was beneficial. But once scientists had to announce in advance exactly what they were looking for, the success rate plummeted. Only 8 percent of the studies (two out of twenty-five) published after 2000 confirmed the preregistered hypothesis.” – p.148. This suggests that much of the previous research’s hypotheses had changed to match the outcome.

Unfortunately, transparency isn’t only a problem of changing goals. There are also problems involving the sharing of science research. “‘Sharing of regents, as long as I’ve worked in the field has been highly variable,’ Mark Winey at the University of California, Davis told me.” -p.150. Even though it is often required, many labs still refuse to do it. This is because, although sharing information often helps speed biomedical progress, other than that, there is little motivation for scientists themselves to share research. In fact, there is motivation not to share research.

Steven Salzberg, an experienced genome analyst, recounts when he was once ‘scooped’. It happened when his lab and a Swedish lab both raced to complete the first Conifer genome. “Salzberg posted data as he went along and his competitors were clearly watching his progress, because when his genome was getting close, the other group rushed a paper into Nature. Salzberg says the European work was much less complete, ‘but journals don’t really care. They want the headlines.’” – p.153. It is more rewarding for the scientists to cut corners and get their work published first, rather than done well. Salzberg’ transparency hurt him in the end.

This was all quite surprising to me. It made me think about how to best do research. Would it be best to be completely secretive or share everything? In the end, I think the real problem lies with the field’s preference of hype over quality. Scientists shouldn’t be afraid to make mistakes so much that they lose sight of the truth or feel they can’t share data. “Right now, the structure of science makes it difficult for scientists to live by the values that often motivated them to go into research.” – p.163. Over the years, I have learned that people rarely do things without reason, and I think that scientists are no exception. As seen in this chapter, transparency is definitely a problem in the biomedical field, but I think it is rooted in what how what helps researchers careers, doesn’t necessarily help science progress.