I’m packing for vacation. The book that I will NOT pack is: Rigor Mortis, how sloppy science creates worthless cures, crushes hope and wastes billions by Richard Harris. I read it already two times, and anyone interested in science, or trying to deliver a piece of it once in a while, should read it. It makes you realise what we do, what we publish and what we read. And then, it makes you humble (or sad, or furious, or happy).Image may be NSFW.
Clik here to view.
The book starts with a paper from 2012 that hit science frontally. Scientists tried to reproduce breakthrough findings, mainly in cancer research, from 53 studies published in well-known high-impact papers. After a long struggle they could reproduce the results from 6 studies only. Each of these 53 studies were seen as highly innovative and relevant, that inspired many colleagues to change directions and embark on the published findings and that stimulated funding bodies to award proposals building on these findings (that appeared non-reproducible in the end). That’s for lab-based science. As you might be more into clinical epidemiology, that field was already exposed in 2005 by John Ioannidis, with his claim that most research findings are false.
Of note, the non-reproducibility has nothing to do with fraude. It is just that (sometimes unknown) lab conditions are critical for a result, or that a model is simply not suited for delivering results that can be extrapolated. Neither are false findings from epidemiological research due to fraude, but merely results of suboptimal study designs, inappropriate analytical methods and flagrantly wrong interpretation of p-values. Pooled in a meta-analysis crap than becomes truth. Why does this get published? Well, all the symptoms of the diseased system pass by: publication pressure for (young) scientists, horny journal editors, failing peer review, and so on.
Very recognizable must be the following (at least to some): “… the typical scientist is born in an area and dies in an area. Once people develop an expertise it’s very hard to get out of a field. It becomes their social unit. Colleagues in the field are the ones who determine your funding. They are your friends. They are at the conferences you go to.“ Watch out: befor you know you are part of the travelling circus.
At some places in the world this has led to the Science in transition movement, that aims to cure this disease. Impact factors and H-index do not matter (that much) anymore, societal impact of your work is key, and the current system of publishing must be fought. My dean, prof. Frank Miedema, leads the movement in the Netherlands, and guess what, he appears in this book (on page 225). Invited to Stanford he apparently “complained that scientists at his medical center (i.e., UMC Utrecht) publish 3,500 papers a year” and then asked the crowd “I don’t know who reads them. Have you read one of our papers?” The answer was silence……. Happy to hear him that proud.
This book is an excellent read, full of “slide-quotable one-liners”, and providing the (self-perceived) scientist with a reality-check. Naturally is doesn’t apply to us. My next testable hypothesis: my dean reads my blogs faster (after release) than any of my peer-reviewed low/middle/high-impact publications. Enjoy your summer vacation (if your career allows you to). Arrividecci
Image may be NSFW.
Clik here to view.
Image may be NSFW.Clik here to view.
