“It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.”
-Arthur Conan Doyle: Sherlock Holmes
After 16 years of editing journals for The Physiological Society (Editor-in-Chief The Journal of Physiology, 2011–2016; and Experimental Physiology, 2006–2011), I was asked by Editor-in-Chief Gary Sieck to reflect on challenges, opportunities, and the future of publishing in physiological sciences before I take on my next assignment as Head of Physiology, Anatomy & Genetics at Oxford. I have never been a great one for looking back, although history can always teach us lessons for the future, so this editorial takes a brief backward and forward look at physiology and its dissemination.
The Challenge and the Opportunity
In a world of interconnectivity and immediacy, the generation of new scientists in physiology (Generation Y or “Millennials”) never experienced the analog life of typewriters, letraset, and fountain pens in the laboratory. Instead, this generation is at the sharp end of impact metrics, the hype index, and massification of biomedical research. Where should they position themselves? Where should physiology position itself? Let me first address the latter. Physiology as a discovery science has never been under so much perceived pressure as we transition the post-genomic era. The subject is often viewed as a phenotyping tool for the genomics community as they translate the genetic code in the hunt for the “Holy Grail” in finding cures for disease. This gene-phenotype link in understanding disease and complex traits has never been so hotly debated (4, 5, 8) (also see Experimental Biology 2015 in Boston: https://www.youtube.com/watch?v=A_q_bOWc8i0). Indeed, leaders in genome science now publically acknowledge the outcome for human health has been disappointing (so far) (7), as many (funding bodies) are entering the “trough of disillusionment” from the peak of “inflated expectations.” To continue on the hype cycle, we must move to the “slope of enlightenment,” and this is where physiology has had a call to return to center stage (4, 5). However, as we move toward the “plateau of productivity” in the cycle, this does not mean a call to typewriters and letraset. Technology drives science, and if good questions are asked, then modern tools will facilitate unlocking the secrets of nature. Moreover, having the time to think and write out the narrative is also important. We can often learn from our humanities and mathematical colleagues here who have different tools in the box–like the fountain pen. This was well illustrated when talking with Sir Andrew Wiles (Fermat's last theorem) at my College about solving big problems. Sometimes the (slow) speed of the old-fashioned analog tool can be used to create a sharpness of mind for that clever experiment or eureka moment that the (fast) speed of the digital process cannot replicate.
Lost or Found in Translation?
Whilst we need to unravel the causes of diabetes, cancer, hypertension, and neurodegenerative disease, and have the funding to support such programs, we also need to see parity of esteem in our funding and publishing to tackle blue-sky questions because they are interesting questions in their own right. The list of fascinating questions is long, for example: Why does a woodpecker not get a concussion? Why does the stomach not digest itself? How can a goose fly over Mt. Everest? How does the body match ventilation to metabolism in exercise so that blood gas homeostasis is maintained? These questions may have little apparent translational utility, but in another setting they may form the bedrock of future translation. Even if they don't translate in our lifetime, does it really matter in the big picture of knowledge? As Proust says, the real voyage of discovery is not in seeking new landscapes but in having new eyes. Physiology can of course reposition itself in the hype cycle if it is mindful of the lessons from the past. If we use the Nobel Prize in Physiology or Medicine as the gold standard, the historical literature is littered with examples of discovery, where in truth very few were ever made or undertaken with a translational aim, or, I suspect, were ever a direct result of an RO1 or a PPG supporting such an aim. This is where blue skies research is so important–history tells us this type of research has a long incubation period and is based on a high degree of serendipity. Why has mission-led research failed to crack some of the big biological problems? Placing a man on the moon was a much easier aspiration to fulfil than finding a cure for cancer because the underlying principles to be successful in going to the moon are better understood. One problem we have in physiological and biological science is that our discipline is based on very few laws compared with the physical sciences. However, to translate, you need physiological principles to base the translation on.
Who Says It's True?
Whilst there is a need to fund big science and big data projects, there is also a need to fund small science and small data projects and challenge both with rigorous peer review. But has peer review failed our community (2, 6) given the disturbing lack of reproducibility in preclinical science that Robert Bailey claims is estimated to cost the U.S. around $28Bpa? These trends first appeared (publically) 5 years ago when Bayer Pharmaceuticals reported that only 25% of 67 preclinical studies in mainly cancer biology could be reproduced. Worse to come was the dismal 11% reproducibility by the California pharma company Amgen when they attempted to replicate findings. This is not a new phenomenon and is an inconvenient truth that has been largely ignored (3). Several leading academics have raised the issue in the past. Most notably John Ioannidis (2) and the former Editor-in-Chief of the BMJ Richard Smith (6). But as Smith says, the flaws of peer review are easier to highlight compared with its attributes where it has notably been compared with democracy: “a system full of problems although the least worst we have.” Nevertheless, if the standards of biomedical research were applied to space travel, no one would ever get in a rocket given the poor failure rate to reproduce findings. Why is this?
Placing a human into space flight carries risk; whereas every attempt is made to minimize that risk, risk appetite in biomedical sciences is encouraged. Variability in observations led to the need to base perceived truth on the probability it is true, often in the presence of narrowly defined experimental conditions. As biomedical science enters a crisis point, the apparent lack of reproducibility is starting to damage the trust the taxpayer has given us. We have a number of drivers and events that probably all contribute to varying degrees in conflating a perfect storm for our community as a result of conscious and unconscious bias. For example, the pressure to publish in high-impact journals to facilitate career progression, commercialisation of intellectual property to make money, “p hacking,” metrics hunting, and not disclosing the full method (losing the competitive edge) can have a negative impact. More worrying is the trend for well regarded journals that prevent detailed methods being published or relegating them to supplements.
Future of Physiological Publishing
Multimedia, immediacy, access to data repositories, and ability to see the experiment are all exciting prospects for the future that should help support the ferocity of science. However, we must be circumspect with our metrics. The metrics generation has created a culture that “the number” is a quality measure of the outcome. In publishing terms, we know this is fraught, so I will not rehearse all the arguments here. When Hirsch (1) first mooted the h-index, it was generally seen to have considerable face validity in reflecting a way to quantify an individual's research output. At least in the physical sciences, Hirsch calculated the h-index of Nobel Prize winners and found 84% of them had an h-index of at least 30. I would argue this would be seen as a rather poor score in biomedical science for Deans, promotion, and sectional committees electing to prestigious national fellowships of learned societies. Yet many people in influential circles often give too much credence to volume drivers instead of taking the time to read and understand the actual science. Can we put a number to real discovery, and, if so, what should it measure? Currently, we take “Kodak moments” at one point in time of a nonlinear process against a backdrop of scientific fashions. It is not until we look back and document the path of discovery that we see how difficult it is to predict. Over the past 20 years, have we inadvertently overpromised and under delivered, and been a victim of the hype cycle? Are we in danger of entering the territory of Emperor's new clothes? The public and our legislators are looking hard at us as the red lights flash.
Notwithstanding the flashing lights, the future is in our hands as authors, editors, and academics to raise standards for the current generation and the next generation. Physiological research and our journals are not immune from the problems highlighted above. However, publishing in physiology is bright if we are mindful of the minefields and recognize that truly outstanding science has one common theme–reproducibility and longevity. So what would I recommend? Read the literature carefully and cite it correctly. Give credit where credit is due. Show the data! Follow the data, especially if it goes against your pet hypothesis. Think Marcel Proust. As editors, we should be encouraging a full disclosure of the method or recipe like in cooking. I was always raised to show the raw signal because a lot of our science can reside close to the signal-to-noise levels. If the observation is sound, then complex statistics are not needed. As was once said to me as a graduate student, “You don't need to fly around the moon six times to show it is not made of blue cheese on the other side.” Importantly, if the discovery is robust, it will stand the test of time since others will wish to repeat it and build upon it. It is easy to have your moment in the spotlight and hunt glory, but unless science is conducted rigorously, disaster can await that few ever fully recovery from. Remember, “cold fusion,” MMR vaccine and autism claims, and more recently the RIKEN stem-cell tragedy. There is nothing wrong in being wrong if the science is presented in good faith. After all, there are examples of notable Nobel Prizes that have been awarded for mistakes. As Confucius says: “If you make a mistake and do not correct it, then this is a mistake.” In the end, the record must be correct.
No conflicts of interest, financial or otherwise, are declared by the author(s).
- ©2016 Int. Union Physiol. Sci./Am. Physiol. Soc.