the statements made. Reviews and commentaries are used where available, although many landmark studies are also referenced. The approach taken is to cite sufficient papers to support a point, rather than give a comprehensive a review of it. The number of references cited reflect the wealth of evidence on the deficiencies in medical research.
I am grateful to the University of Dundee for providing the facilities to research and write this book, and to my colleagues for their support and encouragement. Special thanks are due to my long‐time colleague and friend Linda Irvine: she made several gently delivered, trenchant criticisms of early drafts and picked out factual, logical and grammatical failings in later drafts. The irony of writing a book about evidence is that it may contain errors of fact or interpretation. Equally there may be errors of omission. I take full responsibility for all flaws.
Aims of this Book
The advent of the randomised controlled trial (RCT) provided a method to generate reliable evidence on the effectiveness of treatments. It enables a fair comparison of groups [1], and should identify which treatments are beneficial and which are of little or no value. Since the early randomised controlled trials (RCTs) of the mid‐twentieth century, the design of trials has been progressively refined, making them the bedrock on which modern medicine is built. The findings of RCTs are used by regulatory authorities across the world to licence effective treatments. Other methods of testing treatments have been proposed, but none matches the ability of high quality randomised trials to provide good estimates of treatment effectiveness [2]. The RCT deserves its status as the gold standard for assessing the effectiveness of treatment.
The RCT now occupies a leading place in medical research, with tens of thousands of trials being published annually [3, 4]. These studies should provide high quality evidence across all fields of medicine, but that promise has not been fulfilled. At issue is the quality of the evidence. Concerns are growing ‘about the reliability and validity of the underlying research that supports regulatory and clinical decision‐making’ [5], with authors describing ‘the pervasiveness of poor quality clinical evidence’ [6] and concluding that ‘much of the published medical research is apparently flawed, cannot be replicated and/or has limited or no utility’ [7]. In a particularly trenchant comment, Ian Roberts and colleagues concluded that ‘the knowledge system underpinning healthcare is not fit for purpose and must change’ [8]. This book evaluates that proposition.
To investigate the knowledge system for healthcare, this book poses five questions: 1) what are the problems; 2) how common are they; 3) to what extent do they bias evidence; 4) why do they occur, and 5) how can they be prevented? Medical research is a vast enterprise, so the book focuses on the two most important research methods: the randomised controlled trial and the systematic review. Answers to the first three questions will provide a broad assessment of the quality of evidence (Chapters 2–5). The answer to the fourth question, on the causes of poor quality, highlights issues around misconduct and how the structures and incentives in the research environment influence the quality of evidence (Chapters 6 and 7). The final chapter presents an approach for developing a comprehensive strategy to the quality problem. This is supported by an Appendix, which lists the initiatives that have been proposed to improve research evidence. First, to introduce the nature of evidence in medicine, Chapter 1 provides a brief review of the rationale for treatments from ancient times to the present day.
This book focuses on the flaws in medical evidence. It does not review the important findings from the many high quality studies that have provided convincing evidence on the effectiveness of a wide range of treatments. Instead, the book describes the variety of deficiencies that afflict much research. This assessment could lead to an overly negative assessment of the state of medical evidence. The reality is that clinical research is spread across a spectrum from high to very low quality studies. The aim of the book is to use a detailed review of the flaws and their causes, to develop a strategy to prevent poor quality and misleading research. If implemented, the strategy could prevent some of the more egregious studies from being published, and rectify some with less reprehensible weaknesses. In this way, it could help move the spectrum upwards in quality. The book should be read from a constructive perspective of what is going wrong and how can we change it, in which the focus on flaws also provides the motivation for action.
REFERENCES
1 1. Chalmers, I. (2011). Why the 1948 MRC trial of streptomycin used treatment allocation based on random numbers. J. R. Soc. Med. 104: 383–386.
2 2. Byar, D.P., Simon, R.M., Friedewald, W.T. et al. (1976). Randomized clinical trials. Perspectives on some recent ideas. N. Engl. J. Med. 295: 74–80.
3 3. Bastian, H., Glasziou, P., and Chalmers, I. (2010). Seventy‐five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. https://doi.org/10.1371/journal.pmed.1000326.
4 4. Viergever, R.F. and Li, K. (2015). Trends in global clinical trial registration: an analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013. BMJ Open https://doi.org/10.1136/bmjopen‐2015‐008932.
5 5. Wallach, J.D., Gonsalves, G.S., and Ross, J.S. (2018). Research, regulatory, and clinical decision‐making: the importance of scientific integrity. J. Clin. Epidemiol. 93: 88–93.
6 6. Ioannidis, J.P.A., Stuart, M.E., Brownlee, S. et al. (2017). How to survive the medical misinformation mess. Eur. J. Clin. Investig. 47: 795–802.
7 7. Eshre, C.W.G. (2018). Protect us from poor‐quality medical research. Hum. Reprod. 33: 770–776.
8 8. Roberts, I., Ker, K., Edwards, P. et al. (2015). The knowledge system underpinning healthcare is not fit for purpose and must change. BMJ https://doi.org/10.1136/bmj.h2463.
CHAPTER 1 The Rationale for Treatment: A Brief History
The development of modern medicine has rightly been described as ‘the greatest benefit to mankind’ [1]. Vaccination, anaesthetics, aseptic surgery, antibiotics, insulin for diabetes and drugs to prevent and treat heart disease form part of a very long list of treatments that have transformed healthcare. These are the fruits of many years of careful clinical investigation supported by extensive research. However, medicine has a checkered history in which ineffective treatments were widely used: as Benjamin Franklin pithily remarked in the eighteenth century, ‘God heals, and the doctor takes the fees’ [2]. Studies of the history of medicine show that the basis on which treatments have been used has varied greatly across the centuries [3]. This chapter explores the rationale behind the use of treatments, and the way this has changed over time. It concludes by describing the progress towards present‐day methods for testing the effectiveness of treatments.
THEORY AS JUSTIFICATION FOR TREATMENT
In ancient times diseases were attributed to supernatural causes, spirits and demons. Treatments involved spells and prayers, or the wearing of amulets, which were intended to drive the malign forces from the patient [4]. Theories gradually evolved towards biological and physical causes of disease,
