am grateful to the University of Dundee for providing the facilities to research and write the second edition of this book. The success of the first edition was due in no small part to my colleagues for their support and encouragement. As the style and much of the content of this new edition follows the original, I am pleased to be able to thank them again for their help. Special thanks are due to Geraldine Fardon whose constructive comments highlighted imprecise, vague, and confusing text. I take full responsibility for any remaining flaws.
Preface to the First Edition
This book was written to meet the needs of health professionals as medicine moves to be evidence‐based. The initial idea arose during discussion with younger colleagues and students on difficulties of interpreting the medical literature. It quickly became apparent that their needs would be best met by a short book detailing criteria for critical appraisal.
The book is organised in two parts. The first five chapters provide an introduction to critical appraisal of quantitative research, indicating how papers can be read and how the results can be interpreted. Experienced researchers could easily omit these chapters. The final six chapters provide annotated checklists for critical appraisal. The first of these contains the general questions which can be asked of any study using a quantitative methodology. The succeeding five chapters review in turn the questions which are specific for each quantitative method. For convenience each of these five chapters concludes with a combined list of general and specific questions.
The book has been written to be simple and quick to use. Technical terms are avoided where possible, and the assessment criteria are explained but not justified. A larger and less accessible text would have been needed to give a proper rationale for each the checklists. To keep this a pocket guide, it was also decided to omit evaluations of other topics such as qualitative methods, health economics, clinical audit, decision analysis, and screening tests. An argument could be made for the inclusion of each, but to include them all would nearly double the size of the book. I hope the checklists prove useful.
Preface to the Second Edition
Several developments have highlighted the need for a second edition of this book. Recent work, particularly from the new discipline of meta‐research, has provided important evidence for critical appraisal. By examining and comparing large numbers of studies, this research has clarified the nature of and impact of many sources of bias. A separate development is the recognition that critical appraisal should distinguish between the risk of bias and the value of the findings to individuals, health care systems, and the wider society. Critical appraisal checklists should provide separate sets of questions for the two issues. Finally, many years of teaching critical appraisal skills have provided the author with an understanding of the difficulties that students and health professionals encounter when evaluating papers. These developments have led to improvements in the guidance given in this book.
This new edition follows the format and the spirit of the first: it is written to be easy to use and, where possible, technical terms are avoided. However, substantial modifications have been made. A new Introduction clarifies the aims of critical appraisal. The second chapter, entitled ‘Do Not Read the Paper’, is also new; it provides a simple method for eliciting the information needed for critical appraisal from published studies. This is supported by Chapter 5, ‘The In‐Depth Interrogation’, which is a much‐amended version of the original ‘Standard Appraisal Questions’. In addition, Chapter 4 on ‘Interpreting the Results’ has been updated in line with current views on the nature of statistical significance and the interpretation of confidence intervals.
All of the chapters that provide checklists for critical appraisal of specific research designs have been revised and updated, particularly Chapter 9, ‘Appraising Randomised Controlled Trials’, and Chapter 11, ‘Appraising Systematic Reviews’. A new chapter for the critical appraisal of cohort studies that evaluate interventions has been added. This research design is now being frequently used in medical research (the separate chapter on conventional cohort studies has been retained). All the checklists distinguish between critical and important sources of bias and they evaluate risk of bias separately from value.
Two new chapters at the end of the book provide methods for synthesising the findings of critical appraisal. One provides a method for summarising risk of bias. The second explores the important concept of certainty of evidence and describes how it is assessed. A further new chapter reviews the wide range of factors that determine the value of research findings to society as a whole. This second edition provides a much‐needed update to the guidance on critical appraisal.
CHAPTER 1 Introduction to Critical Appraisal
Critical appraisal is the process of assessing the strengths and weaknesses of published studies. It involves a structured and rigorous evaluation of all the stages of the research, from design to analysis and interpretation. All studies have imperfections, so the question is not if there are flaws, but whether they are likely to be important. Critical appraisal is a method of systematically examining research studies to determine the worth of their findings.
The aims of critical appraisal
Critical appraisal assesses two issues: bias in study design or conduct, and the value that the findings have for clinical practice. Bias occurs when systematic errors distort the results of a study. For example, a clinical trial should provide a good estimate of the benefit of a treatment, but weaknesses in design or conduct could produce a misleading estimate. There are many types of bias and these are explored through a series of questions that examine the main deficiencies in research.
The value of findings is the extent to which they will provide benefits for patients or the general population. Many factors contribute to the value of research findings. For a clinical trial, this would include the size of the treatment benefit, the importance of the outcome measure to patients, and whether the patients to be treated in a clinical setting are similar to the participants in the study. As with bias, there are sets of questions that assess value.
Three stages of critical appraisal
The critical appraisal of a paper is conducted in three stages: identify the research design, select the appropriate appraisal tool for that study, and apply the tool to assess the bias and the value of the research. This section introduces these stages, with subsequent chapters explaining in detail how to carry them out.
Identify the research design
Research studies can ask different types of questions. Some may be concerned with the effectiveness of treatments; others may investigate the likely prognosis of an illness. The research design should be matched to the research question. To evaluate treatment effectiveness a randomised controlled trial would be used, whereas prognosis would be investigated by a cohort study. The other designs are surveys, case–control studies, cohort studies that test interventions, and systematic reviews. Chapter 3 of this book provides guidance on identifying the correct design.
Select the appropriate appraisal tool
Critical appraisal tools have been developed to enable a forensic evaluation of research studies. They provide lists of questions that
