style="font-size:15px;"> 56 56. Mafi, J.N. and Parchman, M. (2018). Low‐value care: an intractable global problem with no quick fix. BMJ Qual. Saf. 27: 333–336.57 57. Levinson, W., Kallewaard, M., Bhatia, R.S. et al. (2015). ‘Choosing Wisely’: a growing international campaign. BMJ Qual. Saf. 24: 167–174.
58 58. Best, M. and Neuhauser, D. (2005). Pierre Charles Alexandre Louis: master of the spirit of mathematical clinical science. Qual. Saf. Health Care 14: 462–464.
CHAPTER 2 Sources of Bias in Randomised Controlled Trials
The great strength of randomised controlled trials is that they provide a fair comparison of treatments. However flaws in the design and conduct of clinical trials can hamper the ability to make fair comparisons. This chapter explores the nature and frequency of these problems. It uses the findings from meta‐research studies that explore the quality of the methodology of large series of clinical trials, to identify the extent of these deficiencies [1, 2].
In essence, clinical trials are conducted in three stages. First, patients are allocated to receive the new treatment or the conventional one (or placebo). Then they are followed up over time to allow the effects of the treatment to occur. Finally the health status of the patients is assessed to show whether the new treatment has a better outcome than the comparator.
This chapter explores the types of flaws that occur at these three stages. The main concern is with the risk of bias. Bias occurs when the findings of a clinical trial do not provide a fair assessment of the true benefit of the new treatment. Poor quality of the study methods can increase the risk of bias. This chapter investigates the sources of bias and the impact these have on estimates of treatment effect.
METHOD OF TREATMENT ALLOCATION
In clinical trials patients are assigned to different treatment groups using a sequence of random numbers. The process of randomisation produces two groups that, on average, are similar at baseline on factors such as severity and duration of disease. When the groups are similar at baseline, any differences between them at follow‐up will be due to the effect of the treatment. Deficiencies in the method of randomisation could cause imbalances at baseline, resulting in differences in outcomes between the groups that are unrelated to treatment. This section identifies the problems that frequently occur with treatment allocation.
Generation of the Random Assignment
Meta‐research studies have shown that the randomisation process is often flawed. For many trials the method is not described, is poorly described, or is well described but clearly inadequate [3–5]. An evaluation of 2 groups of trials, comprising 1,376 and 984 studies, found that 39.6% and 52.2% had flawed or poorly reported methods for generating the randomisation sequence [6].
The recommended method of randomising patients is by computer‐generated random numbers. Other techniques, such as allocation by day of admission to hospital or by even or odd dates of birth, have been used, but are thought to be unreliable. Careful reviews of large series of trials have shown that poor quality and inadequately described methods of generating the randomisation sequence commonly exaggerate the apparent benefit of treatment [4, 7].
Importance of Concealed Allocation
The assignment of patients to treatment should correspond exactly to the randomisation sequence. However the process could be distorted, if the clinician recruiting the patient knew in advance which treatment the next patient would be given. For example, concerns about possible side effects might lead a clinician to decide not to recruit more severely ill patients if they were to be randomised to receive the new treatment. Thus fewer severely ill patients would be allocated to the active treatment. The resulting differences between the groups at baseline would bias the estimate of treatment benefit.
The solution is to ensure those involved in the trial have no access to the randomisation sequence, a process called allocation concealment. To achieve this, the randomisation is commonly handled by a remote site, such as a clinical trials unit, which could, for example, provide the treatments in separate containers labelled A or B. The methods used to conceal treatment allocation should be clearly described in the trial report, to give reassurance that bias is unlikely to have occurred.
Overviews of trials have shown that inadequate or poorly reported methods of allocation concealment are common. A major review of over 20,000 trials found that allocation concealment was adequate in only 35% of trials [5]. Other overviews in different clinical specialties found that the process was adequately described in 53% of neurological trials [8], 43% of surgical trials [3] and 27% of trials in multiple sclerosis [9]. An extreme example comes from the field of oral health in which only 15% of trials had low risk of bias for allocation concealment [10]. These studies suggest that as many as two third of trials are at risk of producing biased estimates of treatment.
Evidence that the Randomisation Process Is Subverted
Randomisation will usually produce two treatment groups with similar sample sizes, but only rarely will they have identical numbers of patients. A review found that many more trials had groups with identical sample sizes than could occur by chance [11]. The conclusion is that someone may have modified the randomisation sequence to prevent disparity in the size of the two groups, an action described as ‘forcing cosmetic credibility’ [11].
Randomisation should also result in the two groups being similar at baseline on characteristics such as age and clinical signs and symptoms. Small differences between groups commonly occur, but many trials have much more marked imbalances between groups than would be expected by chance. This has been documented for participant age [12] and for important clinical predictors of outcome [13]. These imbalances suggest that the randomisation sequence has been altered, with the likely consequence that the estimates of treatment effect will be biased.
A few studies have explored whether researchers admit to deciphering the randomisation sequence. Schulz and colleagues found that many clinicians try to decode the sequence [14]. Paludan‐Muller and colleagues [15] reviewed surveys of the reasons why clinicians do this. The most common reasons were that a doctor had a preference for a treatment for a particular patient, or had a desire to show that the new treatment was effective. Some researchers admitted to distorting the randomisation sequence by entering two or more patients at the same time, so that a particular patient could be allocated to a preferred treatment [15]. In some trials the treatment allocation codes are delivered in sealed envelopes [16], enabling some clinicians to subvert the randomisation by opening the envelopes before entering the patients [15]. Whatever the method of manipulation, deviations from random allocation could lead to bias.
Does Integrity of Allocation Concealment Matter?
A seminal paper by Schulz and colleagues in 1995 showed that studies that reported inadequate concealment treatment allocation exaggerate the estimates of treatment benefit [17]. The initial overview evaluated 250 trials of interventions in pregnancy and childbirth. Since then this finding has been replicated and extended by two overview studies covering thousands of trials across all areas of medicine [7, 18]. Their finding is that poor or inadequately described methods of allocation concealment lead to exaggerated treatment effects.
PROBLEMS IN MEASURING THE OUTCOME
The effectiveness
