4.11). In one of these projects examining the effects of chemotherapy for soft tissue sarcoma,112 the evidence for a benefit of chemotherapy on survival was stronger when it was based just on participants included in the original trial analyses (hazard ratio = 0.85; 95% CI: 0.72 to 1.00, p = 0.06), compared to when all possible participants were included (hazard ratio = 0.90, 95% CI: 0.77 to 1.04, p = 0.16).
Figure 4.11 Percentage of participants excluded from the original analyses of trials included in 14 IPD meta‐analysis projects in the cancer field.112
Source: Jayne Tierney, reproduced with permission from Tierney and Stewart.112
Therefore, it is important to check that data on all, or as many as possible, participants recruited to a trial are included in an IPD meta‐analysis (assuming that all the individuals meet the IPD meta‐analysis inclusion criteria for population and setting). If good records are maintained for a trial, it is possible to recover data on participants who were excluded from the original trial analyses, as part of the IPD collection process, and incorporate them into the meta‐analysis. For example, in the same 14 cancer meta‐analyses described previously, when IPD were collected, approximately 1,800 participants who had been excluded from the original trial analyses were re‐instated, and without them most meta‐analysis results would have been biased towards the research intervention, albeit to a small degree in most cases.112
A major advantage of IPD meta‐analysis is the ability to include all outcomes of relevance to the meta‐analysis, irrespective of whether they have been published or not, thereby overcoming the potential biases associated with differential reporting of outcomes,113 and providing a more balanced view of benefits and harms. For example, in a systematic review of laparoscopic versus open surgery for the repair of inguinal hernia, based on the available published aggregate data from three trials, the risk of persistent pain was found to be significantly greater with laparoscopic repair (odds ratio = 2·03, 95% CI: 1·03 to 4·01).114 However, when IPD were collected, data were available for a further 17 trials that had not published results for this outcome, and the combined meta‐analysis results showed that the risk of persistent pain was actually lower with laparoscopic repair (odds ratio = 0·54, 95% CI: 0·46 to 0·64). Recognising that some outcomes measured in trials may not be reported, it is always worth checking trial protocols, registry entries and with trial investigators to firmly establish which outcomes can be made available when IPD are provided.115
4.6.4 Measurement of the Outcome
Direct access to IPD does not usually allow the assessment of whether the measurement or ascertainment of outcomes differed between intervention groups. However, contact with trial investigators can provide useful clarification of the methods used, to help determine whether these are appropriate, and therefore allow the risk of bias associated with, for example, differential outcome assessments to be judged with more certainty. If the IPD are sufficiently detailed, outcomes may be defined more appropriately or consistently across trials. For example, a new standardised composite outcome might be constructed from a series of component variables.
For time‐to‐event outcomes, such as survival or time to relief of symptoms, bias can occur if certain participants are followed‐up for a longer duration than others, such that event rates appear higher. Therefore, for these outcomes, it is also desirable to check that follow‐up is balanced by treatment group. This can be achieved by selecting those trial participants who are event‐free, then using censoring as the event and the date of censoring as the time‐to‐event in a ‘reverse Kaplan‐Meier’ analysis. In one trial included in an IPD meta‐analysis of adjuvant chemotherapy for soft tissue sarcoma,102 all participants who were still alive had been followed for death for a minimum of about nine years, and subsequently to the same degree in both treatment groups (Figure 4.12(a)). In another smaller trial from the same meta‐analysis, although participants were followed for a long time, there was an imbalance by group (Figure 4.12(b)), which became less of an issue when the trial investigator provided updated IPD with extended follow‐up (Figure 4.12(c)).
Trials that stop early can produce results that are overly in favour of treatment or control,117 and therefore introduce bias into a subsequent meta‐analysis. For those based on time‐to‐event outcomes, obtaining IPD with updated follow‐up can go some way to addressing this issue. For example, in an IPD meta‐analysis examining adjuvant chemotherapy for locally advanced bladder cancer, three of the included trials were stopped early, because they had interim results in favour of adjuvant treatment. However, the IPD meta‐analysis project helped alleviate this potential bias, as it included IPD with updated follow‐up for the three trials, which produced results that were less in favour of adjuvant treatment (Table 4.5).118 IPD also allows non‐proportional hazards (non‐constant hazard ratios) to be examined (see Chapter 5).
Figure 4.12 ‘Reverse’ Kaplan‐Meier analysis of participants who are event‐free for (a) a trial with balanced follow‐up, and (b) a trial with imbalanced follow‐up that was (c) subsequently updated with longer follow‐up when IPD were collected. Each was included in an IPD meta‐analysis of adjuvant chemotherapy for soft tissue sarcoma.102
Source: Sarcoma Meta-Analysis Collaboration. Adjuvant chemotherapy for localised resectable soft-tissue sarcoma of adults: meta-analysis of individual data. Lancet 1997;350(9092):1647–54.
Table 4.5 Alleviating potential bias in trials that stopped early for perceived benefit (included in an IPD meta‐analysis of adjuvant chemotherapy for locally advanced bladder cancer) through updated follow‐up
Source: Jayne Tierney, adapted with permission.118
| Trial | Skinner | Studer | Stockle |
|---|---|---|---|
| Outcome analysed | Survival | Survival | Disease‐free survival |
| % participants with updated follow‐up since published analysis | 100 | 22 | 100 |
| Hazard ratio estimated from published statistics or Kaplan‐Meier curves | 0.65 | 0.86 | 0.39 |
