the research team may handle different trials at different times, keeping track of the process and outputs can be challenging. Producing a detailed plan, and adopting a standardised approach to data checking and harmonisation, will help to ensure that the process is implemented consistently across trials, and between those managing the IPD. For example, using a checklist for all data checkers to follow, together with a common suite of statistical analysis code, can help to ensure and maintain consistency.
Regardless of the extent of checking and data transformation needed, it is always sensible to use formal database or statistical software code to carry out the different steps. The code and the associated outputs help to maintain a detailed log of the checks, and any conversions or modifications to the data, thereby providing a comprehensive and transparent audit trail for each trial. It is also important to record where checks have identified problems, how these were (or were not resolved), and equally to record where no problems were identified.
The information generated for each trial may be held on a number of forms, spreadsheets or as output from statistical software. A summary document is a useful means of bringing together the various elements of checking, querying and decision‐making, and might include hyperlinks to the different outputs, together with correspondence from trial teams. Where resource allows, ideally two individuals would independently check each trial, blinded to the other’s results, and compare and discuss the findings. At the very least, another research team member should review the checking results, and discuss problems arising. Any major or sensitive issues should be raised with senior research team members, prior to any dialogue with the trial investigators.
In the following sub‐sections, we suggest a range of checks for IPD obtained from randomised trials evaluating treatment effects,7,9,43,101 but most of these are applicable to other types of primary study.
4.5.2 Initial Checking of IPD for Each Trial
When IPD are received for a trial, and often before processing the data further, it is worth conducting some preliminary checks. For example, it is useful to confirm that all the participants randomised appear to have been included, and check that there are no obvious omissions or duplicates in the sequence of participant identifiers (if they have been provided). Similarly, it is helpful to check which outcomes, baseline covariates and other variables are included in the IPD, and whether any that are ‘missing’ were truly not collected in the trial (called systematically missing variables; Chapter 18) or were recorded, but not included in the IPD supplied. If the latter, then either more complete IPD should be requested again, or a full explanation for non‐provision sought.
For time‐to‐event outcomes, it is worth checking that the extent of follow‐up for each trial is sufficient for the condition and outcome of interest, and encouraging trial investigators to supply up‐to‐date follow‐up information where possible.7 Although short follow‐up may not introduce bias, it might prevent a trial, and therefore an IPD meta‐analysis, picking up benefits or harms of interventions that take a long time to accrue, such as late side effects of treatment or late recurrence of disease. For example, in an IPD meta‐analysis evaluating the addition of chemotherapy local treatment for soft tissue sarcoma,102 the median follow‐up for survival was reported for seven of the included trials as being between 16 and 64 months,46 which is rather short for this type of cancer. Trial investigators supplying IPD were asked to provide updated information, which extended the median follow‐up for these trials to between 74 and 204 months,46 and thus allowed a more reliable examination of the effects of chemotherapy in the long term (Figure 4.5).102 Such updating may not be necessary if most or all events have already occurred, and may not be feasible if, for example, studies are very old, no longer obtaining follow‐up information, or resources are limited.
Figure 4.5 Median follow‐up based on published aggregate data compared to updated obtained IPD for seven trials included in an IPD meta‐analysis of adjuvant chemotherapy for soft tissue sarcoma.46
Source: Stewart et al.46, © 2006, John Wiley & Sons.
While ideally these checks should make use of database or statistical code, the value of scanning the data by eye should not be underestimated, as it can help members of the central research team get a feel for the trial as a whole, and even highlight unusual patterns or peculiarities.
4.5.3 Harmonising IPD across Trials
If data providers have followed the supplied data dictionary closely when preparing their IPD, much of the data harmonisation will have been done already, and minor adjustments may be all that are required. If trial investigators are unable or unwilling to prepare data according to suggested pre‐specified formats, the central research team should accept data in whichever format is most convenient, and recode it as necessary.
Beyond simply aligning trial IPD to the data dictionary, there is also the opportunity to standardise definitions of outcomes or participant‐level variables,7,43 such as scoring or staging systems. For example, in an IPD meta‐analysis examining the effects of chemotherapy for soft tissue sarcoma,102 different definitions of histological grade were used in the included trials, but with input from trial investigators, it was possible to translate each of these into a high‐ or low‐grade disease category, allowing exploration of treatment effectiveness according to grade.43 It may also be necessary to construct new standardised variables for use in analyses. For example, in an IPD meta‐analysis of the effects of antenatal diet and physical activity on maternal and foetal outcomes, the research team collected data on each woman’s height, baseline weight and parity, as well as the gestational age at birth and foetal birthweight for each baby. This allowed researchers to generate a standardised meta‐analysis definition of ‘small for gestational age’ (< 10th centile), using a bulk birthweight centile calculator.103
4.5.4 Checking the Validity, Range and Consistency of Variables
IPD should be checked for invalid, outlying or implausible values,7,9,43 as these are the sorts of errors that can occasionally be missed during data input or checking of the original trials, or occasionally could arise as a result of variable recoding by the IPD meta‐analysis project team. Such checks might highlight, for example, records of unusually old or young patients, or those with abnormally high or low levels of important biomarkers, which need to be queried with trial teams. Of course, such apparently implausible values may turn out to be accurate. For example, in a trial included in an IPD meta‐analysis of neoadjuvant chemotherapy for cervical cancer,104 some invalid codes were used for the brachytherapy (internal radiotherapy) variable (Figure 4.6), which were then queried and clarified as representing not applicable. When variables are related, checks should be carried out to ensure that they are coherent and consistent. For example, in a randomised trial, the recorded date of any event should follow the recorded date of randomisation and occur on or before the recorded date of last follow‐up.
At this stage, it is also useful to perform a simple descriptive analysis of the IPD from each trial to provide, for
