of methods for: accounting for clustering of participants within trials, quantifying and exploring heterogeneity, modelling participant‐level variables (and any interactions), and any sensitivity analyses. These should be specified in sufficient detail in the protocol or in a separate statistical analysis plan (SAP), in order to permit scrutiny and, if necessary, replication (through access to the same IPD)
An initial power calculation or sample size justification, conditional on assuming IPD are available from particular trials
An approximate timetable for the project
Details of members of the central research team, the project advisory group or steering committee, patient and public involvement representatives and funders
A dissemination plan
A provisional table of eligible studies
Appendices of, for example, search strategies for individual databases and the data dictionary
Source: Jayne Tierney and Lesley Stewart.
As for other clinical studies, an additional statistical analysis plan (SAP) may be required, in which case, the protocol should focus on the general methods and a summary of the planned analyses, whilst the SAP would provide more detailed explanation of statistical methods and modelling techniques (see guidance in Part 2). The anticipated scale and complexity of the project will help guide whether the protocol is sufficient, or whether an additional SAP is needed.
Although the protocol and SAP should pre‐specify the analyses in some detail, this does not preclude analyses that become necessary to further explore, explain, or add to the main findings. The protocol should make it clear that such analyses may be needed, and will be labelled as post‐hoc or exploratory in any report or publication, to distinguish them clearly from planned per‐protocol analyses. Furthermore, what is planned at the outset may need to be modified according to, for example, emerging information about data availability (including variables recorded in each trial’s IPD), or the identification of new factors that may be prognostic or interact with a treatment effect. Therefore, it can be useful to produce several versions of the protocol and/or SAP, to allow for the incorporation of new proposals and suggestions, either from the project advisory group or trial investigators, or to deal with issues of data availability. Given the inevitable concerns about independence of trial investigators and potential conflicts of interest, it is important to maintain a date‐stamped log of such protocol amendments, including who proposed and sanctioned any substantive ones.
For the benefit of potential collaborators, it is useful to include information on how the project will be managed, alongside a timetable and plans for the dissemination of results. Also, as the first full draft of the protocol will usually be completed after the searches for trials and subsequent screening have been completed (Section 4.2.3), this should include a table of the eligible trials identified to date. For this reason, developing the full draft protocol and undertaking searching and screening often occur concurrently.
The IPD meta‐analysis protocol should be registered in a publicly accessible registry such as PROSPERO (https://www.crd.york.ac.uk/prospero/) when a comprehensive draft is available (e.g. the version to be circulated to trial investigators for comment). If subsequent amendments are needed, this can be reflected in updates to the registration record, with the reason for changes captured and date‐stamped in the associated audit trail held within the registry.
4.2.3 Identifying and Screening Potentially Eligible Trials
Identification of eligible trials should be based on a systematic and comprehensive search of a number of sources, to ensure that all relevant trials are identified, using the same or similar methods to those employed in a conventional systematic review of existing aggregate data.92 However, IPD meta‐analysis projects often have a greater emphasis on searching grey literature sources, such as conference proceedings, as well as trial registers to identify unpublished trials, and any ongoing trials that may complete in time for inclusion in the IPD meta‐analysis.7,9,43 For example, in an IPD meta‐analysis project examining the effects of chemoradiation for cervical cancer,93 about 25% of the included trials were unpublished or published only as an abstract. Although there may be initial concerns about the quality of unpublished trials, this can be formally evaluated (Sections 4.5 and 4.6), and having the accompanying protocol, case report forms and trial IPD enables more detailed quality checking than when relying on aggregate data reported in publications. Indeed, a high standard of reporting does not necessarily correspond to high‐quality trial design and conduct, and similarly, some good‐quality trials can be quite badly reported.
Trial investigators should be asked whether they can supplement the provisional list of eligible trials, and additional appeals might also be made via project websites, social media or conference presentations. A summary of the sources searched for an IPD meta‐analysis of recombinant human bone morphogenetic protein‐2 for spinal fusion is given in Box 4.2.
Box 4.2 Summary of the sources searched for an IPD meta‐analysis of randomised trials examining recombinant human bone morphogenetic protein‐2 for spinal fusion.
Cochrane Central Register of Controlled Trials
MEDLINE 1948 to present
EMBASE 1974 to present
Science Citation Index 1899 to present
Automated “current awareness” searches to June 2012
ClinicalTrials.gov (to identify ongoing or unpublished randomised trials)
Published a call for evidence
Source: Lesley Stewart, listing the sources used by Simmonds et al.,65 with permission.
The screening process for deciding which trials are relevant for inclusion is very similar to that for standard systematic reviews.92 However, with IPD meta‐analysis projects, because there is usually greater contact with trial investigators, any doubt as to the eligibility of a particular trial (e.g. in relation to whether particular variables or outcomes are recorded in the trial’s IPD) can be clarified through discussion. This process should be well documented, so that it can be used to help populate a PRISMA‐IPD flow diagram (Chapter 10).60
4.2.4 Deciding Which Information Is Needed to Summarise Trial Characteristics
After relevant trials are identified, it is important to obtain a good understanding of the attributes and characteristics of these trials, for descriptive purposes. This may include contacting trial investigators to request extra information about the trial population and treatment and control interventions. In addition, gathering
