Differences between protocol and review

The review protocol can be found in Appendix 2. We had consultancy support from Brenda Thomas, Cochrane Stroke Group Trials Search Co-ordinator, and on her advice the outline search strategy in the protocol was considerably extended to include, for example, EMBASE classic, the British Library’s Electronic Table of Contents (Zetoc), British Nursing Index (BNI) and BNI Archive, and Web of Science conference proceedings. Had time allowed we would also have included additional country-specific databases in addition to those in the protocol (e.g. WanFang, Panteleimon, IndMED, KoreaMed), Web of Science cited reference search (forward search) and more hand-searching. The formal patent search was omitted owing to lack of time. Of the head-cooling reports in the review (see Figure 1, which corresponds to the results of stage 2, trial identification and selection in the protocol) only studies that could potentially have been RCTs were screened, assessed and had data extracted by two reviewers.

Criteria for considering studies for this review

Search methods for identification of studies

Appendix 3 (search strategies) contains details of the searches and search terms. The searches were not restricted by publication status, date or language.

Data collection and analysis

Papers in languages other than English

A number of papers in foreign languages required full-text review: French (13), Italian (1), Slovakian (1), German (11), Japanese (3), Russian (8) and Chinese (26). Some of these had no, or an inadequate, English abstract so that it was not clear, for example, if the research was in humans or animals or whether head cooling or systemic cooling had been used without reading at least part of the paper. We had assistance from colleagues and friends with the requisite languages and used Google Translate (http://translate.google.com) to eliminate papers that were not relevant.

A Chinese-speaking intensive care doctor helped with the Chinese papers. She read them all, translated parts and went through them in detail with BH to assess quality and extract data. This did not highlight any that, on grounds of quality, warranted formal professional translation but we did have the study comparing head cooling with systemic cooling translated, as this was a particular comparison of interest with very few studies.⁴⁴ Because the other Chinese studies were not formally translated in full it has been possible only to report the main points and reasons for exclusion Appendix 6 (see Characteristics of Excluded Studies) compared with some of the studies in English where we have reported in more detail, although this is sometimes simply because there was more detail to report (the Chinese papers were mostly short).

Papers on head cooling in neonatal HIE in languages other than English were not assessed because this was not the primary condition of interest and there are recent systematic reviews (see Appendix 5, References to studies in neonatal hypoxic–ischaemic encephalopathy), which were also consulted for information on adverse effects of head-cooling methods and devices, i.e. the reason why papers on neonatal HIE were of interest.

Data extraction

BH and PA independently extracted data from RCTs using a standard form (see Appendix 4). They were not blinded to authors, journal or results. Disagreements were resolved by discussion. BH extracted data from all other studies. Where multiple reports of a trial were available, discrepancies between the reports were noted. Where there was missing information attempts were made to contact investigators.

Assessment of risk of bias

Randomised controlled trials were assessed for adequacy of the randomisation and allocation concealment process, potential for selection bias after allocation and level of masking (blinding of treatment provider, patient, outcome assessor, investigators and analysers of the data) (see Appendix 4).

Data synthesis

We were unable to carry out the full analysis plan specified in the protocol (see Appendix 2) because there were insufficient good-quality RCTs to undertake formal outcome analysis.

Briefly, had there been suitable RCT data, the following analysis was planned. For temperature data the difference in means would have been calculated with 95% confidence intervals (CIs). If sufficient good-quality trials for a meta-analysis had been found then a weighted mean difference would have been calculated. Pooled relative risk and 95% CIs for all-cause mortality and good neurological outcome would have been calculated using a random-effects model. Statistical heterogeneity would have been assessed using the chi-squared test.

However, it was recognised in the protocol that, depending on what was found, description of results might be all that was possible and the available temperature data are tabulated as a descriptive record of the effect of head cooling. No attempt has been made to draw any statistical inference. Data on adverse effects are reported descriptively.