3.1. Developing review questions and protocols and identifying evidence

The technical team drafted review questions which were refined and validated by the GDG, using a Population, Intervention, Comparator, Outcome (PICO framework, and drafted review protocols based on the topics agreed with the stakeholders and included in the scope (see Appendix B) and prepared a protocol for each review question (see Appendix C). These formed the starting point for systematic reviews of relevant evidence. Published evidence was identified by applying systematic search strategies (see Appendix C) to the following databases: Medline (1950 onwards), Embase (1980 onwards), Cumulative Index to Nursing and Allied Health Literature (CINAHL; 1982 onwards), and three Cochrane databases (Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, and the Database of Abstracts of Reviews of Effects). Searches to identify economic studies were undertaken using the above databases, the NHS Economic Evaluation Database (NHS EED) and the Health Technology Assessment (HTA) database.

Where a question was updated directly from CG68 the search strategies used in the CG68 were updated. However for the review question on signs and symptoms, coexisting conditions and first-degree relatives and long-term consequences, the GDG requested some new search for additional terms and these additional searches had no date restriction. No date restrictions were placed on the searches for all new questions.

Searches in Embase and Medline were limited to English language and studies in humans. None of the other searches were limited by language of publication (although publications in languages other than English were not reviewed). Validated search filters were used to identify particular study designs, such as RCTs. There was no systematic attempt to search grey literature (conference abstracts, theses or unpublished trials), nor was hand searching undertaken of journals not indexed on the databases.

Towards the end of the guideline development process, the searches were updated and re-executed to include evidence published and indexed in the databases by 5^th December 2014. 2014.

3.2. Outcomes

The outcomes prioritised in the review questions and protocols reflect the treatment objectives outlined in each question. The minimum important difference (MID) for both dichotomous and continuous outcomes would be decided by looking at appropriate published evidence or under agreement with the GDG following discussion within committee meetings. On the occasion that no published literature on the minimal important difference was identified and the GDG were unable to specify on a default option was used, for example, in the case of dichotomous outcomes was defined as a relative risk reduction or an increase of 25% or more to be considered clinically important.

For this guideline, the effectiveness of interventions/diagnostic strategies to manage coeliac disease has been assessed against a variety of outcomes. The justification for using these outcomes is based on their relevance to people with the condition and the expert consensus opinion of members of the multidisciplinary GDG. When assessing the effectiveness of a particular treatment, information about the effect of that treatment on one or more primary outcomes was sought.

3.3. Process

3.3.4. GRADE process

The body of evidence identified for each therapy or treatment review question (or part of a review question) was presented in the form of a GRADE evidence profile summarising the quality of the evidence and the findings (pooled relative and absolute effect sizes and associated CIs). Where possible, the body of evidence corresponding to each outcome specified in the review protocol was subjected to quantitative meta-analysis. In such cases, pooled effect sizes were presented as pooled risk ratios (RRs), pooled odds ratios (ORs), or mean differences. A random-effects model was used as default.

Where quantitative meta-analysis could not be undertaken, the range of effect sizes reported in the included studies was presented in a GRADE profile.

GRADE was used to assess the quality of evidence for the selected outcomes as specified in ‘The guidelines manual (2012)’. The type of review question determines the highest level of evidence that may be sought. For issues of therapy or treatment, the highest possible evidence level is a well conducted systematic review or meta-analysis of RCTs, or an individual RCT. In the GRADE approach, a body of evidence based on RCTs has an initial quality rating of high, but this may be downgraded to moderate, low or very low if the factors listed above are not addressed adequately. For diagnostic review questions on prognosis, the highest possible level of evidence is a controlled observational study (a cohort study or case–control study), and a body of evidence based on such studies would have an initial quality rating of low, which might be downgraded to very low or upgraded to moderate or high, depending on the factors listed above.

For each review question the highest available level of evidence was sought. Where appropriate, for example, if a systematic review, meta-analysis or RCT was identified to answer a question directly, studies of a weaker design were not considered. Where systematic reviews, meta-analyses and RCTs were not identified, other appropriate experimental or observational studies were sought.

GRADE profiles for interventional evidence

The quality ratings for each study are reported the study’s evidence table and are summarised in the footnotes of each GRADE profile. For this guideline, we inserted footnotes to explain the choice we made while assessing the quality of evidence for each outcomes. These footnotes indicated if we upgraded the evidence level, downgraded the evidence level or left the evidence level unchanged, and gave the rationale for doing this.

The quality of the evidence for each outcome was downgraded where appropriate for the reasons outlined in Table 1.

Table 1

Rationale for downgrading quality of evidence for intervention studies.

3.3.4.1. Modified GRADE for diagnostic evidence

GRADE has not been developed for use with diagnostic studies; therefore a modified approach was applied using the GRADE framework.

Cohort studies within the GRADE approach start at the low quality level due to accepted inherent study design limitations. Within a modified approach, where evidence from cohort studies has been deemed to be the most appropriate source of information to answer a given review question, studies start from a presumption of ‘high quality’ The same criteria (risk of bias, inconsistency, imprecision and indirectness) were used to downgrade the quality of evidence as detailed in Table 2 below.

Table 2

Rationale for downgrading quality of evidence for diagnostic questions.

3.3.4.2. Modified GRADE for qualitative studies

GRADE has not been developed for use with qualitative studies; therefore a modified approach was applied using the GRADE framework.

Qualitative studies within the non-modified GRADE approach start at the very low quality level due to accepted inherent study design limitations. Within a modified approach where qualitative evidence has been deemed to be the most appropriate source of information to answer a given review question, it is acceptable to initially indicate a high quality level to this study type and to assess the quality of evidence from this point. The same criteria (risk of bias, inconsistency, imprecision and indirectness) were used to downgrade the quality of evidence as detailed in Table 3 below.

Table 3

Rationale for downgrading quality of evidence for diagnostic questions.

3.4. Health economics

Literature reviews seeking to identify published cost–utility analyses of relevance to the issues under consideration were conducted for all questions, with the exception of those detailed in sections 4.1, 4.2 and 4.3 (which provided data for the health economic question considered in section 4.4) and 7.1 and 7.2 (which were information questions without a substantive health economic component). In each case, the search undertaken for the clinical review was modified, retaining population and intervention descriptors, but removing any study-design filter and adding a filter designed to identify relevant health economic analyses. Search strategies are provided in full in Appendix C. In assessing studies for inclusion, population, intervention and comparator criteria were always identical to those used in the parallel clinical search; only cost–utility analyses were included. Economic evidence profiles, including critical appraisal according to the Guidelines manual, were completed for included studies; these are shown in Appendix G.

Economic studies identified through a systematic search of the literature are appraised using a methodology checklist designed for economic evaluations (NICE 2012; Appendix E). This checklist is not intended to judge the quality of a study per se, but to determine whether an existing economic evaluation is useful to inform the decision-making of the GDG for a specific topic within the guideline. There are two parts of the appraisal process; the first step is to assess applicability (i.e. the relevance of the study to the specific guideline topic and the NICE reference case) (Table 4).

Table 4

Applicability criteria.

In the second step, only those studies deemed directly or partially applicable are further assessed for limitations (i.e. the methodological quality, Table 5).

Table 5

Methodological criteria.

Where relevant, a summary of the main findings from the systematic search, review and appraisal of economic evidence is presented in an economic evidence profile alongside the clinical evidence.

Original health economic modelling was conducted for 3 questions that were prioritised by the GDG for detailed analysis: order and sequencing of serological tests (see section 5.2), active case-finding (see section 4.4) and dietetic involvement in follow-up (considered as part of the review on frequency of follow-up; see section 5.4. Each analysis relied on broadly the same model, which was originally developed for the serological testing question and subsequently modified to address other questions. Full details of the methods of the models are provided in Appendix G.

In questions for which no published evidence was identified and original analysis was not prioritised, the GDG made a qualitative judgement about cost effectiveness by considering potential differences in resource use and cost between the options alongside the results of the review of evidence of clinical effectiveness

3.5. Agreeing the recommendations

For each review question, recommendations for clinical care were derived using, and linked explicitly to, the evidence that supported them. In the first instance, informal consensus methods were used by the guideline development group to agree short clinical and, where appropriate, cost effectiveness evidence statements, which were presented alongside the evidence profiles. Statements summarising the guideline development group’s interpretation of the evidence and any extrapolation from the evidence used to form recommendations were also prepared to ensure transparency in the decision-making process. The ‘Linking evidence to recommendations’ (LETR) criteria used in moving from evidence to recommendations were:

• relative value placed on the outcomes considered
• consideration of the clinical benefits and harms
• consideration of net health benefits and resource use
• quality of the evidence
• other considerations (including equalities issues).

In areas where no substantial clinical research evidence was identified, the guideline development group considered other evidence-based guidelines and consensus statements or used their collective experience to identify good practice. The health economics justification in areas of the guideline where the use of NHS resources (interventions) was considered was based on guideline development group consensus in relation to the likely cost effectiveness implications of the recommendations. The guideline development group also identified areas where evidence to answer their review questions was lacking and used this information to formulate recommendations for future research

The wording used in the recommendations in this guideline denotes the certainty with which the recommendations were made. Some recommendations were made with more certainty than others. Recommendations are based on the trade-off between the benefits and harms of an intervention, whilst taking into account the quality of the underpinning evidence.

For all recommendations, it is expected that a discussion will take place with the patients about the risks and benefits of the interventions, and their values and preferences. This discussion should help the patient reach a fully informed decision. Terms used within this guideline are:

• ‘Offer’ – for the vast majority of patients, an intervention will do more good than harm
• ‘Do not offer’ – the intervention will not be of benefit for most patients
• Consider’ – the benefit is less certain, and an intervention will do more good than harm for most patients. The choice of intervention, and whether or not to have the intervention at all, is more likely to depend on the patient’s values and preferences than for an ‘offer’ recommendation, and so the healthcare professional should spend more time considering and discussing the options with the patient.

Towards the end of the guideline development process, formal consensus methods were used to consider all the clinical care recommendations and research recommendations that had been drafted previously. The guideline development group identified up to 10 ‘key priorities for implementation’ (key recommendations) and 5 high-priority research recommendations.