NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Collaborating Centre for Mental Health (UK). Self-Harm: Longer-Term Management. Leicester (UK): British Psychological Society; 2012. (NICE Clinical Guidelines, No. 133.)

Cover of Self-Harm: Longer-Term Management

Self-Harm: Longer-Term Management.

Show details

3METHODS USED TO DEVELOP THIS GUIDELINE

3.1. OVERVIEW

The development of this guideline drew upon methods outlined by NICE (further information is available in The Guidelines Manual [NICE, 2009d]). A team of health professionals, lay representatives and technical experts known as the Guideline Development Group (GDG), with support from the NCCMH staff, undertook the development of a patient-centred, evidence-based guideline. There are six basic steps in the process of developing a guideline:

  1. Define the scope, which sets the parameters of the guideline and provides a focus and steer for the development work.
  2. Define review questions considered important for practitioners and service users.
  3. Develop criteria for evidence searching and search for evidence.
  4. Design validated protocols for systematic review and apply to evidence recovered by search.
  5. Synthesise and (meta-) analyse data retrieved, guided by the review questions, and produce GRADE evidence profiles and summaries.
  6. Answer review questions with evidence-based recommendations for clinical practice.

The clinical practice recommendations made by the GDG are therefore derived from the most up-to-date and robust evidence base for the clinical and cost effectiveness of the treatments and services used in the longer-term management of self-harm. In addition, to ensure a service user and carer focus, the concerns of service users and carers regarding health and social care have been highlighted and addressed by recommendations agreed by the whole GDG.

3.2. THE SCOPE

Guideline topics are selected by the Department of Health and the Welsh Assembly Government, which identify the main areas to be covered by the guideline in a specific remit (see The Guidelines Manual [NICE, 2009d] for further information). The NCCMH developed a scope for the guideline based on the remit. The purpose of the scope is to:

  • provide an overview of what the guideline will include and exclude
  • identify the key aspects of care that must be included
  • set the boundaries of the development work and provide a clear framework to enable work to stay within the priorities agreed by NICE and the National Collaborating Centre, and the remit from the Department of Health/Welsh Assembly Government
  • inform the development of the review questions and search strategy
  • inform professionals and the public about expected content of the guideline
  • keep the guideline to a reasonable size to ensure that its development can be carried out within the allocated period.

An initial draft of the scope was sent to registered stakeholders who had agreed to attend a scoping workshop. The workshop was used to:

  • obtain feedback on the selected key clinical issues
  • identify which patient or population subgroups should be specified (if any)
  • seek views on the composition of the GDG
  • encourage applications for GDG membership.

The draft scope was subject to consultation with registered stakeholders over a 4-week period. During the consultation period, the scope was posted on the NICE website (www.nice.org.uk). Comments were invited from stakeholder organisations and the Guideline Review Panel. Further information about the Guideline Review Panel can also be found on the NICE website. The NCCMH and NICE reviewed the scope in light of comments received, and the revised scope was signed off by the Guideline Review Panel.

3.3. THE GUIDELINE DEVELOPMENT GROUP

The GDG consisted of: professionals in psychiatry, clinical psychology, nursing, social work and general practice; academic experts in psychiatry and psychology; and a service user, carer representatives and representatives from service user organisations. The carer perspective was provided through topic group discussion with carers. The service user topic group meetings were coordinated between staff from NCCMH, the service user and the carer representatives. The guideline development process was supported by staff from the NCCMH, who undertook the clinical and health economics literature searches, reviewed and presented the evidence to the GDG, managed the process and contributed to drafting the guideline.

3.3.1. Guideline Development Group meetings

Thirteen GDG meetings were held between November 2009 and June 2010. During each day-long GDG meeting, in a plenary session, review questions and clinical and economic evidence were reviewed and assessed, and recommendations formulated. At each meeting, all GDG members declared any potential conflicts of interest, and service user and carer concerns were routinely discussed as part of a standing agenda.

3.3.2. Service users and carers

Individuals with direct experience of services gave an integral service-user focus to the GDG and the guideline. The GDG included a service user and representatives of a national service user group. They contributed as full GDG members to writing the review questions, helping to ensure that the evidence addressed their views and preferences, highlighting sensitive issues and terminology relevant to the guideline, and bringing service-user research to the attention of the GDG. In drafting the guideline, they contributed to writing the guideline's introduction and Chapter 4, and identified recommendations from the service user and carer perspective.

3.3.3. Special advisors

Special advisors, who had specific expertise in one or more aspects of treatment and management relevant to the guideline, assisted the GDG, commenting on specific aspects of the developing guideline and making presentations to the GDG. Appendix 3 lists those who agreed to act as special advisors.

3.3.4. National and international experts

National and international experts in the area under review were identified through the literature search and through the experience of the GDG members. These experts were contacted to recommend unpublished or soon-to-be published studies, to ensure that up-to-date evidence was included in the development of the guideline. They informed the group about completed trials at the pre-publication stage, systematic reviews in the process of being published, studies relating to the cost effectiveness of treatment and trial data if the GDG could be provided with full access to the complete trial report. Appendix 6 lists researchers who were contacted.

3.4. REVIEW QUESTIONS

Review (clinical) questions were used to guide the identification and interrogation of the evidence base relevant to the topic of the guideline. Before the first GDG meeting, an analytic framework (see Appendix 7) was prepared by NCCMH staff based on the scope and an overview of existing guidelines, and was discussed with the guideline Chair. The framework was used to provide a structure from which the review questions were drafted. Both the analytic framework and the draft review questions were then discussed by the GDG at the first few meetings and amended as necessary. Where appropriate, the framework and questions were refined once the evidence had been searched and, where necessary, sub-questions were generated. Questions submitted by stakeholders were also discussed by the GDG and the rationale for not including any questions was recorded in the minutes. The final list of review questions can be found in Appendix 8.

For questions about interventions, the PICO (patient, intervention, comparison and outcome) framework was used (see Table 1).

Table 1. Features of a well-formulated question on effectiveness intervention – the PICO guide.

Table 1

Features of a well-formulated question on effectiveness intervention – the PICO guide.

For questions that were not related to effectiveness (intervention studies), a different question format was used. Please see the question formats in the review protocols (Appendix 8).

To help facilitate the literature review, a note was made of the best study design type to answer each question. There are four main types of review question of relevance to NICE guidelines. These are listed in Table 2. For each type of question, the best primary study design varies, where ‘best’ is interpreted as ‘least likely to give misleading answers to the question’.

Table 2. Best study design to answer each type of question.

Table 2

Best study design to answer each type of question.

However, in all cases, a well-conducted systematic review (of the appropriate type of study) is likely to always yield a better answer than a single study.

Deciding on the best design type to answer a specific review question does not mean that studies of different design types addressing the same question were discarded.

3.5. SYSTEMATIC CLINICAL LITERATURE REVIEW

The aim of the clinical literature review was to systematically identify and synthesise relevant evidence from the literature in order to answer the specific review questions developed by the GDG. Thus, clinical practice recommendations are evidence-based, where possible, and if evidence is not available informal consensus methods are used (see Section 3.5.6) and the need for future research is specified.

3.5.1. Methodology

A stepwise hierarchical approach was taken to locating and presenting evidence to the GDG. The NCCMH developed this process based on methods set out by NICE (The Guidelines Manual [NICE, 2009d]), and after considering recommendations from a range of other sources. These included:

  • BMJ (British Medical Journal) Clinical Evidence
  • Clinical Policy and Practice Program of the New South Wales Department of Health (Australia)
  • The Cochrane Collaboration
  • New Zealand Guidelines Group
  • NHS Centre for Reviews and Dissemination (CRD)
  • Oxford Centre for Evidence-Based Medicine
  • Oxford Systematic Review Development Programme
  • Scottish Intercollegiate Guidelines Network
  • United States Agency for Healthcare Research and Quality.

3.5.2. The review process

Scoping searches

A broad preliminary search of the literature was undertaken in July 2009 to obtain an overview of the issues likely to be covered by the scope, and to help define key areas. Searches were restricted to clinical guidelines, health technology assessment (HTA) reports, key systematic reviews and RCTs, and conducted in the following databases and websites:

  • BMJ Clinical Evidence
  • Canadian Medical Association Infobase (Canadian guidelines)
  • Clinical Policy and Practice Program of the New South Wales Department of Health (Australia)
  • Clinical Practice Guidelines (Australian Guidelines)
  • Cochrane Central Register of Controlled Trials (CENTRAL)
  • Cochrane Database of Abstracts of Reviews of Effects
  • Cochrane Database of Systematic Reviews (CDSR)
  • Excerpta Medical Database (EMBASE)
  • Guidelines International Network
  • Health Evidence Bulletin Wales
  • Health Management Information Consortium (HMIC)
  • HTA database (technology assessments)
  • Medical Literature Analysis and Retrieval System Online (MEDLINE)/MEDLINE in Process
  • National Health and Medical Research Council
  • National Library for Health Guidelines Finder
  • New Zealand Guidelines Group
  • NHS CRD
  • OmniMedicalSearch
  • Scottish Intercollegiate Guidelines Network
  • Turning Research Into Practice
  • United States Agency for Healthcare Research and Quality
  • Websites of NICE and the National Institute for Health Research HTA Programme for guidelines and HTAs in development.

Existing NICE guidelines were updated where necessary. Other relevant guidelines were assessed for quality using the AGREE instrument (AGREE Collaboration, 2003). The evidence base underlying high-quality existing guidelines was utilised and updated as appropriate. Further information about this process can be found in The Guidelines Manual (NICE, 2009d).

Systematic literature searches

After the scope was finalised, a systematic search strategy was developed to locate all the relevant evidence. Searches were conducted in the following databases:

  • Cumulative Index to Nursing and Allied Health Literature (CINAHL)
  • EMBASE
  • MEDLINE/MEDLINE In-Process
  • Cochrane Database of Abstracts of Reviews of Effects
  • CDSR
  • CENTRAL
  • HTA database
  • Health Management Information Consortium
  • International Bibliography of the Social Sciences
  • American Psychiatric Association Psychological Information Database (PsycINFO)
  • PsycEXTRA (see under PsycNET in abbreviations)
  • PsycBOOKS (see under PsycNET in abbreviations)

The search strategies were initially developed for MEDLINE before being translated for use in other databases/interfaces. Strategies were built up through a number of trial searches, and discussions of the results of the searches with the review team and GDG, to ensure that all possible relevant search terms were covered. To assure comprehensive coverage, search terms for self-harm were kept purposely broad to help counter dissimilarities in database indexing practices and imprecise reporting of study populations by authors in the titles and abstracts of records.

Reference manager

Citations from each search were downloaded into Reference Manager (a software product for managing references and formatting bibliographies) and duplicates removed. Records were then screened against the inclusion criteria of the reviews before being quality appraised (see below). The unfiltered search results were saved and retained for future potential re-analysis to help keep the process both replicable and transparent.

Search filters

To aid retrieval of relevant and sound evidence, study design filters were, where appropriate, used to limit searches to systematic reviews, RCTs and observational studies. The systematic review and RCT filters are adaptations of pre-tested strategies designed by the CRD and the Health Information Research Unit of McMaster University, Ontario. The observational study filter was developed in-house. The filters, which comprise a combination of controlled vocabulary and free-text retrieval methods, maximise sensitivity (or recall) to ensure that as many potentially relevant records as possible are retrieved from a search.

Date and language restrictions

Systematic database searches were initially conducted in March 2010 up to the most recent searchable date. Search updates were generated on a 6-monthly basis, with the final re-runs carried out in January 2011 ahead of the guideline consultation. After this point, studies were only included if they were judged by the GDG to be exceptional (for example, if the evidence was likely to change a recommendation).

Although no language restrictions were applied at the searching stage, foreign language studies were not requested or reviewed unless they were of particular importance to a review question. Date restrictions were applied to searches for systematic reviews and updates of published reviews only (see Appendix 9). No date restrictions were imposed for the remainder of the searches.

Other search methods

Other search methods involved were: (1) scanning the reference lists of all eligible publications (systematic reviews, stakeholder evidence and included studies) for more published reports and citations of unpublished research; (2) sending lists of studies meeting the inclusion criteria to subject experts (identified through searches and the GDG) and asking them to check the lists for completeness, and to provide information of any published or unpublished research for consideration (see Appendix 6); (3) checking the tables of contents of key journals for studies that might have been missed by the database and reference list searches; (4) tracking key studies in the Science Citation Index (prospectively) over time for further useful references.

Full details of the search strategies and filters used for the systematic review of clinical evidence are provided in Appendix 9.

Study selection and quality assessment

All primary-level studies included after the first scan of citations were acquired in full and re-evaluated for eligibility at the time when they were being entered into the study information database. More specific eligibility criteria were developed for each review question and are described in the relevant clinical evidence chapters. Eligible systematic reviews and primary-level studies were critically appraised for methodological quality (see Appendix 11 for quality checklist templates). The eligibility of each study was confirmed by at least one member of the GDG.

For some review questions, it was necessary to prioritise the evidence with respect to the UK context (that is, external validity). To make this process explicit, the GDG took into account the following factors when assessing the evidence:

  • participant factors (for example, gender, age and ethnicity)
  • provider factors (for example, model fidelity, the conditions under which the intervention was performed and the availability of experienced staff to undertake the procedure)
  • cultural factors (for example, differences in standard care and differences in the welfare system).

It was the responsibility of the GDG to decide which prioritisation factors were relevant to each review question in light of the UK context and then decide how they should modify their recommendations.

Unpublished evidence

The GDG used a number of criteria when deciding whether or not to accept unpublished data. First, the evidence must have been accompanied by a trial report containing sufficient detail to properly assess the quality of the data. Second, the evidence must have been submitted with the understanding that data from the study and a summary of the study's characteristics would be published in the full guideline. Therefore, the GDG did not accept evidence submitted as commercial in confidence. However, the GDG recognised that unpublished evidence submitted by investigators might later be retracted by those investigators if the inclusion of such data would jeopardise publication of their research.

3.5.3. Data extraction

Study characteristics and outcome data were extracted from all eligible studies that met the minimum quality criteria and were meta-analysed, using a bespoke database and Review Manager 5.0.25 (Cochrane Collaboration, 2011) and/or Word-based forms (see Appendix 11 for quality checklist templates).

In most circumstances, for a given outcome (continuous and dichotomous) where more than 50% of the number randomised to any group were lost to follow-up, the data were excluded from the analysis (except for the outcome ‘leaving the study early’, in which case the denominator was the number randomised). Where possible, dichotomous efficacy outcomes were calculated on an intention-to-treat (ITT) basis (that is, a ‘once-randomised-always-analyse’ basis). Where there was good evidence that those participants who ceased to engage in the study were likely to have an unfavourable outcome, early withdrawals were included in both the numerator and denominator. Adverse effects were entered into Review Manager as reported by the study authors because it is usually not possible to determine whether early withdrawals had an unfavourable outcome. Where there was limited data for a particular review, the 50% rule was not applied. In these circumstances the evidence was downgraded due to the risk of bias.

Where some of the studies failed to report standard deviations (SDs) (for a continuous outcome) and where an estimate of the variance could not be computed from other reported data or obtained from the study author, the following approach was taken2.

When the number of studies with missing SDs was less than one third and when the total number of studies was at least ten, the pooled SD was imputed (calculated from all the other studies in the same meta-analysis that used the same version of the outcome measure). In this case, the appropriateness of the imputation was made by comparing the standardised mean differences (SMDs) of those trials that had reported SDs against the hypothetical SMDs of the same trials based on the imputed SDs. If they converged, the meta-analytical results were considered to be reliable.

When the conditions above could not be met, SDs were taken from another related systematic review (if available). In this case, the results were considered to be less reliable.

The meta-analysis of survival data was based on log hazard ratios and standard errors. Because individual patient data were not available in included studies, hazard ratios and standard errors calculated from a Cox proportional hazard model were extracted. Where necessary, standard errors were calculated from confidence intervals (CIs) or p-values according to standard formulae (see the Cochrane Handbook for Systematic Reviews of Interventions, version 5.0.2, Higgins et al., 2009). Data were summarised using the generic inverse variance method using Review Manager.

Consultation with another reviewer or members of the GDG was used to overcome difficulties with coding. Data from studies included in existing systematic reviews were extracted independently by one reviewer and cross-checked with the existing data set. Where possible, two independent reviewers extracted data from new studies. Where double data extraction was not possible, data extracted by one reviewer was checked by the second reviewer. Disagreements were resolved through discussion. Where consensus could not be reached, a third reviewer or GDG members resolved the disagreement. Masked assessment (that is, blind to the journal from which the article comes, the authors, the institution and the magnitude of the effect) was not used because it is unclear that doing so reduces bias (Berlin, 2001; Jadad et al., 1996).

3.5.4. Synthesising the evidence

Meta-analysis

Where possible, meta-analysis was used to synthesise the evidence using Review Manager. If necessary, reanalyses of the data or sub-analyses were used to answer review questions not addressed in the original studies or reviews.

Dichotomous outcomes were analysed as relative risks (RR) with the associated 95% CI (for an example, see Figure 1). A relative risk (also called a risk ratio) is the ratio of the treatment event rate to the control event rate. An RR of 1 indicates no difference between treatment and control. In Figure 1, the overall RR of 0.73 indicates that the event rate (that is, non-remission rate) associated with intervention A is approximately three quarters of that with the control intervention or, in other words, the RR reduction is 27%.

Figure 1. Example of a forest plot displaying dichotomous data.

Figure 1

Example of a forest plot displaying dichotomous data.

The CI shows a range of values within which there is 95% confidence that the true effect will lie. If the effect size has a CI that does not cross the ‘line of no effect’, then the effect is commonly interpreted as being statistically significant.

Continuous outcomes were analysed using the mean difference, or SMD when different measures were used in different studies to estimate the same underlying effect (for an example, see Figure 2). If reported by study authors, ITT data, using a valid method for imputation of missing data, were preferred over data only from people who completed the study.

Figure 2. Example of a forest plot displaying continuous data.

Figure 2

Example of a forest plot displaying continuous data.

Heterogeneity

To check for consistency of effects among studies, both the I2 statistic and the chi-squared test of heterogeneity, as well as a visual inspection of the forest plots were used. The I2 statistic describes the proportion of total variation in study estimates that is due to heterogeneity (Higgins & Thompson, 2002). The I2 statistic was interpreted in the follow way based on Higgins and Green (2009):

  • 0 to 40%: might not be important
  • 30 to 60%: may represent moderate heterogeneity
  • 50 to 90%: may represent substantial heterogeneity
  • 75 to 100%: considerable heterogeneity.

Two factors were used to make a judgement about importance of the observed value of I2: first, the magnitude and direction of effects, and second, the strength of evidence for heterogeneity (for example, p-value from the chi-squared test or CI for I2).

Publication bias

Where there was sufficient data, the intention was to use funnel plots to explore the possibility of publication bias. Asymmetry of the plot would be taken to indicate possible publication bias and investigated further.

Where necessary, an estimate of the proportion of eligible data that were missing (because some studies did not include all relevant outcomes) was calculated for each analysis.

3.5.5. Presenting the data to the Guideline Development Group

Study characteristics tables and, where appropriate, forest plots that had been generated with Review Manager were presented to the GDG.

Where meta-analysis was not appropriate and/or possible, the reported results from each primary-level study were included in the study characteristics table and, where appropriate, in a narrative synthesis. Details of studies that were meta-analysed can be found in study characteristics tables in Appendix 15; studies that were only narratively reviewed can be found in the References.

Evidence profile tables

A GRADE3 evidence profile was used to summarise both the quality of the evidence and the results of the evidence synthesis (see Table 3 for an example of an evidence profile). The GRADE approach is based on a sequential assessment of the quality of evidence, followed by judgment about the balance between desirable and undesirable effects, and subsequent decision about the strength of a recommendation.

Table 3. Example of GRADE evidence profile.

Table 3

Example of GRADE evidence profile.

For each outcome, quality may be reduced depending on the following factors:

  • study design (randomised trial, observational study, or any other evidence)
  • limitations (based on the quality of individual studies)
  • inconsistency (see Section 3.5.4 for how consistency was assessed)
  • indirectness (that is, how closely the outcome measures, interventions and participants match those of interest)
  • imprecision (based on the CI around the effect size).

For observational studies, the quality may be increased if there is a large effect, plausible confounding would have changed the effect, or there is evidence of a dose–response gradient (details would be provided under the other considerations column). Each evidence profile also included a summary of the findings: number of patients included in each group, an estimate of the magnitude of the effect and the overall quality of the evidence for each outcome.

3.5.6. Method used to answer a review question in the absence of appropriately designed, high-quality research

In the absence of appropriately designed, high-quality research or where the GDG were of the opinion (on the basis of previous searches or their knowledge of the literature) that there was unlikely to be such evidence, an informal consensus process was adopted. This process focused on those questions that the GDG considered a priority.

Informal consensus

The starting point for the process of informal consensus was that a member of the GDG identified, with help from the systematic reviewer, a narrative synthesis or key study that most directly addressed the review question. Where this was not possible, a brief review of the recent literature was initiated. These were then used as a basis for beginning an iterative process to identify lower levels of evidence relevant to the review question and to lead to written statements for the guideline. The process involved a number of steps:

  1. A description of what is known about the issues concerning the clinical question was written by one of the GDG members.
  2. Evidence from the existing studies was then presented in narrative form to the GDG and further comments were sought about the evidence and its perceived relevance to the review question.
  3. Based on the feedback from the GDG, additional information was sought and added to the information collected. This included studies that did not directly address the review question but were thought to contain relevant data.
  4. A summary of statements that directly addressed the review question were then developed.
  5. Following this, on occasion and as deemed appropriate by the development group, the report was then sent to appointed experts outside of the GDG for peer review and comment. The information from this process was then fed back to the GDG for further discussion of the statements
  6. Recommendations were then developed and could also be sent for further external peer review.
  7. After this final stage of comment, the statements and recommendations were again reviewed and agreed upon by the GDG.

3.5.7. Forming the clinical summaries and recommendations

Once the GRADE evidence profiles relating to a particular review question were completed, summary evidence tables were developed (these tables are presented in the evidence chapters). Finally, the systematic reviewer in conjunction with the GDG produced a clinical evidence summary.

After the GRADE profiles and clinical summaries were presented to the GDG, the associated recommendations were drafted. In making recommendations, the GDG took into account the trade-off between the benefits and downsides of treatment as well as other important factors, such as economic considerations, social value judgements4, the requirements to prevent discrimination and to promote equality5, and the group's awareness of practical issues (Eccles et al., 1998; NICE, 2009d).

Finally, to show clearly how the GDG moved from the evidence to the recommendations, each chapter has a section called ‘from evidence to recommendations’. Underpinning this section is the concept of the ‘strength’ of a recommendation (Schunemann et al., 2003). This takes into account the quality of the evidence but is conceptually different. Some recommendations are ‘strong’ in that the GDG believes that the vast majority of healthcare professionals and service users would choose a particular intervention if they considered the evidence in the same way that the GDG has. This is generally the case if the benefits clearly outweigh the harms for most people and the intervention is likely to be cost effective. However, there is often a closer balance between benefits and harms, and some service users would not choose an intervention whereas others would. This may happen, for example, if some service users are particularly averse to some side effect and others are not. In these circumstances the recommendation is generally weaker, although it may be possible to make stronger recommendations about specific groups of service users. The strength of each recommendation is reflected in the wording of the recommendation, rather than by using labels or symbols.

Where the GDG identified areas in which there are uncertainties or where robust evidence was lacking, they developed research recommendations. Those that were identified as ‘high-priority’ were included in the NICE version of the guideline.

3.6. HEALTH ECONOMICS METHODS

The aim of the health economics was to contribute to the guideline's development by providing evidence on the cost effectiveness of interventions for the longer-term management of self-harm covered in the guideline. This was achieved by:

  • systematic literature review of existing economic evidence
  • decision-analytic economic modelling.

Systematic reviews of economic literature were conducted in all areas covered in the guideline. Economic modelling was undertaken in areas with likely major resource implications, where the current extent of uncertainty over cost effectiveness was significant and economic analysis was expected to reduce this uncertainty, in accordance with The Guidelines Manual (NICE, 2009d). Prioritisation of areas for economic modelling was a joint decision between the Health Economist and the GDG. The rationale for prioritising review questions for economic modelling was set out in an economic plan agreed between NICE, the GDG, the health economist and the other members of the technical team. The economic question selected as a key issue addressed by economic modelling was:

  • Cost-effectiveness of psychological intervention and treatment as usual for prevention of self-harm repetition among people who self-harm.

In addition, literature on the health-related quality of life (HRQoL) of people who self-harm was systematically searched to identify studies reporting appropriate utility scores that could be utilised in a cost-utility analysis.

The rest of this section describes the methods adopted in the systematic literature review of economic studies. Methods employed in economic modelling are described in the respective sections of the guideline.

3.6.1. Search strategy for economic evidence

Scoping searches

A broad preliminary search of the literature was undertaken in July 2009 to obtain an overview of the issues likely to be covered by the scope, and help define key areas. Searches were restricted to economic studies and HTA reports, and conducted in the following databases:

  • EMBASE
  • MEDLINE/MEDLINE In-Process
  • HTA database (technology assessments)
  • NHS Economic Evaluation Database

Any relevant economic evidence arising from the clinical scoping searches was also made available to the health economist during the same period.

Systematic literature searches

After the scope was finalised, a systematic search strategy was developed to locate all the relevant evidence. Searches were restricted to economic evidence (including full and partial economic evaluations) and HTA reports, and conducted in the following databases:

  • CINAHL
  • EconLit
  • EMBASE
  • MEDLINE/MEDLINE In-Process
  • PsycINFO
  • HTA database (technology assessments)
  • NHS Economic Evaluation Database

Any relevant economic evidence arising from the clinical searches was also made available to the health economist during the same period.

The search strategies were initially developed for MEDLINE before being translated for use in other databases/interfaces. Strategies were built up through a number of trial searches, and discussions of the results of the searches with the review team and GDG, to ensure that all possible relevant search terms were covered. To assure comprehensive coverage, search terms for self-harm were kept purposely broad to help counter dissimilarities in database indexing practices, and imprecise reporting of study populations by authors in the titles and abstracts of records.

Reference Manager

Citations from each search were downloaded into Reference Manager (a software product for managing references and formatting bibliographies) and duplicates removed. Records were then screened against the inclusion criteria of the reviews before being quality appraised. The unfiltered search results were saved and retained for future potential re-analysis to help keep the process both replicable and transparent.

Search filters

The search filter for health economics is an adaptation of a pre-tested strategy filter designed by the CRD (CRD, 2007). The search filter is designed to retrieve records of economic evidence (including full and partial economic evaluations) from the vast amount of literature indexed to major medical databases such as MEDLINE. The filter, which comprises a combination of controlled vocabulary and free-text retrieval methods, maximises sensitivity (or recall) to ensure that as many potentially relevant records as possible are retrieved from a search. Full details of the filter are provided in Appendix 12.

Date and language restrictions

Systematic database searches were initially conducted in March 2010 up to the most recent searchable date. Search updates were generated on a 6-monthly basis, with the final re-runs carried out in January 2011 ahead of the guideline consultation. After this point, studies were included only if they were judged by the GDG to be exceptional (for example, the evidence was likely to change a recommendation).

Although no language restrictions were applied at the searching stage, foreign language studies were not requested or reviewed unless they were of particular importance to an area under review. All of the searches were restricted to research published from 1995 onwards in order to obtain data relevant to current healthcare settings and costs.

Other search methods

Other search methods involved scanning the reference lists of all eligible publications (systematic reviews, stakeholder evidence, and included studies from the economic and clinical reviews) to identify further studies for consideration.

Full details of the search strategies and filter used for the systematic review of health economic evidence are provided in Appendix 12.

3.6.2. Inclusion criteria for economic studies

The following inclusion criteria were applied to select studies identified by the economic searches for further consideration:

  • Only studies from Organisation for Economic Co-operation and Development countries were included, because the aim of the review was to identify economic information transferable to the UK context.
  • Selection criteria based on types of clinical conditions and patients as well as interventions assessed were identical to the clinical literature review.
  • Studies were included provided that sufficient details regarding methods and results were available to enable the methodological quality of the study to be assessed, and provided that the study's data and results were extractable. Poster presentations of abstracts were excluded.
  • Full economic evaluations that compared two or more relevant options and considered both costs and consequences were included in the review.
  • Economic studies were included if they used clinical effectiveness data from an RCT, a prospective cohort study or a systematic review and meta-analysis of clinical studies. Studies that had a mirror-image or other retrospective design were excluded from the review.
  • Studies were included only if the examined interventions were clearly described. This involved the dosage and route of administration, and the duration of treatment in the case of pharmacological treatments, and the types of health professionals involved, as well as the frequency and duration of treatment in the case of psychological interventions. Evaluations in which medications were treated as a class were excluded from further consideration.
  • Studies that adopted a very narrow perspective, ignoring major categories of costs to the NHS, were excluded; for example studies that estimated exclusively drug acquisition costs or hospitalisation costs were considered non-informative to the guideline development process.

3.6.3. Applicability and quality criteria for economic studies

All economic studies eligible for inclusion were appraised for their applicability and quality using the methodology checklist for economic evaluations recommended by NICE (NICE, 2009d), which is shown in Appendix 13 of this guideline. The methodology checklist for economic evaluations was also applied to the economic models developed specifically for this guideline. All studies that fully or partially met the applicability and quality criteria described in the methodology checklist were considered during the guideline development process, along with the results of the economic modelling conducted specifically for this guideline.

3.6.4. Presentation of economic evidence

The economic evidence considered in the guideline is provided in the respective evidence chapters, following presentation of the relevant clinical evidence. The references to included studies and the respective evidence tables with the study characteristics and results are provided in Appendix 14. Methods and results of economic modelling undertaken alongside the guideline development process are presented in the relevant evidence chapters. Characteristics and results of all economic studies considered during the guideline development process (including modelling studies conducted for this guideline) are summarised in economic evidence profiles accompanying respective GRADE clinical evidence profiles in Appendix 17.

3.6.5. Results of the systematic search of economic literature

The titles of all studies identified by the systematic search of the literature were screened for their relevance to the topic (that is, economic issues and information on HRQoL in people who self-harm). References that were clearly not relevant were excluded first. The abstracts of all potentially relevant studies (12 references) were then assessed against the inclusion criteria for economic evaluations by the health economist. Full texts of the studies potentially meeting the inclusion criteria (including those for which eligibility was not clear from the abstract) were obtained. Studies that did not meet the inclusion criteria, were duplicates, were secondary publications of one study, or had been updated in more recent publications were subsequently excluded. Finally, two economic studies that fully or partially met the applicability and quality criteria were considered at formulation of the guideline recommendations.

3.7. STAKEHOLDER CONTRIBUTIONS

Professionals, service users, and companies have contributed to and commented on the guideline at key stages in its development. Stakeholders for this guideline include:

  • service users and carer stakeholders: national patient and carer organisations that represent the interests of people whose care will be covered by the guideline
  • local patient and carer organisations: but only if there is no relevant national organisation
  • professional stakeholders' national organisations: that represent the healthcare professionals who provide the services described in the guideline
  • commercial stakeholders: companies that manufacture drugs or devices used in treatment of the condition covered by the guideline and whose interests may be significantly affected by the guideline
  • providers and commissioners of health services in England and Wales
  • statutory organisations: including the Department of Health, the Welsh Assembly Government, NHS Quality Improvement Scotland, the Healthcare Commission and the National Patient Safety Agency
  • research organisations that have carried out nationally recognised research in the area.

NICE clinical guidelines are produced for the NHS in England and Wales, so a ‘national’ organisation is defined as one that represents England and/or Wales or has a commercial interest in England and/or Wales.

Stakeholders have been involved in the guideline's development at the following points:

  • commenting on the initial scope of the guideline and attending a scoping workshop held by NICE
  • contributing possible review questions and lists of evidence to the GDG
  • commenting on the draft of the guideline
  • highlighting factual errors in the pre-publication check.

3.8. VALIDATION OF THE GUIDELINE

Registered stakeholders had an opportunity to comment on the draft guideline, which was posted on the NICE website during the consultation period. Following the consultation, all comments from stakeholders and others were responded to and the guideline updated as appropriate. The Guideline Review Panel also reviewed the guideline and checked that stakeholders' comments had been addressed.

Following the consultation period, the GDG finalised the recommendations and the NCCMH produced the final documents. These were then submitted to NICE for the pre-publication check where stakeholders were given the opportunity to highlight factual errors. Any errors were corrected by the NCCMH, then the guideline was formally approved by NICE and issued as guidance to the NHS in England and Wales.

Footnotes

2

Based on the approach suggested by Furukawa and colleagues (2006).

3

For further information about GRADE, see www​.gradeworkinggroup.org.

4

See NICE's Social Value Judgements: Principles for the Development of NICE Guidance (NICE, 2008): www​.nice.org.uk/aboutnice​/howwework/socialvaluejudgements​/socialvaluejudgements​.jsp.

5
Copyright © 2012, The British Psychological Society & The Royal College of Psychiatrists.

All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Enquiries in this regard should be directed to the British Psychological Society.

Bookshelf ID: NBK126788

Views

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...