Home > Clinical Guides > Attention Deficit Hyperactivity... > METHODS USED TO DEVELOP THIS GUIDELINE

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

National Collaborating Centre for Mental Health (UK). Attention Deficit Hyperactivity Disorder: Diagnosis and Management of ADHD in Children, Young People and Adults. Leicester (UK): British Psychological Society (UK); 2009. (NICE Clinical Guidelines, No. 72.)



The development of this guideline drew upon methods outlined by NICE (The Guidelines Manual1 [NICE, 2006c]). A team of healthcare professionals, lay representatives and technical experts known as the Guideline Development Group (GDG), with support from the NCCMH staff, undertook the development of a patient-centred, evidence-based guideline. There are six basic steps in the process of developing a guideline:

  • define the scope, which sets the parameters of the guideline and provides a focus and steer for the development work
  • define clinical questions considered important for practitioners and service users
  • develop criteria for evidence searching and search for evidence
  • design validated protocols for systematic review and apply to evidence recovered by search
  • synthesise and (meta-) analyse data retrieved, guided by the clinical questions, and produce evidence summaries and profiles
  • answer clinical questions with evidence-based recommendations for clinical practice.

The clinical practice recommendations made by the GDG are therefore derived from the most up-to-date and robust evidence base for the clinical and cost effectiveness of the treatments and services used in the treatment and management of ADHD. In addition, to ensure a service user and carer focus, the concerns of service users and carers regarding health and social care have been highlighted and addressed by recommendations agreed by the whole GDG.


Guideline topics are selected by the Department of Health and the Welsh Assembly Government, which identify the main areas to be covered by the guideline in a specific remit (for further information see The Guidelines Manual 2 [NICE, 2006c]). The remit for this guideline was translated into a scope document by staff at the NCCMH (see Appendix 1).

The purpose of the scope was to:

  • provide an overview of what the guideline will include and exclude
  • identify the key aspects of care that must be included
  • set the boundaries of the development work and provide a clear framework to enable work to stay within the priorities agreed by NICE and the NCCMH and the remit from the Department of Health/Welsh Assembly Government
  • inform the development of the clinical questions and search strategy
  • inform professionals and the public about the expected content of the guideline
  • keep the guideline to a reasonable size to ensure that its development can be carried out within the allocated period.

The draft scope was subject to consultation with registered stakeholders over a 4-week period. During the consultation period, the scope was posted on the NICE website (www.nice.org.uk). Comments were invited from stakeholder organisations and the Guideline Review Panel (GRP). Further information about the GRP can also be found on the NICE website. The NCCMH and NICE reviewed the scope in light of comments received, and the revised scope was signed off by the GRP.


The GDG consisted of: professionals in clinical child and adolescent psychiatry, clinical child and adolescent psychology (and neuropsychology), psychiatry for learning disorders, developmental paediatrics, paediatrics (neurodisability), general practice and nursing; academic experts in child and adolescent psychiatry, paediatric medicine research, forensic clinical psychology, and education; service users and carers. In order to ascertain the experiences of children and young people of stimulant medication for ADHD, the NCCMH commissioned a focus group study. The guideline development process was supported by staff from the NCCMH, who undertook the clinical and health economics literature searches, reviewed and presented the evidence to the GDG, managed the process and contributed to drafting the guideline.

3.3.1. Guideline Development Group meetings

Twenty GDG meetings were held between March 2006 and May 2008. During each day-long GDG meeting, in a plenary session, clinical questions and clinical evidence were reviewed and assessed and recommendations formulated and reviewed. At each meeting, all GDG members declared any potential conflicts of interest, and the concerns of the service users and carers were routinely discussed as part of a standing agenda.

3.3.2. Topic groups

The GDG divided its workload along clinically relevant lines to simplify the guideline development process, and GDG members formed smaller topic groups to undertake guideline work in that area of clinical practice. Topic group 1 covered questions relating to diagnosis and assessment; topic group 2 covered psychological interventions; topic group 3 covered pharmacological interventions; topic group 4 covered education interventions; and topic group 5 covered dietary interventions. These groups were designed to manage the large volume of evidence appraisal efficiently before presenting it to the GDG as a whole. Each topic group was chaired by a GDG member with expert knowledge of the topic area (one of the healthcare professionals). Topic groups refined the clinical definitions of treatment interventions, reviewed and prepared the evidence with the systematic reviewer before presenting it to the GDG as a whole, and helped the GDG to identify further expertise in the topic. Topic group leaders reported the status of the group’s work as part of the standing agenda. They also introduced and led the GDG discussion of the evidence review for that topic and assisted the GDG Chair in drafting that section of the guideline relevant to the work of each topic group.

3.3.3. Service users and carers

Individuals with direct experience of services gave an integral service-user focus to the GDG and the guideline. The GDG included carers and a service user. They contributed as full GDG members to writing the clinical questions, helping to ensure that the evidence addressed their views and preferences, highlighting sensitive issues and terminology associated with ADHD, and bringing service-user research to the attention of the GDG. In drafting the guideline, they contributed to the editing of the first draft of the guideline’s introduction and to the writing of Chapter 4, and identified recommendations from the perspective of service users and carers.

3.3.4. Special advisers

Special advisers, who had specific expertise in one or more aspects of treatment and management relevant to the guideline, assisted the GDG, commenting on specific aspects of the developing guideline and making presentations to the GDG. Appendix 3 lists those who agreed to act as special advisers.

3.3.5. National and international experts

National and international experts in the area under review were identified through the literature search and through the experience of the GDG members. These experts were contacted to recommend unpublished or soon-to-be published studies in order to ensure up-to-date evidence was included in the development of the guideline. They informed the group about completed trials at the pre-publication stage, systematic reviews in the process of being published, studies relating to the cost effectiveness of treatment and trial data if the GDG could be provided with full access to the complete trial report. Appendix 5 lists researchers who were contacted.


Clinical questions were used to guide the identification and interrogation of the evidence base relevant to the topic of the guideline. The questions were developed using a modified nominal group technique. The process began by asking each topic group of the GDG to submit as many questions as possible. The questions were then collated and refined by the review team. The GDG members were then asked to rate each question for importance. At a subsequent meeting, the GDG Chair facilitated a discussion to further refine the questions. The results of this process were then discussed and consensus reached about which questions would be of primary importance and which would be secondary. The GDG aimed to address all primary questions, while secondary questions would only be covered time permitting. The PICO (patient, intervention, comparison and outcome) framework was used to help formulate questions about interventions. This structured approach divides each question into four components: the patients (the population under study); the interventions (what is being done; or test/risk factor); the comparisons (other main treatment options); and the outcomes (the measures of how effective the interventions have been; or what is being predicted/prevented). Appendix 6 lists the clinical questions.

To help facilitate the literature review, a note was made of the best study design type to answer each question. There are four main types of clinical question of relevance to NICE guidelines. These are listed in Text box 2. For each type of question the best primary study design varies, where ‘best’ is interpreted as ‘least likely to give misleading answers to the question’.

Text box 2. Best study design to answer each type of question.

Text box 2

Best study design to answer each type of question.

In all cases, however, a well-conducted systematic review of the appropriate type of study is likely to yield a better answer than a single study.

Deciding on the best design type to answer a specific clinical or public health question does not mean that studies of different design types addressing the same question were discarded.


The aim of the clinical literature review was to identify and synthesise relevant evidence from the literature systematically in order to answer the specific clinical questions developed by the GDG. Thus, clinical practice recommendations are evidence-based, where possible, and if evidence was not available, informal consensus methods were used (see Section 3.5.7) and the need for future research was specified.

3.5.1. Methodology

A stepwise, hierarchical approach was taken to locating and presenting evidence to the GDG. The NCCMH developed this process based on methods set out in The Guidelines Manual3 (NICE, 2006c) and after considering recommendations from a range of other sources. These included:

  • Clinical Policy and Practice Program of the New South Wales Department of Health (Australia)
  • Clinical Evidence Online
  • The Cochrane Collaboration
  • Grading of Recommendations: Assessment, Development, and Evaluation (GRADE) Working Group
  • New Zealand Guidelines Group
  • NHS Centre for Reviews and Dissemination
  • Oxford Centre for Evidence-Based Medicine
  • Oxford Systematic Review Development Programme
  • Scottish Intercollegiate Guidelines Network (SIGN)
  • United States Agency for Healthcare Research and Quality.

3.5.2. The review process

After the scope was finalised, a more extensive search for systematic reviews and published guidelines was undertaken. Existing NICE guidelines were updated where necessary.

At this point, the review team, in conjunction with the GDG, developed an evidence map that detailed all comparisons necessary to answer the clinical questions. The initial approach taken to locating primary-level studies depended on the type of clinical question and availability of evidence.

The GDG decided which questions were best addressed by good practice based on expert opinion, which questions were likely to have a good evidence base and which questions were likely to have little or no directly relevant evidence. Recommendations based on good practice were developed by informal consensus of the GDG. For questions with a good evidence base, the review process depended on the type of clinical question (see below). For questions that were unlikely to have a good evidence base, a brief descriptive review was initially undertaken by a member of the GDG (see Section 3.5.7).

Searches for evidence were updated between 6 and 8 weeks before the stakeholder consultation. After this point, studies were included only if they were judged by the GDG to be exceptional (for example, the evidence was likely to change a recommendation).

The search process for questions concerning interventions

For questions related to interventions, the initial evidence base was formed from well-conducted RCTs that addressed at least one of the clinical questions (the review process is illustrated in Flowchart 1). Although there are a number of difficulties with the use of RCTs in the evaluation of interventions in mental health, the RCT remains the most important method for establishing treatment efficacy. For other clinical questions, searches were for the appropriate study design (see above).

Flowchart Icon

Flowchart 1. Guideline review process (PDF, 77K)

All searches were based on the standard mental health related bibliographic databases (EMBASE, MEDLINE, PsycINFO, Cochrane Library, ERIC) for all trials potentially relevant to the guideline. If the number of citations generated from this search was large (more than 5000), existing systematic reviews and question-specific search filters were developed to restrict the search while minimising loss of sensitivity.

Where the evidence base was large, recent high-quality English-language systematic reviews were used primarily as a source of RCTs (see Appendix 10 for quality criteria used to assess systematic reviews). In some circumstances, however, existing data sets were utilised. Where this was the case, data were cross-checked for accuracy before use. New RCTs meeting inclusion criteria set by the GDG were incorporated into the existing reviews and fresh analyses performed.

After the initial search results had been scanned liberally to exclude irrelevant papers, the review team used a purpose built ‘study information’ database to manage both the included and the excluded studies (eligibility criteria were developed after consultation with the GDG). For questions without good-quality evidence (after the initial search), a decision was made by the GDG about whether to (a) repeat the search using subject-specific databases (for example, CINAHL, AMED, SIGLE or PILOTS), (b) conduct a new search for lower levels of evidence, or (c) adopt a consensus process (see Section 3.5.7). Future guidelines will be able to update and extend the usable evidence base starting from the evidence collected, synthesised and analysed for this guideline.

In addition, searches were made of the reference lists of all eligible systematic reviews and included studies, as well as the list of evidence submitted by stakeholders. Known experts in the field (see Appendix 5), based both on the references identified in early steps and on advice from GDG members, were sent letters requesting relevant studies that were in the process of being published.4 In addition, the tables of contents of appropriate journals were periodically checked for relevant studies.

The search process for questions of diagnosis and prognosis

For questions related to diagnosis and prognosis, the search process was the same as described above, except that the initial evidence base was formed from studies with the most appropriate and reliable design to answer the particular question. That is, for questions about diagnosis, the initial search was for systematic reviews and meta-analyses as well as cross-sectional, factor analytic, genetic and diagnostic studies; for questions about prognosis, it was for cohort studies of representative patients. In situations where it was not possible to identify a substantial body of appropriately designed studies that directly addressed each clinical question, a consensus process was adopted (see Section 3.5.7).

Search filters

Search filters developed by the review team consisted of a combination of subject heading and free-text phrases. Specific filters were developed for the guideline topic, and where necessary, for each clinical question. In addition, the review team used filters developed for systematic reviews, RCTs and other appropriate research designs (see Appendix 8).

Study selection

All primary-level studies included after the first scan of citations were acquired in full and re-evaluated for eligibility at the time they were being entered into the study information database (see Appendix 9 for screen shots of the database). Specific eligibility criteria were developed for each clinical question and are described in the relevant clinical evidence chapters. Eligible systematic reviews and primary-level studies were critically appraised for methodological quality (see Appendix 10 for the quality checklists). The eligibility of each study was confirmed by at least one member of the appropriate topic group.

For some clinical questions, it was necessary to prioritise the evidence with respect to the UK context (that is, external validity). To make this process explicit, the topic groups took into account the following factors when assessing the evidence:

  • participant factors (for example, gender, age, ethnicity)
  • provider factors (for example, model fidelity, the conditions under which the intervention was performed and the availability of experienced staff to undertake the procedure)
  • cultural factors (for example, differences in standard care and differences in the welfare system).

It was the responsibility of each topic group to decide which prioritisation factors were relevant to each clinical question in light of the UK context and then decide how they should modify their recommendations.

Unpublished evidence

The GDG used a number of criteria when deciding whether or not to accept unpublished data. First, the evidence must have been accompanied by a trial report containing sufficient detail to assess the quality of the data properly. Second, the evidence must be submitted with the understanding that data from the study and a summary of the study’s characteristics would be published in the full guideline. Therefore, the GDG did not accept evidence submitted as commercial in confidence. Having said that, the GDG recognised that unpublished evidence submitted by investigators might later be retracted by those investigators if the inclusion of such data would jeopardise publication of their research.

3.5.3. Data extraction

Outcome data were extracted from all eligible studies, which met the quality criteria, into RevMan 4.2.10 (Review Manager, The Cochrane Centre, 2003) or Word tables. Studies with factor analysis were quality assessed using a checklist elaborated and agreed by the GDG members (see Chapter 5).

For each outcome, a hierarchy of most suitable outcome measures was agreed upon by the GDG members. If a study reported more than one relevant outcome measure for a given outcome, only the measure with the highest hierarchy was included in the meta-analysis.

For a given outcome (continuous and dichotomous), where more than 50% of the number randomised to any group were not accounted for by trial authors, the data were excluded from the review because of the risk of bias.5 Where possible, however, dichotomous efficacy outcomes were calculated on an intention-to-treat basis (that is, a ‘once-randomised-always-analyse’ basis). This assumes that those participants who ceased to engage in the study – from whatever group – had an unfavourable outcome. This meant that the 50% rule was not applied to dichotomous outcomes where there was good evidence that those participants who ceased to engage in the study were likely to have an unfavourable outcome (in this case, early withdrawals were included in both the numerator and denominator). Adverse effects were entered into Review Manager as reported by the study authors because it was usually not possible to determine whether early withdrawals had an unfavourable outcome. For the outcome ‘leaving the study early for any reason’, the denominator was the number randomised.

Where some of the studies failed to report standard deviations (for a continuous outcome), and where an estimate of the variance could not be computed from other reported data or obtained from the study author, the following approach was taken:6

  1. When the number of studies with missing standard deviations was small and when the total number of studies was large, the pooled standard deviation from all the other available studies in the same meta-analysis was used. In this case, the appropriateness of the imputation was made by comparing the standardised mean differences (SMDs) of those trials that had reported standard deviations against the hypothetical SMDs of the same trials based on the imputed standard deviations. If they converged, the meta-analytical results were considered to be reliable.
  2. When the number of studies with missing standard deviations was large or when the total number of studies was small, standard deviations were taken from a previous systematic review (where available), because the small sample size may allow unexpected deviation due to chance. In this case, the results were considered to be less reliable.

The meta-analysis of survival data, such as time to any mood episode, was based on log hazard ratios and standard errors. Since individual patient data were not available in included studies, hazard ratios and standard errors calculated from a Cox proportional hazard model were extracted. Where necessary, standard errors were calculated from confidence intervals (CIs) or p-value according to standard formulae (for example, Cochrane Reviewers’ Handbook 4.2.2.). Data were summarised using the generic inverse variance method using Review Manager.

Consultation was used to overcome difficulties with coding. Data from studies included in existing systematic reviews were extracted independently by one reviewer and cross-checked with the existing data set. Where possible, two independent reviewers extracted data from new studies. Where double data extraction was not possible, data extracted by one reviewer was checked by the second reviewer. Disagreements were resolved with discussion. Where consensus could not be reached, a third reviewer resolved the disagreement. Masked assessment (that is, blind to the journal from which the article comes, the authors, the institution and the magnitude of the effect) was not used since it is unclear that doing so reduces bias (Jadad et al., 1996; Berlin, 2001).

3.5.4. Synthesising the evidence

Where possible, meta-analysis was used to synthesise the evidence using Review Manager. If necessary, reanalyses of the data or sub-analyses were used to answer clinical questions not addressed in the original studies or reviews.

Dichotomous outcomes were analysed as relative risks (RR) with the associated 95% CI (for an example, see Figure 1). A relative risk (also called a risk ratio) is the ratio of the treatment event rate to the control event rate. An RR of 1 indicates no difference between treatment and control. In Figure 1, the overall RR of 0.73 indicates that the event rate (that is, non-remission rate) associated with intervention A is about three quarters of that with the control intervention or, in other words, the RR reduction is 27%.

Figure 1. Example of a forest plot displaying dichotomous data.

Figure 1

Example of a forest plot displaying dichotomous data.

The CI shows with 95% certainty the range within which the true treatment effect should lie and can be used to determine statistical significance. If the CI does not cross the ‘line of no effect’, the effect is statistically significant.

Continuous outcomes were analysed as weighted mean differences (WMD), or as an SMD when different measures were used in different studies to estimate the same underlying effect (for an example, see Figure 2). If provided, intention-to-treat data, using a method such as ‘last observation carried forward’, were preferred over data from completers.

Figure 2. Example of a forest plot displaying continuous data.

Figure 2

Example of a forest plot displaying continuous data.

To check for consistency between studies, both the I2 test of heterogeneity and a visual inspection of the forest plots were used. The I2 statistic describes the proportion of total variation in study estimates that is due to heterogeneity (Higgins & Thompson, 2002). The I2 statistic was interpreted in the follow way:

  • Greater than 50%: notable heterogeneity (an attempt was made to explain the variation, for example outliers were removed from the analysis or sub-analyses were conducted to examine the possibility of moderators. If studies with heterogeneous results were found to be comparable, a random-effects model was used to summarise the results [DerSimonian & Laird, 1986]. In the random-effects analysis, heterogeneity is accounted for both in the width of CIs and in the estimate of the treatment effect. With decreasing heterogeneity the random-effects approach moves asymptotically towards a fixed-effects model).
  • 30 to 50%: moderate heterogeneity (both the chi-squared test of heterogeneity and a visual inspection of the forest plot were used to decide between a fixed and random-effects model)
  • Less than 30%: mild heterogeneity (a fixed-effects model was used to synthesise the results).

To explore the possibility that the results entered into each meta-analysis suffered from publication bias, data from included studies were entered, where there was sufficient data, into a funnel plot. Asymmetry of the plot was taken to indicate possible publication bias and investigated further.

An estimate of the proportion of eligible data that were missing (because some studies did not include all relevant outcomes) was calculated for each analysis.

The Number Needed to Treat–Benefit (NNTB) or the Number Needed to Treat–Harm (NNTH) was reported for each outcome where the baseline risk (that is, control group event rate) was similar across studies. In addition, NNTs calculated at follow-up were only reported where the length of follow-up was similar across studies. When the length of follow-up or baseline risk varies (especially with low risk), the NNT is a poor summary of the treatment effect (Deeks, 2002).

Study characteristics tables, generated automatically from the study database, were used to summarise general information about each study (see Appendix 17). Where meta-analysis was not appropriate and/or possible, the reported results from each primary-level study were also presented in the included studies table (and included, where appropriate, in a narrative review).

3.5.5. Presenting the data to the GDG

Study characteristics tables and, where appropriate, forest plots generated with Review Manager were presented to the GDG in order to prepare a GRADE evidence profile table for each review and to develop recommendations.

GRADE evidence profile tables

A GRADE evidence profile was used to summarise both the quality of the evidence and the results of the evidence synthesis (see Table 4 for an example of an evidence profile). For each outcome, quality may be reduced depending on the study design, limitations (based on the quality of individual studies; see Appendix 10 for the quality checklists), inconsistency (see Section 3.5.4 for how consistency was measured), indirectness (that is, how closely the outcome measures, interventions and participants match those of interest), and imprecision (based on the CI around the effect size). For observational studies, the quality may be increased if there is a large effect, plausible confounding would have changed the effect, or there is evidence of a dose-response gradient (details would be provided under the other considerations column). Each evidence profile also included a summary of the findings: number of patients included in each group, an estimate of the magnitude of the effect, and the overall quality of the evidence for each outcome. The quality of the evidence was based on the quality assessment components (study design, limitations to study quality, consistency, directness and any other considerations) and graded using the following definitions:

Table 4. Example of GRADE evidence profile.

Table 4

Example of GRADE evidence profile.

  • High =Further research is very unlikely to change our confidence in the estimate of the effect.
  • Moderate =Further research is likely to have an important impact on our confidence in the estimate of the effect and may change the estimate.
  • Low =Further research is very likely to have an important impact on our confidence in the estimate of the effect and is likely to change the estimate.
  • Very low =Any estimate of effect is very uncertain.

For further information about the process and the rationale of producing an evidence profile table, see GRADE (2004).

Forest plots

Each forest plot displayed the effect size and CI for each study as well as the overall summary statistic. The graphs were organised so that the display of data in the area to the left of the ‘line of no effect’ indicated a ‘favourable’ outcome for the treatment in question.

3.5.6. Forming the clinical summaries and recommendations

Once the GRADE profile tables relating to a particular clinical question were completed, summary tables incorporating important information from the GRADE profiles were developed (these tables are presented in the evidence chapters where used). Finally, the systematic reviewer in conjunction with the topic group lead produced a clinical evidence summary.

Once the GRADE profiles and clinical summaries were finalised and agreed by the GDG, the associated recommendations were drafted, taking into account the trade-off between the benefits and downsides of treatment as well as other important factors. These included economic considerations, values of the GDG and society, and the group’s awareness of practical issues (Eccles et al., 1998).

3.5.7. Method used to answer a clinical question in the absence of appropriately designed, high-quality research

In the absence of level-I evidence (or a level that is appropriate to the question), or where the GDG were of the opinion (on the basis of previous searches or their knowledge of the literature) that there was unlikely to be such evidence in this guideline, an informal consensus process was adopted. This process focused on those questions that the GDG considered a priority.

Informal consensus

The starting point for the process of informal consensus was that a member of the topic group identified, with help from the systematic reviewer, a narrative review that most directly addressed the clinical question. Where this was not possible, a brief review of the recent literature was initiated.

This existing narrative review or new review was used as a basis for beginning an iterative process to identify lower levels of evidence relevant to the clinical question and to lead to written statements for the guideline. The process involved a number of steps:

  1. A description of what is known about the issues concerning the clinical question was written by one of the topic group members.
  2. Evidence from the existing review or new review was then presented in narrative form to the GDG and further comments were sought about the evidence and its perceived relevance to the clinical question.
  3. Based on the feedback from the GDG, additional information was sought and added to the information collected. This may include studies that did not directly address the clinical question but were thought to contain relevant data.
  4. If, during the course of preparing the report, a significant body of primary-level studies (of appropriate design to answer the question) were identified, a full systematic review was conducted.
  5. At this time, subject possibly to further reviews of the evidence, a series of statements that directly addressed the clinical question were developed.
  6. Following this, on occasions and as deemed appropriate by the GDG, the report was then sent to appointed experts outside the GDG for peer review and comment. The information from this process was then fed back to the GDG for further discussion of the statements.
  7. Recommendations were then developed and could also be sent for further external peer review.
  8. After this final stage of comment, the statements and recommendations were again reviewed and agreed upon by the GDG.


The aim of the health economics was to contribute to the guideline’s development by providing evidence on the cost effectiveness of interventions for children, young people and adults with ADHD covered in the guideline, in areas with likely major resource implications. This was achieved by:

  • systematic literature review of existing economic evidence
  • economic modelling, in areas where economic evidence was lacking or was considered inadequate to inform decisions.

3.6.1. Key economic issues

The following economic issues relating to diagnosis and management of children, young people and adults with ADHD were identified by the GDG in collaboration with the health economist as primary key issues that should be considered in the guideline:

  • the cost effectiveness of parent training for pre-school age children and CBT for older children and young people
  • the cost effectiveness of CBT for adults with ADHD
  • the relative cost effectiveness of different pharmacological interventions for children and adults with ADHD
  • the cost effectiveness of intensive medication management for children
  • the relative cost effectiveness of psychological, pharmacological and combination therapies for children.

In addition, literature on health related quality of life (HRQoL) of children and adults with ADHD was systematically searched to identify studies reporting appropriate utility weights that could be utilised in a cost-utility analysis.

The rest of this section describes the methods adopted in the systematic literature review of economic studies. Methods employed in economic modelling are described in the respective sections of the guideline.

3.6.2. Search strategy

For the systematic review of economic evidence on treatments for ADHD the standard mental-health-related bibliographic databases (EMBASE, MEDLINE, CINAHL and PsycINFO) were searched. For these databases, a health economics search filter adapted from the Centre for Reviews and Dissemination at the University of York was used in combination with a general filter for ADHD. Additional searches were performed in specific health economics databases (NHS EED, OHE HEED), as well as in the HTA database. For the HTA and NHS EED databases, the general filter for ADHD was used. OHE HEED was searched using a shorter, database-specific strategy. Initial searches were performed in June 2006. The searches were updated regularly, with the final search conducted 5 weeks before the consultation period.

In parallel to searches of electronic databases, reference lists of eligible studies and relevant reviews were searched by hand. Studies included in the clinical evidence review were also screened for economic evidence.

The systematic search for economic evidence resulted in 47 potentially relevant studies. Full texts of all potentially eligible studies (including those for which relevance/eligibility was not clear from the abstract) were obtained. These publications were then assessed against a set of standard inclusion criteria by the health economists, and papers eligible for inclusion were subsequently assessed for internal validity. The quality assessment was based on the checklists used by the British Medical Journal to assist referees in appraising full and partial economic analyses (Drummond & Jefferson, 1996) (see Appendix 12).

3.6.3. Selection criteria

The following inclusion criteria were applied to select studies identified by the economic searches for further analysis:

  • No restriction was placed on language or publication status of the papers.
  • Studies published from 1990 onwards were included. This date restriction was imposed in order to obtain data relevant to current healthcare settings and costs.
  • Only studies from Organisation for Economic Co-operation and Development countries were included, as the aim of the review was to identify economic and HRQoL information transferable to the UK context.
  • Selection criteria based on types of clinical conditions and patients were identical to the clinical literature review.
  • Studies were included provided that sufficient details regarding methods and results were available to enable the methodological quality of the study to be assessed, and provided that the study’s data and results were extractable. Poster presentations or abstracts were in principle excluded; however, they were included if they reported additional data from studies which had already been published elsewhere and met the inclusion criteria, or if they contained appropriate input data required for economic modelling that were not otherwise available.
  • Full economic evaluations that compared two or more relevant options and considered both costs and consequences (that is, cost-effectiveness analysis, cost-utility analysis, cost-consequences analysis or cost-benefit analysis) were included in the review. HRQoL studies were included if they reported utility weights appropriate to use in a cost-utility analysis.

3.6.4. Data extraction

Data were extracted by the health economist using a standard economic data extraction form (see Appendix 13).

3.6.5. Presentation of economic evidence

The economic evidence identified by the health economics systematic review is summarised in the respective chapters of the guideline, following presentation of the clinical evidence. The characteristics and results of all economic studies included in the review are provided in the form of evidence tables in Appendix 14. Results of additional economic modelling undertaken alongside the guideline development process are also presented in the relevant chapters.


Besides making recommendations based on the clinical and cost effectiveness of interventions for ADHD, an important function of developing this guideline was understanding the experience of ADHD from the service user’s point of view.

In order to provide sufficient breadth of context and depth of understanding of children’s views on taking stimulant medicine, the NCCMH commissioned the London School of Economics to undertake a qualitative focus group study with children and young people on their perceptions of their use of stimulant medication, together with a review of the available literature on young people’s experiences. The full version of this report, including the extensive bibliography, can be found in Appendix 15, and a summary of the findings in Chapter 4.

Besides being reviewed by the GDG, the focus group proposal was also reviewed by a nationally sanctioned ethics committee and local research and development committees. The research team undertaking the focus group interviews and analyses were experienced both in qualitative methodologies and working with young people. Before data collection, they carefully researched the issues on the extra care required both in the design and execution of data collection methods in order to ensure that the information gathered was robust and usable, and that all ethical considerations relating to the vulnerable participant group were met.

3.7.1. Focus group participants

Participants in the study had all been diagnosed with ADHD and all were taking stimulant medication. They were recruited from clinics at three hospitals: Richmond Royal Hospital, London; the Maudsley Hospital, London; and Queen’s Medical Centre, Nottingham.

The sample consisted of 16 children (14 boys and two girls) ranging in age from 9 to 15 years old. All were attending state schools and all were white, with the exception of one child who was of mixed race. Fifty per cent of the children were living in two-parent homes, and 37% lived in single-mother homes. Two children lived with their fathers; and one child lived with his grandmother. Educational achievement and type of employment were used as indicators of socioeconomic status.7 A majority of parents had completed O levels or GCSEs; one parent had attended university. Seventy-two percent of parents’ job types ranged from semi-skilled to skilled work. A majority of mothers did not report having employment.

3.7.2. Data collection

Semi-structured focus groups were used to collect data about how children and young people experience stimulant medication. Allowing children to describe their experiences through qualitative interviews has been found to be both reliable and valid (Deatrick & Faux, 1991; Sorensen, 1992), and there is compelling evidence to suggest that children are competent research participants (Singh, 2007). Children’s competence as research participants is supported by the literature on children’s capacity and competence as patients. Children have been found to be capable of understanding the complexities of their condition; they have the capacity to give informed consent to invasive treatments, to contribute to deliberations over treatment strategies, and, in the case of diabetic children, to take responsibility for administering their own treatment (Alderson et al., 2006; Bluebond-Langner et al., 2005).

Thirteen children were interviewed as part of a series of focus groups. Three children were interviewed one-to-one, either because they were unable to attend the focus groups or because they preferred to be interviewed individually. The interviews took place in a room based at the hospital clinic and lasted approximately 1 hour. Written informed consent was obtained from one parent and also from the participant. Parents were also asked to complete a basic demographic questionnaire.

3.7.3. Methodology of focus groups

Focus groups are a widely used method in qualitative health research, and are often used when the research aim is to gather information in a little-understood or under-researched area. Focus groups elicit a range of experiences, opinions and feelings about a topic (Krueger & Casey, 2000), and the interaction in focus groups can result in enhanced disclosure, as participants challenge each other’s perceptions and opinions.

The collective nature of focus group discussion is often said to provide ‘more than the sum of its parts’ (Wilkinson, 1998). Interactive data result in enhanced disclosure, better understanding of participants’ own agendas, the production of more elaborated accounts, and the opportunity to observe the co-construction of meaning in action. Focus groups are, then, an ideal method for exploring people’s own meanings and understandings of health and illness.

Although focus groups with children are less commonly used in social science health research, market research with children (including market research around health and well-being) more commonly uses a focus group approach (for example, Caruana & Vassallo, 2003). Focus groups with children provide access to children’s own language and concepts and encourage elaboration of children’s own concerns and agendas.

3.7.4. Interviews

Interviews were conducted in a conversational style and included a standard set of open-ended questions (see Appendix 15 for the complete topic guide).

The first half of the interview involved posing broad questions that were followed by more specific probe questions. Principle areas of investigation included children’s understanding of ADHD diagnosis and behaviours, perceptions of how tablets helped them (or not), experiences of stigma, experiences of non-drug interventions for ADHD behaviours, impact of tablets on children’s perceptions of personal agency, and experiences of psychiatric services.

The second half of the interview involved a set of games and a vignette which provided children with the opportunity to elaborate their experiences and perceptions of medication in more creative and imaginative ways. The primary aims in this section of the interview were to contextualise children’s perceptions of tablets within their perceptions, understandings and/or experiences of other means of improving behaviour, and to elicit their ideas about resources that could help them have more positive experiences of an ADHD diagnosis and of medication.

The following methods were used in the second half of the interview (see Appendix 15 for further elaboration):

  1. Children were asked to compare how the experience of taking tablets was similar to, or different from, doing other things that were commonly considered good for them.
  2. Children were asked to respond to a vignette that elicited their ideas about what sorts to things can help a child’s behaviour.
  3. Children were asked to think up and discuss an invention that could help children with ADHD.
  4. Children were asked to rank in order a list of items that described common concerns voiced by school-age children. Each item was written on a separate card, and children were asked to put the cards in order of what they worried about most, to what they worried about least. The list included global warming, having ADHD, taking tablets, exams, homework and friendships. Global warming and exams were included on the list because these concerns were found to be significant sources of anxiety in a recent large cohort study of UK school-age children (Alexander & Hargreaves, 2007)

3.7.5. Data analysis

All interviews were digitally recorded, transcribed and analysed using rigorous qualitative coding practices that meet established criteria of validity and relevance to qualitative health research (Mays & Pope, 2000). Focus groups were coded using content analysis. The coding process captured the data on two analytic levels: individual concepts were coded first, and then these concepts were grouped together under higher order themes. Systematic coding meant that it was possible to code at both the individual level and at the group level. Group-level data were represented in the frequency with which concepts and themes were expressed by group members. Transcript excerpts elucidated the meaning of codes.

A coding frame was drawn up by the lead author of the study, Ilina Singh, and validated within a coding team. The coding team applied the same codes to a transcript in order to discuss their definition and validity. This discussion resulted in refinements to the structure of categories and sub-categories, as well as refinements to individual codes. The coding team was able to reach agreement on the validity of a majority of codes.


Professionals, service users, and companies have contributed to and commented on the guideline at key stages in its development. Stakeholders for this guideline include:

  • service user/carer stakeholders: the national service user and carer organisations that represent people whose care is described in this guideline
  • professional stakeholders: the national organisations that represent healthcare professionals who are providing services to service users
  • commercial stakeholders: the companies that manufacture medicines used in the treatment of ADHD
  • Primary Care Trusts
  • Department of Health and Welsh Assembly Government.

Stakeholders have been involved in the guideline’s development at the following points:

  • commenting on the initial scope of the guideline and attended a briefing meeting held by NICE
  • commenting on the draft of the guideline.


Registered stakeholders had an opportunity to comment on the draft guideline, which was posted on the NICE website during the consultation period. The GRP also reviewed the guideline and checked that stakeholders’ comments had been addressed.

Following the consultation period, the GDG finalised the recommendations and the NCCMH produced the final documents. These were then submitted to NICE. NICE then formally approved the guideline and issued its guidance to the NHS in England and Wales.



Available from: www​.nice.org.uk


Available from: www​.nice.org.uk


Available from: www​.nice.org.uk


Unpublished full trial reports were also accepted where sufficient information was available to judge eligibility and quality (see section on unpublished evidence).


‘Accounted for’ in this context means using an appropriate method for dealing with missing data (for example, last observation carried forward [LOCF] or a regression technique).


Based on the approach suggested by Furukawa and colleagues (2006).


Data were only available on mothers. Fathers’ educational achievement and job types would be more reliable indicators of socioeconomic status.

Copyright © 2009, The British Psychological Society & The Royal College of Psychiatrists.

All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Enquiries in this regard should be directed to the British Psychological Society.

Cover of Attention Deficit Hyperactivity Disorder
Attention Deficit Hyperactivity Disorder: Diagnosis and Management of ADHD in Children, Young People and Adults.
NICE Clinical Guidelines, No. 72.
National Collaborating Centre for Mental Health (UK).
Leicester (UK): British Psychological Society (UK); 2009.


NICE (National Institute for Health and Care Excellence)

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...