U.S. flag

An official website of the United States government

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Carrieri D, Pearson M, Mattick K, et al. Interventions to minimise doctors’ mental ill-health and its impacts on the workforce and patient care: the Care Under Pressure realist review. Southampton (UK): NIHR Journals Library; 2020 Apr. (Health Services and Delivery Research, No. 8.19.)

Cover of Interventions to minimise doctors’ mental ill-health and its impacts on the workforce and patient care: the Care Under Pressure realist review

Interventions to minimise doctors’ mental ill-health and its impacts on the workforce and patient care: the Care Under Pressure realist review.

Show details

Chapter 2Review methods

Research plan

We followed a similar methodology to that used in a previous National Institute for Health Research Health Services and Delivery Research programme project54 that was conducted by many of the same research team members.54,55 Any evidence synthesis that seeks to make sense of interventions aiming to improve doctors’ mental ill-health must take into account the contexts in which these interventions are situated. This generates an in-depth understanding of which components within these interventions matter more (or less) than others, for whom they matter and in what ways. A realist review can synthesise relevant data found within qualitative, quantitative and mixed-methods research. By following an interpretive, theory-driven approach to analysing data from such diverse literature sources, realist reviews move beyond description, to provide findings that coherently and transferably explain how and why contexts can influence outcomes. This is particularly relevant to complex programmes characterised by significant levels of heterogeneity. The plan of investigation followed a detailed protocol based on Pawson et al.’s56 five iterative stages for realist reviews: (1) locating existing theories, (2) searching for evidence, (3) selecting articles, (4) extracting and organising data and (5) synthesising the evidence and drawing conclusions. The reporting is consistent with the Realist And Meta-narrative Evidence Syntheses: Evolving Standards (RAMESES) quality and publication standards for realist reviews.57 The project ran for 18 months from November 2017 to April 2019. Regular project team meetings and stakeholder meetings throughout the lifecycle of the project ensured that multiple perspectives and interpretations were brought to bear on the research.

The protocol has been published in BMJ Open58 and the review has been registered with PROSPERO (CRD42017069870). The review design and methodology is explained in more detail in the sections below and illustrated in Figure 1.

FIGURE 1. Flow diagram of the project.


Flow diagram of the project. Note that dashed arrows indicate iteration where necessary. Reproduced from Wong et al. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which (more...)

Stakeholder group

A stakeholder group was recruited for the CUP review to provide content expertise, feedback on and refinement of our programme theory and to coproduce non-academic outputs. The stakeholder group did not provide us with data, but feedback and advice based on their expertise and experience. A total of 22 people were involved in the stakeholder group during the review, including patient representatives, clinicians, doctors in training, medical educators and academics. Consultations with stakeholder group members took place as part of 2-hour meetings at regular intervals throughout the project and e-mail exchange. Additional ‘satellite’ face-to-face meetings were also arranged with some stakeholders who could not attend the main meetings, as well as Skype™ (Microsoft Corporation, Redmond, WA, USA) and telephone conversations. Table 1 provides a list of the face-to-face meetings, including the number of participants and key topics discussed.



Stakeholder group meetings summary

Stakeholder group meetings took place at the University of Exeter or in London. To leave more time for discussion, we circulated any relevant preparatory material [e.g. summaries of previous meeting(s)], ahead of meetings when necessary. The meetings usually started with a brief slide presentation by our project team to introduce stakeholders to the topic under discussion and realist methods, and to provide a quick update on progress with the review. We presented high-level programme theory to the group in the form of statements and visual prompts to obtain their feedback. Discussions were designed to be more open ended in the early stages of the review, but focused on particular aspects of the programme theory as the project progressed. Later stakeholder groups focused on actionable findings and dissemination of the study. We used a framework called the evidence integration triangle (EIT)59 to ensure that discussions focused on the real-world impact of our study. The three components of the EIT (practical evidence-based interventions; pragmatic, longitudinal measures of progress; and participatory implementation processes) were used to structure discussions at the stakeholder group meetings and the workshop with policy-makers at the end of the project. Of the three components, the pragmatic longitudinal measures of progress proved by far the most challenging (see Chapter 5, Strengths and limitations of this review). Facilitation of the meetings ensured that everyone was able to contribute and voice their opinion, whether in agreement or disagreement. Notes from these meetings were used to set direction for the review and to refine programme theory, rather than as primary data for analysis, and the report does not include any verbatim data excerpts from these meetings.

Discussions with stakeholders helped ground the review in the practical reality experienced by participants and the challenges they faced in their respective roles. The sharing of these experiences re-enforced our decisions at times (e.g. to critically question the focus on ‘resilience’ in doctors) and prompted us to pay close attention to aspects that we had missed at others (e.g. the taboo around doctors expressing vulnerability). Stakeholders’ questions and contributions also prompted us to look more broadly in the literature for substantive explanatory theory, for example leading us to identify social cure theory and an emerging body of work on vulnerability that we drew on to shape our thinking and interpretation (for more information about the substantive theories used see Chapter 3, Summary of the 19 CMOcs in four main groupings). Our engagement with the stakeholder group and policy-makers also ensured that data were ‘translated’ and interpreted in ways relevant to the UK context.

In running the stakeholder group we took care to express realist review terms in everyday language so as to avoid methodological jargon, while still adequately conveying the nuances of the review findings. Stakeholder involvement also contributed significantly to the development of actionable findings in a form that would be usable and engaging. More details on actionable findings emerging from the review are presented in the Discussion.

The stakeholder group included strong patient and public involvement throughout the project. DC led the patient and public involvement component of the review following MP’s guidance. DC organised a briefing session with the patient representatives (n = 4 in total) before the first stakeholder meeting, to set the ground rules and to discuss the terms of their involvement and any key issues that needed to be addressed to facilitate meaningful participation.60 In the stakeholder meetings, patients and members of the public provided significant input to programme theory development, often highlighting aspects and questioning assumptions that the rest of the group were taking for granted (e.g. the relational, financial, psychosocial complexity of the sickness absence pathway in the NHS, the importance of safe spaces in the workplace for doctors, and the complexities of making patients and the public aware that doctors are human and can become ill).

Steering Group

We set up a separate Steering Group to oversee the governance of the project and to provide high-level advice on dissemination strategies and conceptual aspects of the project. The group comprised one academic and clinician, one local NHS transformation manager and one patient representative. The range of expertise of the group included mental health services; nursing and complex intervention research methods; occupational stress in banking; stress involved in ‘fast-moving’ decision-making contexts, such as weather forecasting; and workforce management and resilience at both individual- and system-wide levels in the NHS.

Step 1: locating existing theories

In this first step of the review we identified theories that helped to understand (1) the processes leading to mental ill-health in doctors; and (2) how interventions aiming to support doctors experiencing mental ill-health are supposed to work (and for whom), when they work, when they do not, why they are not effective and why they are not being used. The rationale for this step is that interventions are ‘theories incarnate’ (i.e. interventions are underpinned by assumptions about why certain components are required). Such assumptions usually include a mix of scientific and experiential knowledge, including tacit assumptions that do not always become articulated (e.g. tacit knowledge).61 Interventions are designed in a certain way based on assumptions about what is needed to achieve desired outcome(s).62

To locate these assumptions, we iteratively drew on (1) discussions between DC and the clinical team of therapists working at the NHS Practitioner Health Programme; (2) informal discussions, advice and feedback from key content experts, representing multidisciplinary perspectives in our Stakeholder Group; and (3) an exploratory search of relevant literature.

Building the initial programme theory required iterative discussions (either at our regular face-to-face project team meetings or by e-mail) within the project team to make sense of and synthesise the different theories.

Step 2: searching for evidence

Main search

We developed a search strategy to identify examples of relevant programme theory in the published literature using MEDLINE via Ovid. Searching was designed, piloted and conducted by an information specialist (SB) in consultation with the review team. Search terms were derived from the titles, abstracts and indexing terms of relevant studies already known to the review team from background reading and consultation with stakeholders. These ‘empirically derived’ search terms were supplemented with relevant synonyms selected in consultation with the review team. Several versions of the search strategy were tested in MEDLINE via Ovid by checking that the relevant pre-identified studies were returned and by refining the search terms to optimise the sensitivity and specificity of the search (i.e. maximising the retrieval of known relevant studies while minimising the retrieval of irrelevant studies). In the process of testing and refining the search, we identified additional relevant studies which we also made sure were retrieved in subsequent iterations of the search.

The final search strategy consisted of the three components of the PICo (population, phenomenon of interest, context) question formulation tool:63

  • doctors and medical trainees (the population)
  • effects of mental ill-health in the workplace, such as presenteeism, absenteeism or burn out (the phenomenon of interest)
  • mental ill-health and workplace causes of mental ill-health, such as patient demand or work pressure (the context).

Each of the three components (population, phenomenon of interest, context) comprised relevant search terms combined using the OR Boolean operator. The three components were combined together using the AND Boolean operator. We found that this approach was the most effective way to retrieve our pre-identified set of papers and additional papers identified during the development of the search. Search terms included free-text terms (i.e. terms in the title and abstracts of bibliographic records) and indexing terms (e.g. medical subject headings in MEDLINE). We did not limit the search results by study type, date or language.

In December 2017 the final search strategy was translated and run in a selection of medical and psychology bibliographic databases, including MEDLINE, MEDLINE In-Process and Other Non-indexed Citations and PsycINFO (all via Ovid); and Applied Social Sciences Index and Abstracts (ASSIA) (via ProQuest). The MEDLINE search strategy is reproduced in Appendix 1. Search results were exported to EndNote (X8, Clarivate Analytics, Philadelphia, PA, USA) and de-duplicated using the automated deduplication feature and manual checking.

As our review was particularly focused on the UK context, we supplemented the bibliographic database searches described above (which were conducted in databases that index international literature) with searches that aimed to retrieve UK-based studies. We did this by conducting author, forwards and backwards citation searching on UK-based studies and first authors of studies identified by the bibliographic database searches. UK-based source studies and authors were identified by searching the EndNote library of bibliographic database results for terms such as UK, England, Wales, Ireland and Scotland, in the title and abstract fields. Forwards and author citation searches were conducted on relevant studies thus identified using Scopus and Web of Science. (We first searched for the source study or author and associated citations in Web of Science and if Web of Science did not index the item of interest we repeated the process in Scopus.) Backwards citation searching was conducted manually by inspecting the reference lists of UK-based studies. We also hand-searched journals with a UK focus via the relevant journal websites, including the British Medical Journal and BMC Medicine.

Step 3: selecting articles

The review was limited to English-language literature. We applied the following inclusion criteria:

  • Mental ill-health and its impacts (e.g. presenteeism, absenteeism and workforce retention) – all studies that focused on one or more of these aspects. Note that generic occupational health services targeting whole populations of doctors, rather than doctors experiencing mental ill-health, were not included. Studies about improving clinical practice (and the indirect effect this may have on doctors’ well-being) were labelled as not included/minor relevance.
  • Study design – all study designs.
  • Types of settings – all health-care settings.
  • Types of participants – all studies that included medical doctors.
  • Types of intervention – interventions or resources that focus on improving mental ill-health and minimising its impact.
  • Outcome measures – all mental health outcomes and measures relevant to its impacts (e.g. absenteeism, presenteeism and workforce retention).

Using EndNote, DC screened the titles and abstracts of all articles resulting from the main and supplementary searches (forward citation tracking and author citation tracking). A random 10% sample of the three sets of results were also screened independently for consistency of application of the inclusion criteria by CP (the second reviewer). Small inconsistencies were identified that were resolved through discussion. DC then screened the full texts of the papers resulting from the first round of screening and classified them in categories based on their potential to contribute to programme theory.

We had initially planned to sort included studies into those which could make ‘major’ or ‘minor’ contributions to our programme theory.62 By doing this we intended to prioritise studies from the UK, but also to include studies from other countries that provided useful insights for the UK. Our criteria for classifying studies as ‘major’ or ‘minor’ were as follows (see also Carrieri et al.58).


  • Studies that contributed to the research questions and were conducted in an NHS context.
  • Studies that contributed to the research questions and were conducted in contexts (e.g. publicly and universal-funded health-care systems) with similarities to the NHS.
  • Studies that contributed to the research questions and could clearly help to identify mechanisms which could plausibly operate in the context of the NHS.


  • Studies conducted in health-care systems that were markedly different from the NHS (e.g. fee for service and private insurance scheme systems), but where the mechanisms could plausibly operate in the context of doctors working in the NHS.

However, as the analysis progressed, we noted that even those documents that we had assumed would make a minor contribution still contained important and relevant data for our study, and hence we extracted and analysed data from all included documents.

The full text of a 10% sample of documents from the main search and a separate 10% sample of full texts from the supplementary searches were assessed and discussed between DC and CP to ensure that decisions for final inclusion and classification into categories have been made consistently. Small inconsistencies were identified that were resolved through discussion.

Step 4: extracting and organising data

Once article selection was finalised, DC analysed the full text of the included studies, using NVivo 12 Pro (QSR International, Warrington, UK) to manage the data. Initial coding followed both an inductive mode (codes emerging from the analysis of the literature) and a deductive mode (codes created in advance informed by the initial programme theory, stakeholder group discussions and exploratory literature searching). The coding framework resulting from the analysis of the richest papers (mostly UK policy documents, systematic reviews and qualitative research) was applied to the rest of the studies and refined as the analysis progressed. To ensure consistency in conceptual coding, CP assessed a random 10% sample of coded articles.

The analysis was driven by a realist logic. We sought to interpret and explain mechanisms causing mental ill-health in doctors and medical students (with a particular focus on presenteeism, absenteeism and workforce retention), and to identify relevant contexts or circumstances when these mechanisms were likely to be ‘triggered’. These contexts and mechanisms became our ‘causative factors’ codes. Examples of preliminary ‘causative codes’ included organisation and training levels, doctors’ profession and identity. We simultaneously sought to identify ‘guiding principles’ and features underpinning the interventions and recommendations discussed mostly in policy document, reviews and commentaries. The juxtaposition of these ‘guiding principles’ (underpinning interventions and recommendations) with the ‘causative factors’, allowed us to identify particular configurations of mechanisms and contexts that were more likely to reduce or prevent mental ill-health in doctors, as well as important limits and barriers to the access and effectiveness of such interventions. An obvious example was the link between the ‘coherence’ guiding principle code and the ‘lack of trust towards employer and loss of control’ causative factors code. This link highlighted the importance for doctors to have confidence in an intervention, whereas included studies reported instances in which there was lack of coherence between an intervention and the context in which it was implemented (e.g. the employer or leadership level failing to properly communicate such intervention and/or to making it accessible to the workforce).

We compared and contrasted these configurations of context-mechanism-outcome configurations (CMOcs) with the evolving programme theory, so as to understand the place of and relationships between each CMOc within the programme theory. As the review progressed we iteratively refined the programme theory driven by interpretations of the data included in the literature, and by feedback received by our stakeholders (Figure 2).

FIGURE 2. Programme theory development process.


Programme theory development process. Reproduced from Papoutsi et al. Contains information licensed under the Non-Commercial Government Licence v2.0.

We also coded articles for more descriptive categories, such as relevant background information, study characteristics and recommendations provided. The characteristics of the documents were summarised in Table 2 (see Appendices 24).



Characteristics of the 179 included studies

The aim of the analysis was to reach theoretical saturation, in that sufficient information has been captured to portray and explain the processes leading to mental ill-health and the mechanisms that can remedy this situation. Excerpts coded under specific concepts in NVivo were then exported into a Microsoft Word document (Microsoft Corporation, Redmond, WA, USA). Drawing on the analysis of the literature done in NVivo, Word documents were used as coding reports, to provide a more flexible space to test the viability of different CMOcs and build the narrative of the synthesis. This included adding explanatory text through abductive and retroductive analysis (see Step 5: synthesising the evidence and drawing conclusions).

Step 5: synthesising the evidence and drawing conclusions

We used a realist logic of analysis to analyse and synthesise the data. We used the coding of the included studies conducted on NVivo to draw relationships between contexts, mechanisms and outcomes, and to develop our initial programme theory. To develop and refine the CMOcs, and the programme theory, we made judgements about the relevance and rigour of content within included articles following a series of questions that are commonly used in realist reviews (see also Papoutsi et al.54).


  • Are the contents of a section of text within an included document referring to data that might be relevant to programme theory development?

Judgements about trustworthiness and rigour

  • Are the data sufficiently trustworthy to warrant making changes (if needed) to the programme theory?

Interpretation of meaning

  • If the section of text is relevant and trustworthy enough, do its contents provide data that may be interpreted as functioning as context, mechanism or outcome?

Interpretations and judgements about CMOcs

  • What is the CMOc (partial or complete) for the data?
  • Are there data to inform CMOcs contained within this document or other included documents? If so, which other documents?
  • How does this CMOc relate to CMOcs that have already been developed?

Interpretations and judgements about programme theory

  • How does this (full or partial) CMOc relate to the programme theory?
  • Within this same document are there data which inform how the CMOc relates to the programme theory? If not, are there data in other documents? Which ones?
  • In light of this CMOc and any supporting data, does the programme theory need to be changed?

We used abductive and/or retroductive reasoning (see Glossary), particularly to infer and elaborate on mechanisms (which often remained hidden or were not articulated adequately). This means that we followed a process of constantly moving from data to theory, in order to refine explanations about why certain behaviours are occurring, and tried to frame these explanations at a level of abstraction that could cover a range of phenomena or patterns of behaviour.

We sought relationships between contexts, mechanisms and outcomes both within the same included study and across different sources (e.g. mechanisms inferred from one study could help explain how contexts influenced outcomes in a different study). Therefore, we often synthesised data from different sources to compile CMOcs, as not all parts of the CMOcs were always articulated in the same source.

In summary, the process of evidence synthesis was achieved by the following analytic processes.237

  • Juxtaposition of data sources: comparing and contrasting between data presented in different articles. For example, when data about mental ill-health in doctors in an in-depth qualitative source enabled insights into how outcomes are achieved, as described in a quantitative study.
  • Reconciling ‘contradictory’ or disconfirming data: when outcomes differ in apparently similar circumstances, further investigation is necessary to find explanations for why these different results occurred. This involved a closer consideration of context and what counts as context for different types of ‘problems’, in order to understand how the mechanisms triggered can explain differences in outcomes.
  • Consolidation of sources of evidence: when there are similarities between findings presented in different sources, a judgement needs to be made about whether these similarities are adequate to form patterns in the development of CMOcs and programme theory, or whether there are nuances that need to be highlighted, and to what end.
Copyright © Queen’s Printer and Controller of HMSO 2020. This work was produced by Carrieri et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
Bookshelf ID: NBK555653


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (5.8M)

Other titles in this collection

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...