Research plan

We followed a similar methodology to that used in a previous National Institute for Health Research Health Services and Delivery Research programme project⁵⁴ that was conducted by many of the same research team members.^54,55 Any evidence synthesis that seeks to make sense of interventions aiming to improve doctors’ mental ill-health must take into account the contexts in which these interventions are situated. This generates an in-depth understanding of which components within these interventions matter more (or less) than others, for whom they matter and in what ways. A realist review can synthesise relevant data found within qualitative, quantitative and mixed-methods research. By following an interpretive, theory-driven approach to analysing data from such diverse literature sources, realist reviews move beyond description, to provide findings that coherently and transferably explain how and why contexts can influence outcomes. This is particularly relevant to complex programmes characterised by significant levels of heterogeneity. The plan of investigation followed a detailed protocol based on Pawson et al.’s⁵⁶ five iterative stages for realist reviews: (1) locating existing theories, (2) searching for evidence, (3) selecting articles, (4) extracting and organising data and (5) synthesising the evidence and drawing conclusions. The reporting is consistent with the Realist And Meta-narrative Evidence Syntheses: Evolving Standards (RAMESES) quality and publication standards for realist reviews.⁵⁷ The project ran for 18 months from November 2017 to April 2019. Regular project team meetings and stakeholder meetings throughout the lifecycle of the project ensured that multiple perspectives and interpretations were brought to bear on the research.

The protocol has been published in BMJ Open⁵⁸ and the review has been registered with PROSPERO (CRD42017069870). The review design and methodology is explained in more detail in the sections below and illustrated in Figure 1.

FIGURE 1

Flow diagram of the project. Note that dashed arrows indicate iteration where necessary. Reproduced from Wong et al. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which (more...)

Stakeholder group

A stakeholder group was recruited for the CUP review to provide content expertise, feedback on and refinement of our programme theory and to coproduce non-academic outputs. The stakeholder group did not provide us with data, but feedback and advice based on their expertise and experience. A total of 22 people were involved in the stakeholder group during the review, including patient representatives, clinicians, doctors in training, medical educators and academics. Consultations with stakeholder group members took place as part of 2-hour meetings at regular intervals throughout the project and e-mail exchange. Additional ‘satellite’ face-to-face meetings were also arranged with some stakeholders who could not attend the main meetings, as well as Skype™ (Microsoft Corporation, Redmond, WA, USA) and telephone conversations. Table 1 provides a list of the face-to-face meetings, including the number of participants and key topics discussed.

TABLE 1

Stakeholder group meetings summary

Stakeholder group meetings took place at the University of Exeter or in London. To leave more time for discussion, we circulated any relevant preparatory material [e.g. summaries of previous meeting(s)], ahead of meetings when necessary. The meetings usually started with a brief slide presentation by our project team to introduce stakeholders to the topic under discussion and realist methods, and to provide a quick update on progress with the review. We presented high-level programme theory to the group in the form of statements and visual prompts to obtain their feedback. Discussions were designed to be more open ended in the early stages of the review, but focused on particular aspects of the programme theory as the project progressed. Later stakeholder groups focused on actionable findings and dissemination of the study. We used a framework called the evidence integration triangle (EIT)⁵⁹ to ensure that discussions focused on the real-world impact of our study. The three components of the EIT (practical evidence-based interventions; pragmatic, longitudinal measures of progress; and participatory implementation processes) were used to structure discussions at the stakeholder group meetings and the workshop with policy-makers at the end of the project. Of the three components, the pragmatic longitudinal measures of progress proved by far the most challenging (see Chapter 5, Strengths and limitations of this review). Facilitation of the meetings ensured that everyone was able to contribute and voice their opinion, whether in agreement or disagreement. Notes from these meetings were used to set direction for the review and to refine programme theory, rather than as primary data for analysis, and the report does not include any verbatim data excerpts from these meetings.

Discussions with stakeholders helped ground the review in the practical reality experienced by participants and the challenges they faced in their respective roles. The sharing of these experiences re-enforced our decisions at times (e.g. to critically question the focus on ‘resilience’ in doctors) and prompted us to pay close attention to aspects that we had missed at others (e.g. the taboo around doctors expressing vulnerability). Stakeholders’ questions and contributions also prompted us to look more broadly in the literature for substantive explanatory theory, for example leading us to identify social cure theory and an emerging body of work on vulnerability that we drew on to shape our thinking and interpretation (for more information about the substantive theories used see Chapter 3, Summary of the 19 CMOcs in four main groupings). Our engagement with the stakeholder group and policy-makers also ensured that data were ‘translated’ and interpreted in ways relevant to the UK context.

In running the stakeholder group we took care to express realist review terms in everyday language so as to avoid methodological jargon, while still adequately conveying the nuances of the review findings. Stakeholder involvement also contributed significantly to the development of actionable findings in a form that would be usable and engaging. More details on actionable findings emerging from the review are presented in the Discussion.

The stakeholder group included strong patient and public involvement throughout the project. DC led the patient and public involvement component of the review following MP’s guidance. DC organised a briefing session with the patient representatives (n = 4 in total) before the first stakeholder meeting, to set the ground rules and to discuss the terms of their involvement and any key issues that needed to be addressed to facilitate meaningful participation.⁶⁰ In the stakeholder meetings, patients and members of the public provided significant input to programme theory development, often highlighting aspects and questioning assumptions that the rest of the group were taking for granted (e.g. the relational, financial, psychosocial complexity of the sickness absence pathway in the NHS, the importance of safe spaces in the workplace for doctors, and the complexities of making patients and the public aware that doctors are human and can become ill).

Step 1: locating existing theories

In this first step of the review we identified theories that helped to understand (1) the processes leading to mental ill-health in doctors; and (2) how interventions aiming to support doctors experiencing mental ill-health are supposed to work (and for whom), when they work, when they do not, why they are not effective and why they are not being used. The rationale for this step is that interventions are ‘theories incarnate’ (i.e. interventions are underpinned by assumptions about why certain components are required). Such assumptions usually include a mix of scientific and experiential knowledge, including tacit assumptions that do not always become articulated (e.g. tacit knowledge).⁶¹ Interventions are designed in a certain way based on assumptions about what is needed to achieve desired outcome(s).⁶²

To locate these assumptions, we iteratively drew on (1) discussions between DC and the clinical team of therapists working at the NHS Practitioner Health Programme; (2) informal discussions, advice and feedback from key content experts, representing multidisciplinary perspectives in our Stakeholder Group; and (3) an exploratory search of relevant literature.

Building the initial programme theory required iterative discussions (either at our regular face-to-face project team meetings or by e-mail) within the project team to make sense of and synthesise the different theories.

Step 2: searching for evidence

Step 3: selecting articles

The review was limited to English-language literature. We applied the following inclusion criteria:

• Mental ill-health and its impacts (e.g. presenteeism, absenteeism and workforce retention) – all studies that focused on one or more of these aspects. Note that generic occupational health services targeting whole populations of doctors, rather than doctors experiencing mental ill-health, were not included. Studies about improving clinical practice (and the indirect effect this may have on doctors’ well-being) were labelled as not included/minor relevance.
• Study design – all study designs.
• Types of settings – all health-care settings.
• Types of participants – all studies that included medical doctors.
• Types of intervention – interventions or resources that focus on improving mental ill-health and minimising its impact.
• Outcome measures – all mental health outcomes and measures relevant to its impacts (e.g. absenteeism, presenteeism and workforce retention).

Using EndNote, DC screened the titles and abstracts of all articles resulting from the main and supplementary searches (forward citation tracking and author citation tracking). A random 10% sample of the three sets of results were also screened independently for consistency of application of the inclusion criteria by CP (the second reviewer). Small inconsistencies were identified that were resolved through discussion. DC then screened the full texts of the papers resulting from the first round of screening and classified them in categories based on their potential to contribute to programme theory.

We had initially planned to sort included studies into those which could make ‘major’ or ‘minor’ contributions to our programme theory.⁶² By doing this we intended to prioritise studies from the UK, but also to include studies from other countries that provided useful insights for the UK. Our criteria for classifying studies as ‘major’ or ‘minor’ were as follows (see also Carrieri et al.⁵⁸).

Step 4: extracting and organising data

Once article selection was finalised, DC analysed the full text of the included studies, using NVivo 12 Pro (QSR International, Warrington, UK) to manage the data. Initial coding followed both an inductive mode (codes emerging from the analysis of the literature) and a deductive mode (codes created in advance informed by the initial programme theory, stakeholder group discussions and exploratory literature searching). The coding framework resulting from the analysis of the richest papers (mostly UK policy documents, systematic reviews and qualitative research) was applied to the rest of the studies and refined as the analysis progressed. To ensure consistency in conceptual coding, CP assessed a random 10% sample of coded articles.

The analysis was driven by a realist logic. We sought to interpret and explain mechanisms causing mental ill-health in doctors and medical students (with a particular focus on presenteeism, absenteeism and workforce retention), and to identify relevant contexts or circumstances when these mechanisms were likely to be ‘triggered’. These contexts and mechanisms became our ‘causative factors’ codes. Examples of preliminary ‘causative codes’ included organisation and training levels, doctors’ profession and identity. We simultaneously sought to identify ‘guiding principles’ and features underpinning the interventions and recommendations discussed mostly in policy document, reviews and commentaries. The juxtaposition of these ‘guiding principles’ (underpinning interventions and recommendations) with the ‘causative factors’, allowed us to identify particular configurations of mechanisms and contexts that were more likely to reduce or prevent mental ill-health in doctors, as well as important limits and barriers to the access and effectiveness of such interventions. An obvious example was the link between the ‘coherence’ guiding principle code and the ‘lack of trust towards employer and loss of control’ causative factors code. This link highlighted the importance for doctors to have confidence in an intervention, whereas included studies reported instances in which there was lack of coherence between an intervention and the context in which it was implemented (e.g. the employer or leadership level failing to properly communicate such intervention and/or to making it accessible to the workforce).

We compared and contrasted these configurations of context-mechanism-outcome configurations (CMOcs) with the evolving programme theory, so as to understand the place of and relationships between each CMOc within the programme theory. As the review progressed we iteratively refined the programme theory driven by interpretations of the data included in the literature, and by feedback received by our stakeholders (Figure 2).

FIGURE 2

Programme theory development process. Reproduced from Papoutsi et al. Contains information licensed under the Non-Commercial Government Licence v2.0.

We also coded articles for more descriptive categories, such as relevant background information, study characteristics and recommendations provided. The characteristics of the documents were summarised in Table 2 (see Appendices 2–4).

TABLE 2

Characteristics of the 179 included studies

The aim of the analysis was to reach theoretical saturation, in that sufficient information has been captured to portray and explain the processes leading to mental ill-health and the mechanisms that can remedy this situation. Excerpts coded under specific concepts in NVivo were then exported into a Microsoft Word document (Microsoft Corporation, Redmond, WA, USA). Drawing on the analysis of the literature done in NVivo, Word documents were used as coding reports, to provide a more flexible space to test the viability of different CMOcs and build the narrative of the synthesis. This included adding explanatory text through abductive and retroductive analysis (see Step 5: synthesising the evidence and drawing conclusions).

Step 5: synthesising the evidence and drawing conclusions

We used a realist logic of analysis to analyse and synthesise the data. We used the coding of the included studies conducted on NVivo to draw relationships between contexts, mechanisms and outcomes, and to develop our initial programme theory. To develop and refine the CMOcs, and the programme theory, we made judgements about the relevance and rigour of content within included articles following a series of questions that are commonly used in realist reviews (see also Papoutsi et al.⁵⁴).