• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of trialsBioMed CentralBiomed Central Web Sitesearchsubmit a manuscriptregisterthis articleTrialsJournal Front Page
Trials. 2009; 10: 68.
Published online Aug 13, 2009. doi:  10.1186/1745-6215-10-68
PMCID: PMC2736948

Primary care-based multifaceted, interdisciplinary medical educational intervention for patients with systolic heart failure: lessons learned from a cluster randomised controlled trial

Abstract

Background

Chronic (systolic) heart failure (CHF) is a common and disabling condition. Adherence to evidence-based guidelines in primary care has been shown to improve health outcomes. The aim was to explore the impact of a multidisciplinary educational intervention for general practitioners (GPs) (Train the trainer = TTT) on patient and performance outcomes.

Methods

This paper presents the key findings from the trial and discusses the lessons learned during the implementation of the TTT trial. Primary care practices were randomly assigned to the TTT intervention or to the control group. 37 GPs (18 TTT, 19 control) were randomised and 168 patients diagnosed with ascertained CHF (91 TTT, 77 control) were enrolled. GPs in the intervention group attended four meetings addressing clinical practice guidelines and pharmacotherapy feedback. The primary outcome was patient self-reported quality of life at seven months, using the SF-36 Physical Functioning scale. Secondary outcomes included other SF-36 scales, the Kansas City Cardiomyopathy Questionnaire (KCCQ), total mortality, heart failure hospital admissions, prescribing, depressive disorders (PHQ-9), behavioural change (European Heart Failure Self-Care Behaviour Scale), patient-perceived quality of care (EUROPEP) and improvement of heart failure using NT-proBNP-levels. Because recruitment targets were not achieved an exploratory analysis was conducted.

Results

There was high baseline achievement in both groups for many outcomes. At seven months, there were no significant mean difference between groups for the primary outcome measure (-3.3, 95%CI -9.7 to 3.1, p = 0.30). The only difference in secondary outcomes related to the prescribing of aldosterone antagonists by GPs in the intervention group, with significant between group differences at follow-up (42 vs. 24%, adjusted OR = 4.0, 95%CI 1.2–13; p = 0.02).

Conclusion

The intervention did not change the primary outcome or most secondary outcomes. Recruitment targets were not achieved and the under-recruitment of practices and patients alongside a selection bias of participating GPs, prohibit definite conclusions, but the CI indicates a non-effectiveness of the intervention in this sample. We describe the lessons learned from conducting the trial for the future planning and conduct of confirmatory trials in primary care.

Trial registration

ISRCTN08601529.

Background

Chronic (systolic) heart failure (CHF) is a "common, disabling, deadly, and costly disease" often resulting in hospital admissions with a prevalence rate of approximately 1% in Germany[1,2]. New treatments and care strategies tend to focus on the prevention of hospital admissions and improving prognosis. The adherence of physicians to clinical practice guidelines (CPG)[3] and patients to treatment regimens [4], have been found to be predictors of better patient outcomes[5]. However, there are barriers[6] that must be overcome in order to ensure adequate communication between physicians and patients[7] and in the delivery of evidence-based care [8-10]. Furthermore, deteriorating CHF is associated with decreasing quality of life (QoL)[11,12]. Studies suggest that QoL is predictive of the course of CHF, independent of established somatic predictors of prognosis (such as the left ventricular ejection fraction)[12,13]. Increasing evidence shows that psychological comorbidities also determine QoL in patients with CHF[14].

In recent years, strategies designed to translate this evidence in to practice have consisted of changes in the organisation, delivery and specialisation of care by trained nurses[15], pharmacists[16], and call-centres providing home-based or telephone support or telemedicine [17-19]. These studies have primarily been conducted in a post-discharge setting and addressed the importance of adherence, recognising and acting upon deteriorating symptoms and optimising established pharmacological treatments.

The majority of CHF patients have contact with General Practitioners. However, the evidence for the effectiveness and efficiency of general practice based clinical practice guidelines is limited [20-22]. Combinations of interventions seem most promising[23], along with evidence-based educational strategies for physicians[24,25]. However, interventions that are designed to change GP behaviour must be realistic in terms of the effects of educational strategies on GP knowledge during their careers[26].

The purpose of this study was to develop an educational model for GPs for the management of CHF and to evaluate it as randomised controlled trial. The original aim was to conduct a confirmatory trial. However, under-recruitment of participants meant that we conducted an exploratory trial of a complex intervention combining educational sessions with medication feedback. This paper reports the main findings from the trial and focuses upon the lessons learned from its planning and conduct.

Methods

Participants – Recruitment and Assignment

In a single mail-out, we invited GPs from the region of Northern Baden, Germany, to participate in the study. GPs were eligible for participation if they were certified as a GP and practised as a SHI (statutory health insurance)-affiliated physician. GPs were excluded if they provided unconventional therapies or provided specialist services (e.g. HIV-therapy).

GPs were expected to recruit an average of 7–8 patients in to the study. GPs received a screening algorithm for case finding using electronic medical records. Eligible patients were adults (aged over 40 years) with objective left- or biventricular heart failure, NYHA functional class II-IV, with ascertained ejection fraction of 40% or less (e.g. by echocardiography), with stable symptoms at the time of inclusion, and diagnosis of a chronic, irreversible CHF at least 3 months prior to inclusion. We excluded patients with primary valvular heart diseases and relevant hemodynamic effects, hypertrophic obstructive/restrictive cardiomyopathy (HOCM/RCM), and people with a concomitant terminal illness, dementia or severe psychological illness. We obtained informed consent from all participants.

After recruitment of practices and patients, stratified randomisation of practices was carried out by an external third party using the "single-coin method" based on a computer generated list. The third party stratified practices according to the number of participating patients per practice. Intervention allocations were concealed from particpating GPs and the intervention team by the third party until shortly before the intervention started in October 2005.

Intervention

We conducted a focus group with 13 GPs who were not actively participating in the TTT study. The content of the educational sessions was based on the discussions of this focus group. Both educational interventions were based on predefined learning targets relevant to heart failure including psychosomatic aspects (comorbid depression and anxiety disorders, quality of life, and compliance). We focused on the optimisation of evidence-based pharmacotherapy and the detection of depressive comorbid disorders in patients with CHF. The intervention team consisted of one GP, one cardiologist and one specialist in psychosomatic medicine. The new intervention group (Train the trainer = TTT) participated in a multidisciplinary, andragogic (e.g. inputs of theoretical knowledge with group work in "sandwich technique"), and didactic (e.g. problem-based learning) training course including elements of specific knowledge (e.g. CPG content), communication skills (standardised patients with video observation and peer feedback), detection and management of comorbid mental disorders (diagnostic properties of screening tools) as well as organisation of the practice in order to implement newly acquired knowledge ("my plan for my practice"). For the same reason, structured case-discussions were held on real patients. The interventions are depicted in Figure Figure1,1, as recently recommended[27]. Additionally, GPs in the TTT group received medication feedback for individual patients participating in the study (data from baseline documentation of patients). The existing percentage of patients receiving the target dose of ACE-inhibitors (or angiotensin-II receptor antagonists) and beta-blockers was calculated using current guidelines as shown previously and printed out as a graphical depiction for each patient[28].

Figure 1
Graphical depiction of the (i) timeline, (ii) composition and duration (' = minutes), (iii) content and (iiii) educational format (EF) of the educational interventions; PM = printed material. Objects are represented by squares (to reflect their fixed ...

GPs from the control group received a single three-hour lecture by a senior cardiologist with extensive didactic expertise based on the aforementioned predefined learning targets.

Masking

The nature of the intervention entailed that participating physicians and individuals of the intervention team were unblinded, while patients were not informed about group assignment. No interim analyses were conducted during the observation period, with the exception of data collection on heart failure medication for pharmacotherapy feedback. Neither the study statisticians nor the data monitoring committee saw unblinded data or had any form of contact with study participants.

Objectives

We investigated whether the intervention would improve health-related quality of life (QoL), reduce hospital admissions or mortality, improve self care behaviour, improve satisfaction with provided care and improve disease course in patients with CHF. Compared with the control group, we hypothesised that the new intervention would result in an improvement in QoL (SF-36, Physical Functioning scale) of at least 6.6 points.

Outcomes

The primary outcome was self-assessed quality of life at follow-up at seven months, i.e. Subscale 1 (Physical Functioning) of the German version of the SF-36[29,30] (a generic, multidimensional instrument). Secondary outcomes included further SF-36 subscales, the German version of the Kansas City Cardiomyopathy Questionnaire (KCCQ)[31] (a disease-specific instrument) (KCCQ-os). A mean five-point change in the scales of the SF-36[30,32] and in the overall summary score (KCCQ-os)[33] represents a clinically significant difference. Further pre-specified secondary outcomes were total mortality or heart failure hospital admissions, depressive disorders measured using the PHQ-9[34,35], the European Heart Failure Self-Care Behaviour Scale (EHFScBS)[36], patient-perceived quality of care (EUROPEP)[37], and improvement of heart failure according to NT-proBNP-levels[38,39].

Data collection

The trial was part of the Competence Network of Heart Failure and conducted as a clinical trial according to the principles of ICH/GCP (for example: a list of involved study personnel, screening of eligible patients, informed consent, registration and pseudonymisation of patients)[40]. GPs received an initiation visit by a study nurse, which included an introduction to the trial's investigator file. GPs collected and documented clinical data (history, current clinical status), laboratory results, ECG, detailed medication etc., discontinuation of the study by the patient, and mortality data on pre-specified case report forms according to the Basic Clinical Dataset (BCD) of the Competence Network of Heart Failure a nationwide interdisciplinary research project involving medical doctors and scientists from university and other clinics, research institutes, heart centers, medical practices as well as organisations, associations and industry http://www.knhi.de[41]. This network aims to obtain an additional value for patients, clinicians and scientists by multidimensional connection of these numerous partners by conducting – amongst others – clinical trials). They also took blood samples for the determination of NT-proBNP at baseline and follow-up. NT-proBNP was measured using the Elecsys 2010 analyser from Roche Diagnostics, Germany. GPs also documented the number of practice visits by patients, referrals to a cardiologist, and hospital admissions.

Sample size calculation

For the primary outcome, we took into account the expected small effect of the intervention and the natural deterioration of QoL, esp. in the scale physical functioning. Therefore, we assumed mean small decline of 3.3 points in the control group due to the natural disease trajectory and a small mean improvement under the intervention by 3.3 points, resulting in the expected net difference of 6.6 points in QoL in favour of the intervention. The standard deviation from cross-sectional analyses is reported to be 23[30]. Using the serial correlation from baseline to follow-up measurements of 0.6[30], we can calculate the anticipated variance (SD) of the difference of 423.2 (20.57). Therefore, in a two-sided sample calculation for 80% power at an α-level of 5%, 154 patients were needed per arm. To account for the clustered design, the literature indicated that intra-class correlation coefficients (ICC = GP variance/[GP variance + patient variance]) of health outcomes (like QoL) are generally lower than 0.05[42,43]. As the German primary care is characterised by small, office-based practices, we could expect that each participating GP would come from another practice. Our assumption was that the mean change of GP would deviate less than ± 5 points from the overall mean in 95% of all practices. This results in a SD (Variance) of GP of 2.5 (6.25) and an ICC of 0.015 to account for the variable responses of GPs and patients to the intervention. Thus, with k = number of GPs (60), n = sample-size from conventional planning (308), we obtain the total N = k*n*(1-ICC)/(k-n*ICC) = 329. We anticipated a drop out rate of 30%[2], as a result of which 329 cases would provide 70% power. Accordingly, 329*10/7 = 470 cases represented 100%. In order to obtain a total sample of 470 patients, we planned to include 60 practices with 7–8 patients per practice, assuming a practice prevalence of 1% similar to the estimated average prevalence in the German population[2,44].

Statistical methods

In order to identify group differences in the baseline characteristics of GPs and patients, we used chi-square and t-tests. For continuous dependent variables (score at follow-up), comparisons between the intervention and control practices were assessed using linear mixed effects regression models (SAS 9.1 proc mixed) with the practice as a random effect nested in groups. These analyses accounted for the intraclass correlation within each practice attributable to clustering from randomisation according to practice. The analysis model included the fixed effect group (intervention vs. control), and the covariates score at baseline, age, sex, and NYHA functional class.

We analysed the effect of the intervention on binary outcomes using SAS proc glimmix and generalised linear mixed effects models with a logit link. We analysed the effect of the intervention on count-data outcomes using generalised linear mixed effects models with a poisson link, and accounting for overdispersion. Continuous outcomes are presented as adjusted mean differences and binary or count outcomes as adjusted odds-ratios (ORs) with 95% confidence intervals (CIs).

Results

Participant flow and follow-up

Figure Figure22 shows the flow of participating practices (=GPs) and patients through the trial. We approached a total of 750 GPs in a single mail-out in October 2004. 667 did not respond; 26 were interested, but finally refused to participate due to work load; and 20 failed to find eligible patients. 37 GPs ultimately participated. Between March and September 2005, these GPs screened 17416 patients for eligibility and ultimately recruited 168 eligible patients in 37 practices. Following patient recruitment, we randomised 18 GPs to the TTT group and 17 GPs to the Standard care group. Both groups displayed similar cluster sizes. Some recruiting GPs failed to recruit the intended number of eligible patients. At the participant level, 14 patients were ultimately lost in the TTT group and 13 in the Standard group (for details see Figure Figure2).2). Table Table11 shows similar characteristics for GPs in the TTT and Standard group. Table Table22 shows similar baseline characteristics for participants in the TTT and Standard group [45].

Table 1
Baseline characteristics of all 37 participating general practices from Baden-Wuerttemberg.
Table 2
Baseline comparison of intervention and control group patients (n = 168).
Figure 2
Flow of clusters and participants through the trial.

Interventions

Only few participating GPs from TTT sporadically missed one of the educational sessions, all of which were held at the University of Heidelberg. The IG (n = 18) was divided into two subgroups: TTT subgroups (n = 10/n = 8) came to the educational meetings in October (10/8) and November 2005 (8/8), and – after new group formation – in January (7/9) and March (9/8) 2006. The Standard group (n = 19) attended a lecture in October 2005.

Psychosocial outcomes

Table Table33 shows all patient-reported outcomes at baseline and at seven-month follow-up. These were the scores for Quality of life (SF36- and KCCQ), Behaviour change (EHFScBs), Patient satisfaction (EUROPEP), and Comorbid Depression (PHQ-9).

Table 3
Mean generic quality of life (SF-36), disease-specific quality of life (KCCQ), European heart failure self-care behaviour scale (EHFScBs), patient satisfaction (EUROPEP) and depression status (PHQ-9-D) scores for groups at baseline and seven-month follow-up ...

Somatic and performance outcomes

Mortality or hospital admissions

Mortality data were available for all participants. More deaths occurred in the TTT group than in the Standard group (9 vs. 4) (Table (Table4),4), although this difference was not statistically significant (adjusted odds ratio = 2.0, 95% confidence interval 0.6 to 7.1; P = 0.27). A total of 31 heart failure admissions occurred in the TTT group and 34 in the Standard group (adjusted odds ratio = 0.8, 95% confidence interval 0.3 to 2.1; P = 0.63). The combined end point (mortality or heart failure admission) resulted in 29 cases in the TTT group and 18 cases in the Standard group. The Poisson model indicated that this difference between groups was not significant (adjusted odds ratio = 1.4, 95% confidence interval 0.7 to 2.9; P = 0.35).

Table 4
Heart failure hospital admissions and total mortality during seven-month trial follow-up, selected prescription rates for patients at seven-month trial follow-up, mean NT-proBNP-values at baseline and seven-month follow-up and primary care activity data ...

Primary care activity data

Practice attendances by patients and referral rates are shown in Table Table4.4. There were no statistically significant differences between the groups. Prescription rates for evidence-based pharmacotherapy at follow-up were high, e.g. for the combination of an ACE inhibitor or an angiotensin-2 receptor antagonist with a β-blocker we found a rate of 76.5% for the IG and 77.9% for the CG, although this group difference adjusted for baseline rates (adjusted odds ratio 0.8, 95% confidence interval 0.2 to 2.9; P = 0.78) was not significant. With a prescription rate of 42.4% for aldosterone antagonists at seven-month follow-up, TTT showed a significantly greater increase than Standard with a prescription rate of 23.5% (adjusted odds ratio 4.0, 95% confidence interval 1.2 to 13.0; P = 0.02).

NT-proBNP

Mean NT-proBNP plasma levels (SD) were 2462.5 pg/ml (2821.5) in TTT (n = 87) and 2732.2 pg/ml (5793.5) in Standard (n = 69) (Table (Table4).4). Mean levels decreased in both groups. There was a non-significant difference between the groups (adjusted mean difference for the transformed levels = 0.17 points, 95% confidence interval 0.04 to 0.39; P = 0.11).

Discussion

Summary of main findings

The low power of the trial due to under-recruitment does not allow definitive conclusions regarding the effectiveness of the intervention in terms of the primary outcome (generic QoL, i.e. scale physical functioning of SF-36), but failed to show effectiveness in a seven-month follow-up. The intervention combined multifaceted, interdisciplinary educational sessions with pharmacotherapy feedback for GPs and was compared to a control group that received only a standard lecture. In terms of further patient-reported outcomes, we found impaired QoL, but good self-care and a generally high level of satisfaction with care in both groups, with no significant differences or changes. The intervention led to a significantly higher increase in the prescription rate of aldosterone antagonists and seemed to lead to a concomitant increase in primary care activity (by increase of practice attendances).

The following discussion is structured along the descriptive and exploratory character of the study and includes the lessons we have learned from planning and conducting the trial.

Comparison with existing literature

We compared QoL, self-care, patient satisfaction with care and performance measured by guideline adherence. Our study confirms that patients with CHF report impaired QoL[11]. Trials with a positive impact on the QoL in heart failure patients, typically use complex and multidisciplinary interventions at the patient level (heart failure programmes) in a post-discharge setting[46,47]. These patients were by definition unstable as they were hospitalised due to heart failure and more symptomatic ("ill") with a lower QoL at enrolment, while not necessarily more "diseased", as indicated by the past medical history and substantial comorbidities[46,47]. Patients in our trial were enrolled in primary care and had stable CHF at enrolment.

Although reporting comorbidity and impaired QoL scores, our patient sample revealed a higher level of self-care than previously reported[36]: Mean sum scores of around 24 (SD: 6.8–7.9) were better than in the sample from the validation study in which 442 patients from two centres in Sweden, three in the Netherlands and one in Italy were included[36]. Mean sum scores were 33.3 (7.8) vs. 29.6 (9.0) for subgroups extra care vs. standard care (defined by extra patient education about heart failure). Data on self-care from a trial testing community pharmacists showed similar high ("bad") baseline scores: Mean (SD) scores were 31.1 (8.7) vs. 30.6 (9.1) at baseline. Self-care improved in both groups at three and six month's follow-up (26.1 vs. 26.6 and 26.6 vs. 28.3, respectively). An intensive heart failure programme in a physician-nurse directed heart failure clinic had better, similar baseline scores to our sample (23.6 vs. 25.5 for intervention and control groups). The intervention improved self-care in a 3 months follow up, while scores at 12 months were similar as at baseline again. Still, there were significant between group differences because of worsening self-care scores in the control group (30.2 at 12 months). This worsening self-care was not found in our (active) control group (at 7 months).

The high overall satisfaction with care scores in our sample are similar to those found in the German sample in a European comparative patient survey in general practice using the EUROPEP instrument[37].

Comparable educational interventions in the CME setting have proven their efficacy in other clinical fields[24,25]; outcomes are typically knowledge or performance based rather than using patient-related outcomes: For CHF, educational programmes for GP peer review groups have not been shown to be beneficial for performance as assessed by prescription behaviour[48]. Compared with the literature (from 2002), GPs in our study (2005) showed high guideline adherence with regard to the prescription of evidence-based pharmacotherapy[8]. Even more recent studies suggest potential for improvement with regard to evidence-based pharmacotherapy [48-50], although the samples in these studies are of minor validity due to a lack of ascertained left ventricular systolic dysfunction (LVSD) and therefore not directly comparable. LVSD remains the crucial indication for prescribing evidence-based pharmacotherapy, preferably towards target doses[3,9,51]. Therefore, a rationale for the TTT-intervention was that patients with LVSD and a suspected low level of care could benefit from optimised treatment[3,51]. However, there is still a lack of evidence for the effectiveness of (both non-pharmacological and pharmacological) treatment of patients with "undefined" (e.g. diastolic) heart failure ("elderly female patient in general practice")[52].

In our patient sample, prescription rates of ACE-inhibitors (or A2RA) and beta-blockers were "already" very high at baseline, allowing primarily up-titration towards target doses[28]. This effect of TTT could be shown in a separate, more detailed analysis on guideline adherence[53]. Generally, interventions to improve guideline adherence to the prescription and target dosing of ACE-inhibitors (a hospital-based intervention) [54] or beta-blockers with mixed results [55,56] are more successful if they succeed in professional engagement[55]. The latter was the case in our tailored intervention.

Strengths and limitations of the study

Internal Validity

The processes of recruiting practices and patients and concealed randomisation, ensured equivalent groups of both GPs and patients at baseline and there was a good follow-up rate of patients (84% for the primary outcome). However, it is not possible to blind GPs to treatment group in such studies, which may have biased the activity of GPs as well as patient responses to questionnaires. Originally, the trial included a follow-up immediately before the intervention to detect any observer bias (the Hawthorne effect); we had to amend this design, however, on account of feasibility and work overload for the documenting GPs.

The insufficient recruitment of participating GPs and patients with left ventricular systolic dysfunction (LVSD) meant that we had to turn the trial, and its research question concerning the primary endpoint (QoL), into an exploratory trial.

External validity

We were not able to perform a non-responder analysis, but we have some indication that our sample of GPs was highly selective (Table (Table1):1): While the participating GPs were representative in terms of mean age and the number of GPs in one practice, they also had larger than average list sizes: the average list size in Germany is approximately 800 patients, and patients are free to change their doctor at any time. Almost all participating GPs adopted the German disease management programmes (average German participation rate approximately 53% in 2006) and also actively took part in peer review groups ("quality circles") (average German participation rate approximately 30% in 2006). Furthermore, a selection bias is supported by a high guideline-adherence at baseline regarding evidence-based pharmacotherapy (Table (Table22).

Meaning of the study – lessons learned

Due to the low power of the study it is only possible to speculate on the possible reasons for the lack of effectiveness of the intervention on QoL. We identified the following possible reasons:

1. Testing against an unblinded and active control group hampers proving efficacy but this cannot be changed due to the nature of complex interventions and ethical considerations.

2. Given the indirect design of our study, with the intervention at the level of the GP and outcome measurement at the level of the patient, our follow-up may have been set too early: While all GPs received the TTT-course, our evaluation in month 6 showed that GPs had implemented approximately 30–40% of the self-determined implementation targets (Figure (Figure11).

3. The intervention may have a (transient or permanent) negative effect because of up-titration[28,53] of β-blockers by GPs might have led to significant (transient) worsening of symptoms such as dyspnoea. Furthermore, increased activity of GPs in other contexts (including more scheduled attendances or serum level controls) could cause patients to perceive that they are more ill than was previously the case.

4. Our intervention was possibly carried out at a too late stage in the course of the disease as patients had a mean history of CHF of approximately six years. Improved patient outcomes have been shown in newly diagnosed patients or following a deterioration requiring hospital admission. Our patient sample shows a higher level of self-care behaviour than previously shown, suggesting that the patients in our study had already adapted their lifestyle to their diagnosis.

Lessons learned

In this trial, we encountered problems administering the study which we documented systematically along with possible solutions for how such similar problems could be dealt with in future studies (see Additional file 1, table S1).

Regarding the conceptual and planning phase of the trial, several factors hampered a tailoring the trial to general practice (see Additional file 1, table S1). The funding was too small for a clinical trial of this size and a longer intervention timeline, as recommend for complex interventions, could have had more impact[57].

In the operational phase we differentiated between the recruitment of investigators (GPs) and the recruitment of study participants. GPs were recruited from a single region of Germany (Northern Baden) and we encountered problems recruiting participants. Moreover, while we temporarily achieved the recruitment target of 60 GPs 26 GPs who had expressed an interest ultimately declined. Each GP was telephoned and asked for their reasons for not participating and the main reasons cited were the perceived workload and time commitment as well as the small financial incentive for participation. Extending the recruitment catchment area would have been a possible solution but was not feasible. In the UK, working with practice-based research networks (PBRNs) is considered a priority by national funding programmes, for example the Primary care research Network (PCRN) http://www.ukcrn.org.uk/index/networks/primarycare.html or the Medical research council general practice research framework (MRCGPRF) http://www.gprf.mrc.ac.uk/. For more ambitious research questions, a German PBRN would offer a valuable solution with regard to the recruitment of practices. However, it would be problematical to generate a representative sample of practices in such a network.

Case finding was another important barrier to the sufficient recruitment of study patients, as 20 GPs failed to find any eligible patients despite substantial screening efforts. The different reasons are listed in table S1 (see Additional file 1).

The inclusion criterion LVSD was another barrier as was a lack of understanding of LVSD by GPs (see Additional file 1, table S1). Extending the catchment area of recruitment and funding for a mobile study echocardiography could have been potential solutions but were not feasible for logistical reasons.

Regarding the analysis, our actual ICCs revealed numbers close to 0, hence an ICC as small as 0.015 was a sufficiently conservative assumption. We additionally found a low mortality rate (7.7%), and considering the lower total drop-out rate, a sample size of 390 participants would have been needed.

Conclusion

Due to considerable under-recruitment, the study was no longer satisfactorily powered to discover the pre-specified effect size. However, the observed data resulted in a confidence interval which definitely excludes a difference of 6.6 points in favour of the intervention group. Hence the study is conclusive in so far that the intended moderate treatment effect has been significantly refuted. Our study sample showed selection bias with high performing GPs with access to specialist care and patients that had adapted their self-care behaviour. Therefore, the study can be conceptualised as an exploratory trial[53]. The patient outcome results are valuable for future comparisons and could also be used for pooling in a meta-analysis.

The lessons learned are that measures implemented to increase internal validity (clinical trial design) were at the cost of study power and generalisability. No effective counter-measure was available at the time of the recruitment phase. For primary care-based clinical trials, our trial underlines the need for primary care research development and the establishment of research networks.

Abbreviations

A2RA: Angiotensin-2 receptor antagonist; ACE: Angiotensin converting enzyme; CHD: Coronary heart disease; CHF: Chronic (systolic) heart failure; CI: Confidence interval; CME: Continuous medical education; COPD: Chronic obstructive pulmonary disease; CPG: Clinical practice guideline; EHFScBS: European Heart Failure Self-care Behaviour Scale; EUROPEP: Patients in Europe evaluate general practice care; GFR: Glomerular filtration rate; GP: General practitioner; ICC: Intra-class correlation coefficient; ICD: Implantable cardioverter defibrillator; KCCQ: Kansas City Cardiomyopathy Questionnaire; LVEF: Left ventricular ejection fraction; LVSD: Left ventricular systolic dysfunction; NT-proBNP: N-terminal Brain Natriuretic Peptide; NYHA: New York Heart Association; OR: Odds ratio; PAD: Peripheral arterial disease; PC: Primary Care; PHQ-9: Depression module of the Patient Health Questionnaire; PTCA: Percutaneous Transluminal Coronary Angioplasty; QoL: (Health-related) Quality of life; RCT: Randomised controlled trial; SF-36: MOS 36-item short-form health survey; TTT: Train the trainer

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

DS, TMT, WH, GG, JS, SC and FPK designed the study. DS and FPK analysed the results. All authors interpreted the results. FPK wrote the manuscript, and all authors contributed to writing revisions and approved the final manuscript. FPK is the guarantor.

Supplementary Material

Additional file 1:

Table S1: Problems and possible solutions according to each phase of the clinical trial. The table lists the problems encountered during the phases of the trial and its potential solutions

Acknowledgements

We thank all participating GPs for their participation in the study, which entailed recruiting patients, collecting data and participating in the educational events: R. Barabasch, J. Barlet, M. Boehme, W. Boehme, R. Cairns, L. Dietz, R. Ding, J. Eberhardt, P. Engeser, P. Faust, U. Geib, S. Guerra, H. Hergemöller, M. Holzapfel, B. Hoops, F. Hoß, C. Karle, H.-D. Klimm, T. Koch, F. Kraner, U. Kümmritz, M. Mayer, K. Niethammer, D. Nolinski, S. Panahi-Bota, T. Pennekamp, G. Riedel, U. Ritter, U. Saueressig, P. Schlüter, G. Scheumann, R. Schober, R. Schwenke, W. Stroeck, J. Trott, S. Vierling, D. Zeisse-Suess

We thank Annika Baldauf, study nurse, for her hard work, and Stefanie Joos for helpful comments regarding the "lessons learned" aspect, both Department of General Practice and Health Services Research; Andrew Remppis for lecturing the cardiological content of the new intervention and the content of the standard intervention, Department of Cardiology, Angiology and Pneumology, University Hospital Heidelberg; Christiane Muth for the provision of a Clinical Practice Guideline, research fellow, Institute for General Practice, Chronic Care and Health Services Research, University of Frankfurt; Jana Juenger*,**and Nicole Lossnitzer*, Alina Sperling** and her team for the support in planning the educational intervention and provision of standardised patients, Department of Psychosomatic and General Internal Medicine(*) and competence centre for exams(**), University Hospital Heidelberg; We thank Christiane Prettin, Angelika Beyer and Bianca Scholze for data management, all Coordination Centre Clinical Trials Leipzig.

Special thanks go to the study participants.

Funding

The study was supported by the Competence Network of Heart Failure, funded by the German Ministry of Education and Research (BMBF), grant no. 01GI0205.

Ethical approval

Institutional review boards of the medical faculty of the University of Heidelberg and Medical Association of the state of Baden-Württemberg.

References

  • McMurray JJ, Pfeffer MA. Heart failure. Lancet. 2005;365:1877–1889. doi: 10.1016/S0140-6736(05)66621-4. [PubMed] [Cross Ref]
  • McMurray JJ, Stewart S. Epidemiology, aetiology, and prognosis of heart failure. Heart. 2000;83:596–602. doi: 10.1136/heart.83.5.596. [PMC free article] [PubMed] [Cross Ref]
  • Swedberg K, Cleland J, Dargie H, Drexler H, Follath F, Komajda M, Tavazzi L, Smiseth OA, Gavazzi A, Haverich A, Hoes A, Jaarsma T, Korewicki J, Lévy S, Linde C, Lopez-Sendon JL, Nieminen MS, Piérard L, Remme WJ, Task Force for the Diagnosis and Treatment of Chronic Heart Failure of the European Society of Cardiology Guidelines for the diagnosis and treatment of chronic heart failure: executive summary (update 2005): The Task Force for the Diagnosis and Treatment of Chronic Heart Failure of the European Society of Cardiology. Eur Heart J. 2005;26:1115–1140. doi: 10.1093/eurheartj/ehi166. [PubMed] [Cross Ref]
  • Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353:487–497. doi: 10.1056/NEJMra050100. [PubMed] [Cross Ref]
  • Komajda M, Lapuerta P, Hermans N, Gonzalez-Juanatey JR, van Veldhuisen DJ, Erdmann E, Tavazzi L, Poole-Wilson P, Le Pen C. Adherence to guidelines is a predictor of outcome in chronic heart failure: the MAHLER survey. Eur Heart J. 2005;26:1653–1659. doi: 10.1093/eurheartj/ehi251. [PubMed] [Cross Ref]
  • Fuat A, Hungin AP, Murphy JJ. Barriers to accurate diagnosis and effective management of heart failure in primary care: qualitative study. BMJ. 2003;326:196–201. doi: 10.1136/bmj.326.7382.196. [PMC free article] [PubMed] [Cross Ref]
  • Rogers AE, Addington-Hall JM, Abery AJ, McCoy AS, Bulpitt C, Coats AJ, Gibbs JS. Knowledge and communication difficulties for patients with chronic heart failure: qualitative study. BMJ. 2000;321:605–607. doi: 10.1136/bmj.321.7261.605. [PMC free article] [PubMed] [Cross Ref]
  • Cleland JG, Cohen-Solal A, Aguilar JC, Dietz R, Eastaugh J, Follath F, Freemantle N, Gavazzi A, van Gilst WH, Hobbs FD, Korewicki J, Madeira HC, Preda I, Swedberg K, Widimsky J, IMPROVEMENT of Heart Failure Programme Committees and Investigators. Improvement programme in evaluation and management; Study Group on Diagnosis of the Working Group on Heart Failure of The European Society of Cardiology Management of heart failure in primary care (the IMPROVEMENT of Heart Failure Programme): an international survey. Lancet. 2002;360:1631–1639. doi: 10.1016/S0140-6736(02)11601-1. [PubMed] [Cross Ref]
  • Shekelle PG, Rich MW, Morton SC, Atkinson CS, Tu W, Maglione M, Rhodes S, Barrett M, Fonarow GC, Greenberg B, Heidenreich PA, Knabel T, Konstam MA, Steimle A, Warner Stevenson L. Efficacy of angiotensin-converting enzyme inhibitors and beta-blockers in the management of left ventricular systolic dysfunction according to race, gender, and diabetic status: a meta-analysis of major clinical trials. J Am Coll Cardiol. 2003;41:1529–1538. doi: 10.1016/S0735-1097(03)00262-6. [PubMed] [Cross Ref]
  • Swedberg K, Ekman I. Integrating heart failure guidelines into clinical practice. Eur Hear J Suppl. 2005;7:J21–J25. doi: 10.1093/eurheartj/sui059. [Cross Ref]
  • Hobbs FD, Kenkre JE, Roalfe AK, Davis RC, Hare R, Davies MK. Impact of heart failure and left ventricular systolic dysfunction on quality of life: a cross-sectional study comparing common chronic cardiac and medical disorders and a representative adult population. Eur Heart J. 2002;23:1867–1876. doi: 10.1053/euhj.2002.3255. [PubMed] [Cross Ref]
  • Juenger J, Schellberg D, Kraemer S, Haunstetter A, Zugck C, Herzog W, Haass M. Health related quality of life in patients with congestive heart failure: comparison with other chronic diseases and relation to functional variables. Heart. 2002;87:235–241. doi: 10.1136/heart.87.3.235. [PMC free article] [PubMed] [Cross Ref]
  • Konstam V, Salem D, Pouleur H, Kostis J, Gorkin L, Shumaker S, Mottard I, Woods P, Konstam MA, Yusuf S. Baseline quality of life as a predictor of mortality and hospitalization in 5,025 patients with congestive heart failure. SOLVD Investigations. Studies of Left Ventricular Dysfunction Investigators. Am J Cardiol. 1996;78:890–895. doi: 10.1016/S0002-9149(96)00463-8. [PubMed] [Cross Ref]
  • Steptoe A, Mohabir A, Mahon NG, McKenna WJ. Health related quality of life and psychological wellbeing in patients with dilated cardiomyopathy. Heart. 2000;83:645–650. doi: 10.1136/heart.83.6.645. [PMC free article] [PubMed] [Cross Ref]
  • Doughty RN, Wright SP, Pearl A, Walsh HJ, Muncaster S, Whalley GA, Gamble G, Sharpe N. Randomized, controlled trial of integrated heart failure management: The Auckland Heart Failure Management Study. Eur Heart J. 2002;23:139–146. doi: 10.1053/euhj.2001.2712. [PubMed] [Cross Ref]
  • Holland R, Brooksby I, Lenaghan E, Ashton K, Hay L, Smith R, Shepstone L, Lipp A, Daly C, Howe A, Hall R, Harvey I. Effectiveness of visits from community pharmacists for patients with heart failure: HeartMed randomised controlled trial. BMJ. 2007;334:1098. doi: 10.1136/bmj.39164.568183.AE. [PMC free article] [PubMed] [Cross Ref]
  • Weinberger M, Oddone EZ, Henderson WG. Does increased access to primary care reduce hospital readmissions? Veterans Affairs Cooperative Study Group on Primary Care and Hospital Readmission. N Engl J Med. 1996;334:1441–1447. doi: 10.1056/NEJM199605303342206. [PubMed] [Cross Ref]
  • Phillips CO, Wright SM, Kern DE, Singa RM, Shepperd S, Rubin HR. Comprehensive discharge planning with postdischarge support for older patients with congestive heart failure: a meta-analysis. JAMA. 2004;291:1358–1367. doi: 10.1001/jama.291.11.1358. [PubMed] [Cross Ref]
  • Clark RA, Inglis SC, McAlister FA, Cleland JG, Stewart S. Telemonitoring or structured telephone support programmes for patients with chronic heart failure: systematic review and meta-analysis. BMJ. 2007;334:942. doi: 10.1136/bmj.39156.536968.55. [PMC free article] [PubMed] [Cross Ref]
  • Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8:iii–72. [PubMed]
  • Thomson O'Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2001:CD003030. [PubMed]
  • Wensing M, van der WT, Grol R. Implementing guidelines and innovations in general practice: which interventions are effective? Br J Gen Pract. 1998;48:991–997. [PMC free article] [PubMed]
  • Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care. 2005;21:380–385. doi: 10.1017/S026646230505049X. [PubMed] [Cross Ref]
  • Sanci LA, Coffey CM, Veit FC, Carr-Gregg M, Patton GC, Day N, Bowes G. Evaluation of the effectiveness of an educational intervention for general practitioners in adolescent health care: randomised controlled trial. BMJ. 2000;320:224–230. doi: 10.1136/bmj.320.7229.224. [PMC free article] [PubMed] [Cross Ref]
  • Sanci L, Coffey C, Patton G, Bowes G. Sustainability of change with quality general practitioner education in adolescent health: a 5-year follow-up. Med Educ. 2005;39:557–560. doi: 10.1111/j.1365-2929.2005.02172.x. [PubMed] [Cross Ref]
  • van Leeuwen YD, Mol SS, Pollemans MC, Drop MJ, Grol R, Vleuten CP van der. Change in knowledge of general practitioners during their professional careers. Fam Pract. 1995;12:313–317. doi: 10.1093/fampra/12.3.313. [PubMed] [Cross Ref]
  • Perera R, Heneghan C, Yudkin P. Graphical method for depicting randomised trials of complex interventions. BMJ. 2007;334:127–129. doi: 10.1136/bmj.39045.396817.68. [PMC free article] [PubMed] [Cross Ref]
  • Peters-Klimm F, Müller-Tasch T, Schellberg D, Remppis A, Barth A, Holzapfel N, Jünger J, Herzog W, Szecsenyi J. Guideline adherence for pharmacotherapy of chronic systolic heart failure in general practice: a closer look on evidence-based therapy. Clin Res Cardiol. 2008;97:244–252. doi: 10.1007/s00392-007-0617-6. [PubMed] [Cross Ref]
  • Ware J, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30:473–483. doi: 10.1097/00005650-199206000-00002. [PubMed] [Cross Ref]
  • Bullinger M, Kirchberger I. SF-36 Fragebogen zum Gesundheitszustand, Handanweisung. In: Kirchberger I, editor. Göttingen, Hogrefe Verlag für Psychologie. 1998.
  • Green CP, Porter CB, Bresnahan DR, Spertus JA. Development and evaluation of the Kansas City Cardiomyopathy Questionnaire: a new health status measure for heart failure. J Am Coll Cardiol. 2000;35:1245–1255. doi: 10.1016/S0735-1097(00)00531-3. [PubMed] [Cross Ref]
  • Wyrwich KW, Bullinger M, Aaronson N, Hays RD, Patrick DL, Symonds T. Estimating clinically significant differences in quality of life outcomes. Qual Life Res. 2005;14:285–295. doi: 10.1007/s11136-004-0705-2. [PubMed] [Cross Ref]
  • Spertus J, Peterson E, Conard MW, Heidenreich PA, Krumholz HM, Jones P, McCullough PA, Pina I, Tooley J, Weintraub WS, Rumsfeld JS, Cardiovascular Outcomes Research Consortium Monitoring clinical changes in patients with heart failure: A comparison of methods. American Heart Journal. 2005;150:707–715. doi: 10.1016/j.ahj.2004.12.010. [PubMed] [Cross Ref]
  • Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [PMC free article] [PubMed] [Cross Ref]
  • Löwe B, Spitzer RL, Gräfe K, Kroenke K, Quenter A, Zipfel S, Buchholz C, Witte S, Herzog W. Comparative validity of three screening questionnaires for DSM-IV depressive disorders and physicians' diagnoses. J Affect Disord. 2004;78:131–140. doi: 10.1016/S0165-0327(02)00237-9. [PubMed] [Cross Ref]
  • Jaarsma T, Strömberg A, Martensson J, Dracup K. Development and testing of the European Heart Failure Self-Care Behaviour Scale. The European Journal of Heart Failure. 2003;5:363–370. doi: 10.1016/S1388-9842(02)00253-2. [PubMed] [Cross Ref]
  • Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, Hjortdahl P, Olesen F, Reis S, Ribacke M, Szecsenyi J, European Task Force on Patient Evaluations of General Practice Care (EUROPEP) Patients in Europe evaluate general practice care: an international comparison. Br J Gen Pract. 2000;50:882–887. [PMC free article] [PubMed]
  • Richards AM, Doughty R, Nicholls MG, MacMahon S, Sharpe N, Murphy J, Espiner EA, Frampton C, Yandle TG, Australia-New Zealand Heart Failure Group Plasma N-terminal pro-brain natriuretic peptide and adrenomedullin: prognostic utility and prediction of benefit from carvedilol in chronic ischemic left ventricular dysfunction. Australia-New Zealand Heart Failure Group. J Am Coll Cardiol. 2001;37:1781–1787. doi: 10.1016/S0735-1097(01)01269-4. [PubMed] [Cross Ref]
  • Hess G, Moecks J, Zdunek D. N-Terminal-proBNP (NT-proBNP) as an indicator of cardiac dysfunction. A study in patients presenting with suspected cardiac disorders. Z Kardiol. 2005;94:247–254. doi: 10.1007/s00392-005-0205-6. [PubMed] [Cross Ref]
  • Rickham PP. Human experimentation. Code of ethics of the world medical association. Declaration of Helsinki. Br Med J. 1964;2:177. doi: 10.1136/bmj.2.5402.177. [PMC free article] [PubMed] [Cross Ref]
  • Osterziel KJ, Hewer A. [Expert research network in cardiology – the Heart Failure Competence Network. Within the scope of the health research program "Competence in Medicine" the Heart Failure Competence Network established in 2003] Z Kardiol. 2005;94:542–544. doi: 10.1007/s00392-005-0265-7. [PubMed] [Cross Ref]
  • Campbell MK, Mollison J, Steen N, Grimshaw JM, Eccles M. Analysis of cluster randomized trials in primary care: a practical approach. Fam Pract. 2000;17:192–196. doi: 10.1093/fampra/17.2.192. [PubMed] [Cross Ref]
  • Campbell MK, Fayers PM, Grimshaw JM. Determinants of the intracluster correlation coefficient in cluster randomized trials: the case of implementation research. Clin Trials. 2005;2:99–107. doi: 10.1191/1740774505cn071oa. [PubMed] [Cross Ref]
  • Dietz R, Osterziel KJ, Cleland JG. [Management of heart failure: ambition and reality in the early 21st Century] Z Arztl Fortbild Qualitatssich. 2003;97:98–103. [PubMed]
  • Winkler J. Scaling social status in the german national health surveys. RKI-Schriften. 1998. pp. 69–74.
  • Bruggink-André de la Porte PW, Lok DJ, van Wijngaarden J, Cornel JH, Pruijsers-Lamers D, van Veldhuisen DJ, Hoes AW. Heart failure programmes in countries with a primary care-based health care system. Are additional trials necessary? Design of the DEAL-HF study. European Journal of Heart Failure. 2005;7:910–920. doi: 10.1016/j.ejheart.2004.11.004. [PubMed] [Cross Ref]
  • de la Porte PW, Lok DJ, van Veldhuisen DJ, van Wijngaarden J, Cornel JH, Zuithoff NP, Badings E, Hoes AW. Added value of a physician-and-nurse-directed heart failure clinic: results from the Deventer-Alkmaar heart failure study. Heart. 2007;93:819–825. doi: 10.1136/hrt.2006.095810. [PMC free article] [PubMed] [Cross Ref]
  • Kasje WN, Denig P, Stewart RE, de Graeff PA, Haaijer-Ruskamp FM. An educational programme for peer review groups to improve treatment of chronic heart failure and diabetes mellitus type 2 in general practice. Journal of Evaluation in Clinical Practice. 2006;12:613–621. doi: 10.1111/j.1365-2753.2005.00625.x. [PubMed] [Cross Ref]
  • Bongers FJ, Schellevis FG, Bakx C, Bosch WJ van den, Zee J van der. Treatment of heart failure in Dutch general practice. BMC Fam Pract. 2006;7:40. doi: 10.1186/1471-2296-7-40. [PMC free article] [PubMed] [Cross Ref]
  • Dobre D, van Jaarsveld CH, Ranchor AV, Arnold R, de Jongste MJ, Haaijer Ruskamp FM. Evidence-based treatment and quality of life in heart failure. J Eval Clin Pract. 2006;12:334–340. doi: 10.1111/j.1365-2753.2006.00564.x. [PubMed] [Cross Ref]
  • European Society of Cardiology; Heart Failure Association of the ESC (HFA); European Society of Intensive Care Medicine (ESICM) Dickstein K, Cohen-Solal A, Filippatos G, McMurray JJ, Ponikowski P, Poole-Wilson PA, Strömberg A, van Veldhuisen DJ, Atar D, Hoes AW, Keren A, Mebazaa A, Nieminen M, Priori SG, Swedberg K, Vahanian A, Camm J, De Caterina R, Dean V, Dickstein K, Filippatos G, Funck-Brentano C, Hellemans I, Kristensen SD, McGregor K, Sechtem U, Silber S, Tendera M, Widimsky P, Zamorano JL, Tendera M, Auricchio A, Bax J, Böhm M, Corrà U, della Bella P, Elliott PM, Follath F, Gheorghiade M, Hasin Y, Hernborg A, Jaarsma T, Komajda M, Kornowski R, Piepoli M, Prendergast B, Tavazzi L, Vachiery JL, Verheugt FW, Zamorano JL, Zannad F. ESC guidelines for the diagnosis and treatment of acute and chronic heart failure 2008: the Task Force for the diagnosis and treatment of acute and chronic heart failure 2008 of the European Society of Cardiology. Developed in collaboration with the Heart Failure Association of the ESC (HFA) and endorsed by the European Society of Intensive Care Medicine (ESICM) Eur J Heart Fail. 2008;10:933–989. doi: 10.1016/j.ejheart.2008.08.005. [PubMed] [Cross Ref]
  • Tzanetos K, Leong D, Wu RC. Office management of patients with diastolic heart failure. CMAJ. 2009;180:520–527. [PMC free article] [PubMed]
  • Peters-Klimm F, Muller-Tasch T, Remppis A, Szecsenyi J, Schellberg D. Improved guideline adherence to pharmacotherapy of chronic systolic heart failure in general practice – results from a cluster-randomized controlled trial of implementation of a clinical practice guideline. J Eval Clin Pract. 2008;14:823–829. [PubMed]
  • Thilly N, Briancon S, Juilliere Y, Dufay E, Zannad F. Improving ACE inhibitor use in patients hospitalized with systolic heart failure: a cluster randomized controlled trial of clinical practice guideline development and use. J Eval Clin Pract. 2003;9:373–382. doi: 10.1046/j.1365-2753.2003.00441.x. [PubMed] [Cross Ref]
  • Ansari M, Shlipak MG, Heidenreich PA, Van Ostaeyen D, Pohl EC, Browner WS, Massie BM. Improving guideline adherence: a randomized trial evaluating strategies to increase beta-blocker use in heart failure. Circulation. 2003;107:2799–2804. doi: 10.1161/01.CIR.0000070952.08969.5B. [PubMed] [Cross Ref]
  • LaPointe NM, DeLong ER, Chen A, Hammill BG, Muhlbaier LH, Califf RM, Kramer JM. Multifaceted intervention to promote beta-blocker use in heart failure. Am Heart J. 2006;151:992–998. doi: 10.1016/j.ahj.2005.06.038. [PubMed] [Cross Ref]
  • Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321:694–696. doi: 10.1136/bmj.321.7262.694. [PMC free article] [PubMed] [Cross Ref]

Articles from Trials are provided here courtesy of BioMed Central

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • Compound
    Compound
    PubChem Compound links
  • MedGen
    MedGen
    Related information in MedGen
  • PubMed
    PubMed
    PubMed citations for these articles
  • Substance
    Substance
    PubChem Substance links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...