• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of implemsciBioMed CentralBiomed Central Web Sitesearchsubmit a manuscriptregisterthis articleImplementation Science : ISJournal Front Page
Implement Sci. 2009; 4: 61.
Published online Sep 23, 2009. doi:  10.1186/1748-5908-4-61
PMCID: PMC2936828

A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies

Abstract

Context

Significant resources and time are invested in the production of research knowledge. The primary objective of this randomized controlled trial was to evaluate the effectiveness of three knowledge translation and exchange strategies in the incorporation of research evidence into public health policies and programs.

Methods

This trial was conducted with a national sample of public health departments in Canada from 2004 to 2006. The three interventions, implemented over one year in 2005, included access to an online registry of research evidence; tailored messaging; and a knowledge broker. The primary outcome assessed the extent to which research evidence was used in a recent program decision, and the secondary outcome measured the change in the sum of evidence-informed healthy body weight promotion policies or programs being delivered at health departments. Mixed-effects models were used to test the hypotheses.

Findings

One hundred and eight of 141 (77%) health departments participated in this study. No significant effect of the intervention was observed for primary outcome (p < 0.45). However, for public health policies and programs (HPPs), a significant effect of the intervention was observed only for tailored, targeted messages (p < 0.01). The treatment effect was moderated by organizational research culture (e.g., value placed on research evidence in decision making).

Conclusion

The results of this study suggest that under certain conditions tailored, targeted messages are more effective than knowledge brokering and access to an online registry of research evidence. Greater emphasis on the identification of organizational factors is needed in order to implement strategies that best meet the needs of individual organizations.

Trial Registration

The trial registration number and title are as follows: ISRCTN35240937 -- Is a knowledge broker more effective than other strategies in promoting evidence-based physical activity and healthy body weight programming?

Introduction

Currently, there is substantial political and societal pressure to demonstrate the integration of the best available research evidence with local contextual factors, so as to provide the most effective health services in optimizing health outcomes [1]. The purpose of this randomized controlled trial was to evaluate the impact of three knowledge translation and exchange (KTE) strategies in promoting the incorporation of research evidence by public health decision makers into public health policies and programs related to healthy body weight promotion in children.

Background

Knowledge translation and exchange: what we know

The integration of research evidence into public health policy and program decision making is commonly referred to as evidence-informed decision making [2], and strategies to promote it as KTE. However, it is well known that the decision-making process is complex, and that multiple forms of knowledge impact both the process as well as the decision. In this study, we were interested in exploring the use of research evidence in decisions concerning the provision of public health services for promoting health body weight in children. In Canada, program managers in public health departments typically make recommendations to senior management on the specific interventions and strategies that could be provided to address particular population issues (e.g., healthy weights) [3]. Managers typically explore different options and make decisions about interventions that fit within the social and political climate of their respective regions. We explored whether research evidence influenced these decisions made by program managers concerning whether and which interventions they recommended their health department make available in order to promote healthy body weight in children.

Factors identified previously in the KTE literature known to contribute to clinical and program planning decisions include those related to individual decision makers, the system, patients, and research evidence [4]. At the individual decision-maker level, important factors include past experiences (e.g., clinical or managerial experiences with patients/clients, policy makers, events, or circumstances), beliefs, values, and skills; the environment/system level includes resources (both human and financial), legislation, protocols, and societal norms; patient preferences; and research evidence (e.g., multiple ways of knowing) [5-8]. The intent of evidence-informed decision making is not to suggest that health policy and program decisions be determined solely by research evidence, but rather research evidence be considered within the context of the setting or circumstance, societal expectations, health care resources, and professional expertise.

Barriers consistently identified to evidence-informed decision making in the KTE literature include: lack of time; limited access to research evidence (e.g., many health departments can identify relevant research evidence in the published domain, but experience significant challenges in obtaining the full text in a cost-efficient and timely way) [9,10]; limited capacity to appraise and translate research evidence; and resistance to change (e.g., lack of motivation to stop doing what has traditionally been done) [11-17]. System-level changes needed to support evidence-informed decision making include: researchers gaining a better appreciation of the context in which decision makers function and building more collaborative relationships with decision makers [3,18,19].

Three KTE strategies are currently being widely used to promote evidence-informed decision making. These include: freely accessible web-based resources that summarize research evidence; tailored and targeted messages that connect relevant research evidence to specific decision makers [20]; and knowledge brokers (KBs), who work one-on-one with decision makers to facilitate evidence-informed decision making [21]. The internet is established as an essential component of KTE [22], and significant resources have been and continue to be allocated to these strategies. Several web-based resources have been developed with the intent of compiling the best available research evidence by topic area or health care discipline (e.g., Medline Plus, More EBN, health-evidence.ca). Some have gone one step further to synthesize the results of the evidence to answer specific practice-based questions [23]. However, there is a scarcity of literature evaluating the effectiveness of web-based resources in achieving evidence-informed decision making.

Tailored and targeted messages have gained momentum as a popular KTE strategy [24-27]. 'Tailored' implies that the message is focused on the specific scope of decision-making authority of the intended user, while 'targeted' indicates that the content of the message is relevant and directly applicable to the decision currently faced by the intended audience. Evidence indicates that computer-tailored messages are associated with increased uptake compared to standardized messages [28], and that electronic targeted messages to subgroups with common interests is effective in promoting evidence-informed decision making [29]. While tailored, targeted messages have been shown to improve uptake of systematic reviews [30], questions remain as to what content is most wanted and required for different audiences, what the most effective communication channels are [28], and which organizations will benefit most from such a KTE strategy.

KBs have been implemented widely in private industry [31-33], and more recently in healthcare settings [21,34,35]. In fact, a great many organizations in Canada have quickly moved to adopt KB roles with little more than anecdotal evidence supporting their effectiveness. A KB acts as a catalyst for systems change, establishing and nurturing connections between researchers and end users [36], and facilitating learning and exchange of knowledge [37]. The anecdotal evidence suggests that KBs improve the quality and usefulness of evidence that is employed in decision making [38], while promoting a decision-making culture that values the use of evidence [39,40]. Furthermore, the heightened degree of interaction with decision makers through knowledge brokering is assumed by many to be the optimal KTE strategy in comparison to less interactive strategies; however, this has yet to be proven [34]. Given the lack of evaluation of each of these KTE strategies individually or in comparison to one another, the timing was right for conducting this study.

Healthy body weight

The problems of obesity, overweight, and physical inactivity have been identified in children [41]. According to the latest Canadian Fitness and Lifestyle Research Institute Physical Activity Monitor [42], 90% of Canadian children and youth aged five to 17 are not active enough to promote good health. Many of the risks associated with obesity in children cluster in cardiovascular disease risk factors known as the insulin resistance syndrome, and have been identified in children as young as five years of age [43]. In addition, overweight in childhood increases the risk of death from ischemic heart disease in adulthood two-fold over 57 years, and the incidence of Type 2 diabetes is increasing and is attributable to obesity [44]. Most alarming, however, is the knowledge that physical activity patterns and chronic disease conditions track from childhood into adulthood [45-55]. Canadian research estimates that physical inactivity and obesity resulted in expenditures of $5.3 and $4.3 billion in direct and indirect costs, representing 2.6% and 2.2%, respectively, of total health care costs in Canada [56].

The literature demonstrates that regular aerobic activity increases exercise capacity and plays a role in both the primary and secondary prevention of cardiovascular disease [57-60]. Furthermore, regular physical activity has been shown to enhance health, reduce the risk for all-cause mortality, prolong life, and improve quality of life [61-67]. The evidence suggests that the best primary strategy for improving the long-term health of children and adolescents may be in creating a lifestyle pattern of regular physical activity and healthy eating that will carry over to the adult years [68].

Promoting healthy body weight in children: the role of public health

Public health departments in Canada are responsible for promoting the health of the population, preventing disease, and providing medical care to treat communicable diseases. They provide services that focus on promoting the health of individuals as well as health promotion within schools and worksites, nutritional counseling, physical activity promotion, development of community strengths to promote/improve health, and the promotion of healthy environments [69]. The public health sector in Canada is structured generally with a medical officer of Health at the head of the organization and who has senior decision-making authority (subsequent to the local/regional board of health) for the services provided by that organization to a designated local community or region. The public health workforce responsible for the promotion of physical activity and chronic disease prevention is comprised primarily of public health nurses, nutritionists, physical activity experts, and health promotion officers. At the time this study was conducted (July 2004 to February 2006), all provinces and territories in Canada held mandates requiring public health departments to develop and implement strategies to promote healthy body weight in children. Despite these mandates, there was limited capacity (time, skill, access) among public health decision makers, and limited resources to utilize the best available research evidence with which to plan and implement effective healthy body weight promotion programs and services.

Methods

Design

This randomized controlled trial funded in 2003 by the Canadian Institutes of Health Research, was the first in Canada to evaluate the effectiveness of a KB in comparison to other KTE interventions on promoting evidence-informed decision making in public health departments. Following ethics approval (McMaster University Faculty of Health Sciences Research Ethics Board) and recruitment, participating health departments were stratified according to size of population served and randomly allocated to groups using computer-generated random numbers. Given the background work conducted by the research team, as well as findings from the literature, stratifying public health departments by size of population served prior to randomization was deemed necessary. The three strata were defined as: health departments serving a population size below 50,000; a population size between 50,000 and 250,000; and a population size above 250,000. The Statistics Canada Peer Groups were used to allocate public health departments to each strata. The public health departments were randomly allocated to intervention groups in equal numbers within strata by computer generated pseudorandom draws using standard algorithms. Three health departments that remained unselected after equal allocation within strata were assigned to treatment groups randomly across strata. The health department was the unit of analysis. The study process is shown in Figure Figure11.

Figure 1
Flow chart of data collection from baseline to post intervention. Flow chart showing the process of data collection from baseline to post intervention.

The framework proposed by Dobbins et al. [70] is one of many frameworks [71-76] that have been developed to illustrate the process of knowledge translation and evidence-informed decision making. Dobbins' framework was used to guide the development of the KTE strategies (tailored, targeted messages and KB) and identify relevant outcomes for this study. The framework demonstrates the complex inter-relationships that exist between the five stages of innovation identified by Rogers, [77] (knowledge, persuasion, decision, implementation, and confirmation), and four types of characteristics, organizational, environmental, individual, and the innovation [78], as the knowledge translation process occurs. The framework also identifies the variety of possible outcomes that can be observed including: knowledge and attitudes; decision making; implementation (e.g., putting research knowledge into public health policy and practice, guideline development); and outcomes (e.g., changes in public health policy and practice). This study focused on the measurement of outcomes, specifically changes in public health policies and programs at the local public health department level.

The hypotheses were: public health departments exposed to tailored, targeted messages and the KB would report greater evidence-informed decision making than those exposed to a repository of quality assessed systematic reviews evaluating public health interventions (health-evidence.ca); knowledge brokering would result in greater evidence-informed decision making than tailored, targeted messages; and characteristics of the organization would have significant impacts on the effect of the KTE interventions on evidence-informed decision making. More specifically, we hypothesized that certain organizational characteristics (e.g., research culture, or the value organizations placed on the use of research evidence in decision making) would have an impact on the effectiveness of the KTE interventions to promote evidence-informed decision making. A previous study with Canadian public health decision makers illustrated that public health departments that valued the use of research evidence in decision making were significantly more likely to use research evidence for program planning decisions than health departments that put less value on research evidence [79,80]. Therefore, we hypothesized that organizations that placed lower value on using research evidence in decision making would experience less improvement in evidence-informed decision making than those who valued research evidence more highly.

Sample and recruitment

The sample was comprised of regional and local public health departments in Canada. Eligible participants from participating health departments were directly responsible for making program decisions related to healthy body weight promotion in children. This included program managers and/or coordinators in Ontario, and program directors in the rest of Canada. All health departments in Canada were invited to participate. Health departments in Canada were identified through provincial databases. Participants were recruited into the study in a two-stage process. First, consent from the most senior person in the public health department (e.g., medical officer of health or chief executive officer) was sought. If written consent was obtained, the name of the person most directly responsible for making decisions related to healthy body weight promotion among children was identified and contacted. A letter of invitation was then sent directly to the potential participant followed by a telephone call to ascertain consent to participate in the study and answer any questions.

Intervention

The three interventions were implemented simultaneously during 2005. The content used in the KTE interventions (healthy body weight promotion in children) was summarized from seven rigorous systematic reviews and will be described in greater detail in the outcomes section. The least interactive KTE intervention was access to health-evidence.ca (HE group). Health-evidence.ca is a repository of all systematic reviews published since 1985 evaluating any public health intervention. All participants in the study received electronic communication about the availability of this site. Upon searching this site for reviews evaluating strategies to promote healthy body weight in children (to mimic the standard way in which electronic sources are utilized in practice), those in the HE group would have become aware of the title, citation, and assessment of the methodological quality of seven systematic reviews evaluating the effectiveness of interventions to promote healthy body weight in children. Participants in the HE group also had access to the published abstracts, and the full text articles (copyright purchased for this study) through Health-evidence.ca. Finally, a short summary for each of the systematic reviews, written by the research team, identified the key findings and recommendations for public health policy and practice that were directly applicable to the types of decisions for which the participants were responsible. Such summaries are written for all of the well-done systematic reviews appearing in health-evidence.ca and are available to all users, while targeted primarily at the level of program managers.

The moderately interactive KTE intervention included tailored, targeted messages plus access to health-evidence.ca (TM group). The TM intervention included sending participants a series of emails that included the title of the seven systematic reviews followed by a link to the full reference, including abstracts, on health-evidence.ca. The online reference offered a link to the short summaries, and finally, the full text of each review. Over seven successive weeks, on the same day each week and the same time of day, participants in the TM group were sent an email indicating that a systematic review related to healthy body weight promotion in children was available in full text at the link provided. At the URL linked within the email message, participants also received access to the PDF version of the systematic review, the published abstract of the review, as well as the short summary written. Finally, the text of the message was worded to say, 'this message is number XX in a series of seven emails you will receive on healthy body weight promotion in children as part of the KTE strategy you are being exposed to in this randomized controlled trial'.

The most interactive KTE intervention included both the HE and TM components and a KB who worked one on one with decision makers in the public health departments. One full-time KB provided knowledge brokering services to all English speaking participants allocated to the KB group (n = 30). A second Francophone KB (0.2 full time equivalent) provided KB services to French-speaking participants also allocated to the KB group (n = 6). The KBs were Master's prepared, had extensive knowledge and expertise in public health decision making, as well as an understanding of the research process. Specific tasks conducted by the KB included: ensuring relevant research evidence related to healthy body weight promotion was transferred to the public health decision makers in ways that were most useful to them, assisting them to develop the skill and capacity for evidence-informed decision making, and assisting them in translating evidence into local practice.

Approximately twenty percent of KB time was spent facilitating knowledge and skill development either through face-to-face interaction such as workshops or online strategies such as webinars, interactive web-enabled meetings, or conferences. Eighty percent of the brokers' time was spent preparing for and directly interacting with participants. The proportion of time the KB spent preparing for interaction with participants was 40% to 50% early in the project and declined to 30% as both public health decision makers and the KB became more skilled in their respective roles. KB activities were classified into the following categories: initial and ongoing needs assessments; scanning the horizon; knowledge management; KTE; network development, maintenance, and facilitation; facilitation of individual capacity development in evidence-informed decision making; and facilitation of and support for organizational change. These activities were carried out through regular electronic and telephone communication, and one site visit to each health department of one to two days in length. As well, each health department was invited to attend a one-day workshop held regionally (four cities) across Canada. A more complete description of the KB intervention is published elsewhere [81]. However, the main activities of the KB intervention are described.

At the start of the intervention, the KB conducted assessments at the individual, organizational, and environmental levels, in order to identify strengths, knowledge, and capacity for evidence-informed decision making. The KB then worked with participants to generate a plan for developing individual and organizational capacity for evidence-informed decision making. In order to facilitate participant access to the best available evidence, the KB consistently scanned the horizon for new evidence and resources of interest to participants. This activity involved maintaining subscriptions to related list serves, electronic distribution lists, and e-table of contents alerts from relevant journals. The majority of the KB's time was spent doing KTE, which was facilitated by developing and maintaining a trusting relationship with participants. The KB-initiated communication with participants occurred at a minimum of once per month, and more frequently as requested. The KB also offered a site visit to each public health department. The purpose of the site visit was to facilitate the building of a trusting relationship between the health department and the KB, as well as to enable the KB to learn more about the local context. This facilitated the tailoring of KB services to the specific needs of each local environment. Furthermore, the activities conducted by the KB during each site visit varied according to specific needs and goals identified by each health department. In many cases, the KB participated in team program planning sessions and assisted in the interpretation of evidence from the tailored, targeted messages and its incorporation into local program plans. The KB also conducted training sessions in many health departments to assist participants and their colleagues in developing their capacity to be critical consumers of different knowledge sources. Opportunities to facilitate knowledge, skills development, and capacity for evidence-informed decision making occurred during all interactions with the KB at the individual (email, telephone, site visit) and group level (site visit, regional workshop, webinars). Finally, during the regional workshops, the KB presented the results of the systematic reviews to participants, facilitated discussion concerning the results as well as implications for local program and public health policy development. KBs also encouraged participants to engage in individual and joint problem-solving related to evidence-informed decision making, and enabled face-to-face contact with the KB to promote credibility and trust.

Data collection

The data were collected using a telephone-administered survey (knowledge transfer and exchange data collection tool) at baseline (August 2004) and immediately post-intervention (February 2006). Items in the questionnaire were chosen from questionnaires previously tested and used in diffusion of innovation and research utilization studies [11,77,82-88]. We tested the modified questionnaire for reliability and validity among public health decision makers, and have reported a Cronbach alpha of 0.65 for reliability elsewhere [11,80,89]. The questionnaire is available from the corresponding author upon request. The questionnaire was administered twice to participants at baseline, one month apart.

Independent variables

Data were collected on organizational, environmental, and individual characteristics shown previously to be related to evidence-informed decision making [79], and measured using seven-point Likert scales. Organizational characteristics included: organizational culture (e.g., research culture, or the value placed on using research evidence in decision making, and the expectation to demonstrate use of research evidence in decision making), staff training in research methods and critical appraisal, and decision-making style. The environmental characteristic included collaboration with other community organizations. Individual characteristics included age, education, position, perceived influence over the decision-making process, and perception of the barriers to using research evidence in public health decision making. All variables were measured in the same direction.

Dependent variables

Two dependent variables were evaluated: global evidence-informed decision making and public health policies and programs. For global evidence-informed decision making, participants were asked to report on the extent to which research evidence was considered in a recent program-planning decision (previous 12 months) related to healthy body weight promotion. This is a common way of measuring research use in the KTE field. Participants were asked to quantify their response ranging from one (not at all) to seven (completely). However, given many have suggested that this is not an optimal way of measuring evidence-informed decision making, we developed a second outcome variable, labeled 'public health policies and programs'. This measure was derived as the sum of actual strategies, policies, and/or interventions for healthy body weight promotion in children being implemented by the health department.

Eleven policies, programs, and/or interventions with good evidence of effectiveness were identified from seven systematic reviews assessed as being of high methodological quality [90-96] (Table (Table1).1). Each systematic review was assessed for methodological quality by two independent reviewers using a previously developed and tested quality assessment tool [97,98]. Reviewers met to discuss ratings, and consensus on all ratings was achieved. Only those systematic reviews attaining seven points or higher out of a total of ten possible points were deemed of sufficient methodological quality to inform public health policy and practice. Participants were asked whether the public health policies and programs were being implemented by their health department (yes/no). The total number was summed and compared across groups from baseline to post intervention.

Table 1
Healthy body weight policies and programs (HPPs)

Analysis

Mixed-effects models were used to conduct tests of the two hypotheses related to the treatment effects, which is a standard approach to the analysis of designs with repeated measurements [99]. Repeated measurements over time were modeled as nested within participants, and time of observation was coded to estimate the differences between groups in scores at the average of the two baseline observations, and then the change from baseline to the post-intervention follow-up. The interaction of this change with the randomized treatment assignment is the appropriate estimate of the treatment effect, such that we tested whether change following the intervention differs among the intervention groups. These mixed-effects models provide for appropriate adjustment for the repeated measurements with participants when testing treatment effects, and they also allow for flexible handling of missing data. The moderating roles of selected predictor characteristics (hypothesis three) were also tested by evaluating their three-way interactions with time and treatment.

Results

All 141 public health departments in Canada were invited to participate in this study, of which 108 (77%) agreed to do so. Stated reasons for not participating included undergoing restructuring, involved in too many research studies, or the topic was not a priority. Thirty-six public health departments were assigned to each of the three intervention groups. No statistically significant differences were observed between groups at baseline on important independent and dependent variables.

Follow-up data

Participation by province and territory ranged from 29% to 100% with the sample consisting primarily of health departments serving both urban and rural populations (46%). Table Table22 presents a description of the study sample. Follow-up data were collected from 88 of 108 (81.5%) participating public health departments. Reasons for not participating in the follow-up survey were lack of time and not having anyone working in healthy body weight promotion. Among the HE, TM, and KB groups, similar drop-out rates were observed of seven, six, and seven health departments, respectively.

Table 2
Baseline characteristics of public health departments and decision-makers

Intervention integrity

It is unknown to what extent the HE group accessed http://www.health-evidence.ca. To our knowledge, all those exposed to the TM intervention received 100% of the intervention. For those exposed to the KB intervention, approximately 70% received the full intervention (e.g., frequency, intensity) with approximately 15%, respectively, not engaging at all, or to a limited extent. Organizations were analyzed according to their assigned group.

Outcomes

The estimates from the mixed-effects models are presented in Table Table3.3. The table gives estimated pair-wise differences for the TM and KB groups, relative to control (HE group), as well as overall tests of group differences at baseline and for the change from baseline to follow-up. In addition, the standard deviation in outcome between and within health departments over time is provided. This gives an indication of the degree of variation in the outcomes that remains unexplained after accounting for the intervention. For both outcomes, most of the remaining variation appears as unexplained changes over time within health departments. Table 3 shows that baseline scores do not differ significantly between groups for either outcome, although the TM group possibly had fewer public health policies and programs at baseline compared to the HE group (p < 0.06).

Table 3
EIDM outcomes baseline to follow-up

As shown in Table 3, the intervention had no significant effect on global evidence-informed decision making (p < 0.45), although all groups improved to some extent. For public health policies and programs, as is shown in Figure Figure2,2, a significant effect of the intervention was observed (p < 0.01). For this outcome, the TM group improved significantly from baseline to follow-up in comparison to the HE and KB groups that showed no significant change. With respect to hypothesis three, some organizational characteristics were shown to moderate the intervention effect, although not always in the hypothesized direction. When organizational research culture was added to the mixed-effects models as a predictor, the group/time/culture interaction was significant (p < 0.03). This three-way interaction is illustrated in Figure Figure3,3, with the predictions for each group shown at relatively low (four of seven) and high (six of seven) values of the extent to which health departments reported they valued research evidence.

Figure 2
Framework for Research Dissemination and Utilization. This figure depicts the primary author's framework for research dissemination and utilization.
Figure 3
A. Comparison of Health Policies and Programs (HPP) Post-Intervention. This figure shows the association between the interventions (control, tailored messaging, and knowledge brokering) and the number of evidence-supported health policies and programs ...

As Figure Figure33 illustrates for health departments with low organizational research culture, the intervention effect was as we hypothesized -- the control group was unchanged, the TM group improved somewhat, and the KB group improved most. However, when organizational research culture was high (six on a seven-point scale), the HE group remained unchanged, the KB group decreased (fewer public health policies and programs), and the TM group increased significantly. Similar trends were observed for organizational characteristics, such as expectation to use research evidence and frequency of hearing the term research evidence. However, no significant differences in results were observed when multiple organizational characteristics were included in the models, therefore only the results of organizational research culture have been presented.

Discussion

Generally the results of this randomized controlled trial show the need to match the organizational research culture to intervention type, and in particular support the hypothesis that tailored, targeted messages plus website informational materials can be an effective strategy for facilitating evidence-informed decision making. The results indicate that the 'right' evidence, 'pushed' out to the right decision maker working in an organization supportive of evidence-informed decision making, leads to outcomes in the hypothesized direction. In addition, simply having access to an online registry of research evidence appears to have no impact on evidence-informed decision making. Finally, knowledge brokering does not appear to be effective in promoting evidence-informed decision making overall, although there appears to be a trend toward a positive effect when organizational research culture is perceived as low.

These findings are supported by published studies showing that simple KTE interventions can be as effective as complex, multi-faceted ones [100-102]. A recent meta-analysis evaluating the effectiveness of KTE strategies found that reminders resulted in improved uptake of research evidence compared to more complex, multi-faceted KTE strategies [103]. It might be that complex, multi-faceted interventions dilute the key messages of the intervention making it difficult for decision makers to know what they should do.

As is depicted in Figure Figure2,2, that TM is optimal to both HE and KB interventions, it may be that TM provides decision makers with just the 'right amount' of information that has direct relevance to their practice, thereby making it easier to incorporate the evidence into program planning decisions. These results are supported by Hawkins et al., who found that TM employs strategies of personalization, feedback, and content matching, and that these factors work together to facilitate research use [104]. In our study, the TM intervention employed personalization and content matching, given that each decision maker received individualized messages directly matched to their current area of decision-making authority. The results suggest that passive KTE strategies, such as access to high quality synthesized evidence that the HE group had access to, is insufficient to facilitate evidence-informed decision making. It implies that in order for KTE strategies to be effective, the evidence (e.g., evidence that is relevant, high quality and synthesized) must be actively delivered directly to decision makers, rather than requiring decision makers to access it themselves, even if it is in one place. Furthermore, the results depicted in Figure Figure22 also imply that TM is optimal in comparison to knowledge brokering. It may be that the brokering intervention implemented in this study sought to address multiple aspects of the process of evidence-informed decision making, namely questioning practice, turning practice-based issues into answerable, searchable research questions, and being a critical consumer of all forms of evidence. It may be that greater attention was paid to developing skill and capacity in these areas, which may have slowed down the process of decision making during the period of follow-up. This may partially explain why the KB intervention in Figure Figure22 appears to have had no impact on evidence-informed decision making from baseline to follow-up or in comparison to the TM intervention.

The results also suggest that TM is only effective for certain organizations. As is depicted in Figure Figure3,3, the extent to which the organization valued research evidence in decision making affected the impact of the TM intervention. For example in Figure Figure3a3a where research culture was perceived as relatively low, the TM group only benefited slightly from the TM intervention. In comparison however, the TM group improved greatly when the research culture was rated as high, as is shown in Figure Figure3b.3b. It may be that those health departments reporting a high research culture are already motivated to use research evidence and what they require most is facilitated access to rigorous, summarized, relevant research evidence personalized to their decision making needs, in order to achieve evidence-informed decision making. McGregor et al. [105] reported similar findings that policy-makers were more likely to use the results of technology assessments when they requested information, and when they received the information while still engaged in making a decision on that topic. Furthermore, the results suggest that TM is not sufficient for public health departments with low research culture. It may be that in organizations with low research culture, there are other barriers to using research evidence in decision making, and that facilitating access to the research evidence does not overcome these challenges. It implies that other strategies may need to be employed to overcome barriers to evidence-informed decision making, prior to implementing a TM strategy.

One might question why organizations with high organizational research culture require tailored, targeted messages. One explanation might be that decision makers face incredible barriers to evidence-informed decision making, most notably time to find, retrieve and translate research evidence. Therefore, KTE strategies that minimize the time barrier and that assist the process of translation by tailoring and targeting findings to decision maker needs, intuitively make sense.

The inability to demonstrate a positive effect of the KB intervention counters widely held assumptions that a customized, highly interactive KTE strategy results in greater evidence-informed decision making [106]. Some have suggested that knowledge brokering is a long and involved process [107-109]. It addition to the explanations provided earlier, it is also possible that the duration and intensity of the KB intervention was insufficient to facilitate significant changes in evidence-informed decision making. It is also possible, as suggested recently [110,111], that facilitation of a community of practice was an important element missing from our KB intervention. Our KB also did not have access to a network of KBs for guidance and support, which has been shown to be crucial for optimal implementation of similar roles [112-114].

An interesting finding was the moderating effect research culture had on public health policies and programs for those in the KB group. For example, as Figure Figure3a3a illustrates, health departments with perceived low research culture exposed to the KB intervention experienced a significant and positive improvement in the number of public health policies and programs. However, no benefit and possibly a decrease in public health policies and programs was observed when research culture was high. This suggests that a KB may be effective for certain health departments and not others, particularly those not engaged in evidence-informed decision making. It is possible that KBs facilitate the development of capacity and support in health departments with low research culture, which may be an important precursor to evidence-informed decision making. Cillo [115] reported similar results, suggesting a KB had greater success when the role was matched well to both organizational context and the complexity of market knowledge. The findings demonstrate the importance of assessing characteristics of each organization (e.g., research culture), and then using this information to tailor KTE strategies to meet the needs of each organization. Furthermore, it may be that those organizations exposed to the KB intervention that had high research culture at baseline did not perceive the KB intervention as useful and did not engage in the intervention, or it may be that exposure to the KB resulted in some health departments revisiting their efforts towards evidence-informed decision making, and in doing so, re-evaluated recent policy and program decisions.

Outcome measurement

Our primary outcome, Global evidence-informed decision making, may not be optimal for measuring KTE effectiveness despite its consistent use in the literature. It is likely that this measure is too vague to elicit reliable and valid responses. One conclusion is that concrete outcome measures, such as public health policies and programs that are tied to specific behaviors and/or programs, may provide a more concrete measure of evidence-informed decision making. However, challenges still exist with the use of this measure because the existence of organizational policies does not necessarily translate into actual services being provided, and it is unclear what the optimal data source is for identifying public health policies and programs that are being implemented. Priorities for future research include: development and testing of data collection tools for measuring more objectively evidence-informed decision making outcomes; and continued exploration of subjective measures so as to better understand evidence-informed decision making and KB processes, as well as indicators of KB success.

Limitations

The limitations in this study include: the source of data, participant turnover, exposure to the intervention, and self-reported outcome measures. A decision to have just one participant from each organization provide data was made following an assessment of Canadian public health departments that suggested services for healthy body weight promotion in children were coordinated across organizations. In reality, these programs span multiple divisions (e.g., healthy lifestyles, family health) and multiple teams within divisions (e.g., nutrition, physical activity, schools), resulting in many public health professionals working simultaneously on different interventions, programs, and policies, usually with limited knowledge of what others are doing. It is possible that the participants in our study had inadequate knowledge to accurately report on all public health policies and programs provided by their organization. This may have led to both under- and over-reporting of this outcome. One strategy to overcome this issue would be to have multiple participants from each health department participate in data collection and report only on those interventions, programs, or policies directly relevant to themselves. Furthermore, while the KB encouraged multiple decision makers from each public health department to participate in the KB intervention, for many health departments only one decision maker was exposed to the intervention. This likely resulted in insufficient exposure to the intervention among health department staff to facilitate change at the organizational level.

A significant limitation of this study was high participant turnover. While the majority of health departments (81.5%) completed the study, different decision makers completed the baseline and follow-up surveys in 30% of health departments. This reflects the transient nature of public health in Canada and in the United States, and is not something that could have been avoided. This high turnover rate may have resulted in substantial error in outcome measurement and may explain some of the huge variation in the number of public health policies and programs observed from baseline to follow-up in some health departments. This continues to represent a significant issue, and one that we are unable to quantify in terms of its impact and in which direction on the data. Furthermore, given that up to 30% of participants either did not engage with the KB at all or to a limited extent, it is possible that the results of this study are generalizable only to those health departments that would engage with the KB.

The challenges we encountered in conducting this randomized controlled trial raise issues concerning the appropriateness of using empirical designs in evaluating the effectiveness of KTE strategies, particularly in public health settings. Of particular importance is the inability, through randomization, to eliminate all differences (particularly organizational ones) between comparison groups other than the intervention. This poses difficulties with interpretation of the results as it is unclear whether findings truly represent what actually occurred, or if inherent differences in organizational context masked or moderated the treatment effect. While the findings of this study (TM is effective only for organizations with certain characteristics) contribute important knowledge to the field, additional research is needed to better understand the how, what, where, and when with respect to the effectiveness of the KTE strategies. Future research in this field should integrate other designs to better understand in which circumstances KTE strategies work, for whom, and why [116]. Other designs likely to be useful and used extensively in the business literature are case studies, interrupted time series, 'N of 1' studies, and qualitative studies such as grounded theory. Mixed methods designs will allows us to understand more fully not only if KTE strategies are effective, but also more importantly why or why not, how and why they work, what impedes impact, and when KTE strategies will have the greatest likelihood of having a significant and positive impact. Finally, additional research is required to develop and test a tool for assessing organizational factors associated with evidence-informed decision making. While some assessment tools exist, no one tool currently stands out as being optimal.

Conclusion

The results of this study suggest that tailored, targeted messages are more effective than a KB or access to http://www.health-evidence.ca, particularly in organizations with a culture that highly values research. Lessons learned suggest a greater emphasis on the identification of organizational characteristics so as to identify and implement an optimal array of KTE strategies, that more attention to appropriate outcome measures is needed, and that alternative research designs may be necessary in really understanding KTE impact.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

MD conceived of the study, participated in the analysis and drafted the manuscript. PR provided the intervention and assisted in draft of the manuscript. DC, SH, RC, LO, KD, SM, and SH consulted on the intervention as it was designed and provided, and participated in review of the manuscript. All authors read and approved the final manuscript.

Acknowledgements

The authors gratefully acknowledge funding of the research project from the Canadian Institutes of Health Research, file #14126, and in-kind support of the City of Hamilton Public Health Services and Institut National De Santé Publique du Québec. The authors also gratefully acknowledge the support and guidance of Helen Thomas, Associate Professor (now retired), McMaster University, to the initial grant proposal and during the implementation of the study. The authors report no funding-related or other conflicts of interest in this work. Maureen Dobbins is a career scientist with the Ontario Ministry of Health and Long-Term Care. Results expressed in this report are those of the investigators and do not necessarily reflect the opinions or policies of the Ontario Ministry of Health and Long-Term Care.

References

  • An overview of CIHR. http://www.cihr-irsc.gc.ca/e/30240.html#slide1_e
  • Lomas J, Culyer T, McCutcheon C, McAuley L, Law S. Conceptualizing and combining evidence for health system guidance: Final report. Ottawa, ON; 2005.
  • Dobbins M, DeCorby K, Twiddy T. A knowledge transfer strategy for public health decision makers. Worldviews on Evidence-Based Nursing. 2004;1:120–128. doi: 10.1111/j.1741-6787.2004.t01-1-04009.x. [PubMed] [Cross Ref]
  • Haynes B. Of Studies, Synthesis, Synopses, Summaries and Systems: The 5 S's evolution of Information services for evidence-based healthcare decisions. Evidence Based Nursing. 2007;10:6–7. doi: 10.1136/ebn.10.1.6. [PubMed] [Cross Ref]
  • Estabrooks CA. Will evidence-based nursing practice make practice perfect? Can J Nurs Res. 1998;30:15–36. [PubMed]
  • Haynes RB. Of studies, syntheses, synopses, and systems: The 4 "S" evolution of services for finding current best evidence. ACP J Club. 2001;134:A11–13. [PubMed]
  • Kouri D. Introductory module: Introduction to decision theory and practice. Saskatoon, Saskatchewan, Canada; 1997.
  • Sibbald B, Roland M. Getting research into practice. J Eval Clin Pract. 1997;3:15–21. doi: 10.1111/j.1365-2753.1997.tb00066.x. [PubMed] [Cross Ref]
  • Dobbins M, Rosenbaum P, Plews N, Law M, Fysh A. Information transfer: What do decision makers want and need from researchers? Implementation Science. 2007. p. 2. [PMC free article] [PubMed]
  • LaPelle NR, Luckmann R, Hatheway Simpson E, Martin ER. Identifying strategies to improve access to credible and relevant information for public health professionals: A qualitative study. BMC Public Health. 2006;6:89–101. doi: 10.1186/1471-2458-6-89. [PMC free article] [PubMed] [Cross Ref]
  • Ciliska D, Hayward S, Dobbins M, Brunton G, Underwood J. Transferring public-health nursing research to health-system planning: Assessing the relevance and accessibility of systematic reviews. Can J Nurs Res. 1999;31:23–36. [PubMed]
  • Davis DA, Thomson MA, Oxman AD, Haynes RB. Evidence for the effectiveness of CME: A review of 50 randomized controlled trials. Journal of the American Medical Association. 1992;268:1111–1117. doi: 10.1001/jama.268.9.1111. [PubMed] [Cross Ref]
  • Hunt JM. Barriers to research utilization. J Adv Nurs. 1996;23:423–425. doi: 10.1111/j.1365-2648.1996.tb00001.x. [PubMed] [Cross Ref]
  • Pettengill MM, Gillies DA, Clark CC. Factors encouraging and discouraging the use of nursing research findings. Image J Nurs Sch. 1994;26:143–147. doi: 10.1111/j.1547-5069.1994.tb00934.x. [PubMed] [Cross Ref]
  • Raudonis BM, Griffith H. Model for integrating health services research and health care policy formation. Nursing & Health Care. 1991;12:32–36. [PubMed]
  • Shaperman J. The role of knowledge utilization in adopting innovation from academic medical centers. Hospital & Health Services Administration. 1995;40:401–413. [PubMed]
  • Stolk BJ, Mayo E. PhD Thesis. Univeristy of Western; 1995. Barriers to research utilization perceived by staff public health nurses.
  • Lavis JN, Robertson D, Woodside J, McLeod C, Abelson J. How can research organizations more effectively transfer research knowledge to decision makers? The Milbank Quarterly. 2003;81:221–248. doi: 10.1111/1468-0009.t01-1-00052. [PMC free article] [PubMed] [Cross Ref]
  • Canadian Health Services Research Foundation. Is research working for you? A self-assessment tool and discussion guide for health services management and policy organizations. Ottawa, Ontario, Canada; 2007.
  • Kreuter MW, Wray RJ. Tailored and targeted health communication: Strategies for enhancing information relevance. American Journal of Health Behavior. 2003;27:S227–S232. [PubMed]
  • Canadian Health Services Research Foundation. Knowledge brokering in Canada's health system: What we're doing, what we're reading. Ottawa, Ontario, Canada; 2003.
  • Edwards N, Lockett D, Gurd G, Leonard L. Effectiveness of internet-based dissemination of best practices for public health professionals. Ottawa, ON; 2000.
  • Knowledge Exchange Network (KEN) http://www.cancer.ca/ccs/internet/standard/0,3182,3331_407538771_419967129_langId-en,00.html
  • Grol R, Grimshaw J. From best evidence to best practice: Effective implementation of change in patients' care. Lancet. 2003;362:1225–1230. doi: 10.1016/S0140-6736(03)14546-1. [PubMed] [Cross Ref]
  • Lavis J, Davies H, Gruen RL. Working within and beyond the Cochrane Collaboration to make systematic reviews more useful to healthcare managers and policy makers. Healthcare Policy. 2006;1:21–33. [PMC free article] [PubMed]
  • Russell J, Greenhalgh T, Boynton P, Rigby M. Soft networks for bridging the gap between research and practice: Illuminative evaluation of CHAIN. BMJ. 2004;328:1174. doi: 10.1136/bmj.328.7449.1174. [PMC free article] [PubMed] [Cross Ref]
  • Suggs LS. A 10-year retrospective of research in new technologies for health communication. Journal of Health Communication. 2006;11:61–74. doi: 10.1080/10810730500461083. [PubMed] [Cross Ref]
  • Church J, Saunders D, Wanke M, Pong R, Spooner C, Dorgan M. Citizen participation in health decision-making: Past experience and future prospects. J Public Health Policy. 2002;23:12–32. doi: 10.2307/3343116. [PubMed] [Cross Ref]
  • Kumpers S, Mur I, Hardy B, Maarse H, van Raak A. The importance of knowledge transfer between specialist and generic services in improving health care: A cross-national study of dementia care in England and the Netherlands. International Journal of Health Planning & Management. 2006;21:151–167. doi: 10.1002/hpm.837. [PubMed] [Cross Ref]
  • Heidema AG, Boer JM, Nagelkerke N, Mariman EC, van der A D, Feskens EJ. The challenge for genetic epidemiologists: How to analyze large numbers of SNPs in relation to complex diseases. BMC Genetics. 2006;7:23. doi: 10.1186/1471-2156-7-23. [PMC free article] [PubMed] [Cross Ref]
  • Burnett S, Brookes-Rooney A, Keogh W. Brokering knowledge in organizational networks: The SPN approach. Knowledge and Process Management. 2002;9:1–11. doi: 10.1002/kpm.133. [Cross Ref]
  • Sowe S, Stamelos I, Angelis L. Identifying knowledge brokers that yield software engineering knowledge in OSS projects. Information and Software Technology. 2006;48:1025–1033. doi: 10.1016/j.infsof.2005.12.019. [Cross Ref]
  • von Malmborg F. Networking for knowledge transfer: Towards an understanding of local authority roles in regional industrial ecosystem management. Business Strategy and the Environment. 2004;13:334–346. doi: 10.1002/bse.419. [Cross Ref]
  • Focus on knowledge brokering. http://www.phcris.org.au/publications/focuson/
  • Van Kammen J, Jansen CW, Bonsel GJ, Kremer JA, Evers JL, Wladimiroff JW. Technology assessment and knowledge brokering: The case of assisted reproduction in The Netherlands. Int J Technol Assess Health Care. 2006;22:302–306. doi: 10.1017/S026646230605118X. [PubMed] [Cross Ref]
  • Choi BCK, Pang T, Lin V, Puska P, Sherman G, Goddard M, Ackland MJ, Sainsbury P, Stachenko S, Morrison H, J Epidemiol Community Health. 2005. pp. 632–637. [PMC free article] [PubMed] [Cross Ref]
  • World Health Organization. Linking research into action. Geneva: World Health Organization; 2004. pp. 97–130.
  • New directions for knowledge transfer and knowledge brokerage in Scotland: Office of Chief Researcher Knowledge Transfer Team briefing paper. http://www.scotland.gov.uk/Resource/Doc/69582/0018002.pdf
  • Hartwich F, von Oppen M. In: Adapted Farming in West Africa: Issues, Potentials, and Perspectives. Graef F, Lawrence P, von Oppen M, editor. Stuttgart, Germany: Verlag Ulrich E. Grauer; 2000. Knowledge brokers in agricultural research and extension; pp. 445–453.
  • Van Kammen J, De Savigny D, Sewankambo N. Using knowledge brokering to promote evidence-based policy-making: The need for support structures. Bull World Health Organ. 2006;84:608–612. doi: 10.2471/BLT.05.028308. [PMC free article] [PubMed] [Cross Ref]
  • Ebbeling CB, Pawlak DB, Ludwig DS. Childhood obesity: Public-health crisis, common sense cure. The Lancet. 2002;360:473–482. doi: 10.1016/S0140-6736(02)09678-2. [PubMed] [Cross Ref]
  • Canadian Fitness and Lifestyle Research Institute. Kids Can Play. 2007.
  • Arbeit ML, Johnson CC, Mott DS, Harsha DW, Nicklas TA, Webber LS, Berenson GS. The Heart Smart cardiovascular school health promotion: Behavior correlates of risk factor change. Prev Med. 1992;21:18–32. doi: 10.1016/0091-7435(92)90003-Z. [PubMed] [Cross Ref]
  • Fernandes E, McCrindle B. Diagnosis and treatment of hypertension in children and adolescents. Can J Cardiol. 2000;16:801–811. [PubMed]
  • Umlauf MG, Sherman S. Facilitating research utilization through collaboration. Nursing Connections. 1992;5:37–42. [PubMed]
  • Nicklas TA, Webber LS, Johnson CC, Srinivasan SR, Berenson GS. Foundations for health promotion with youth: A review of observations from the Bogalusa Health Study. J Health Educ. 1995;26(2):S18–S26.
  • Porkka KV, Viikari JSA, Akerblom HK. Tracking of serum HDL-cholesterol and other lipids in children and adolescence: The cardiovascular risk in young Finns study. Prev Med. 1991;20:713–724. doi: 10.1016/0091-7435(91)90066-D. [PubMed] [Cross Ref]
  • Power C, Lake JK, Cole TJ. Measurement and long-term health risks of child and adolescent fatness. International Journal Observational Related Metabolism Disorder. 1997;59:507–526. doi: 10.1038/sj.ijo.0800454. [PubMed] [Cross Ref]
  • Webber LS, Srinivasan SR, Wattingney WA, Berenson GS. Tracking of serum lipids and lipoproteins from childhood to adulthood. Am J Epidemiol. 1991;133:884–899. [PubMed]
  • Whitaker RC, Wright JA, Pepe MS, Seidel KD, Dietz WH. Predicting obesity in young adulthood from childhood and parental obesity. NEJM. 1997;337:869–873. doi: 10.1056/NEJM199709253371301. [PubMed] [Cross Ref]
  • Kelder SH, Perry CL, Klepp KI, Lytle LL. Longitudinal tracking of adolescent smoking, physical activity, and food choice behaviors. Am J Public Health. 1994;84:1121–1126. doi: 10.2105/AJPH.84.7.1121. [PMC free article] [PubMed] [Cross Ref]
  • Klepp KI, Oygard L, Tell GS, Vellar OD. Proceedings, Health behavior in adolescents: Implications for prevention of coronary heart disease. Bergen, Norway: University of Bergen; 1993. The Oslo Youth Study: A 12 year follow-up study of a school-based health education program; pp. 94–105.
  • Malina RM. Tracking of physical activity and physical fitness across the lifespan. Res Q Exerc Sport. 1996;67:S48–S57. [PubMed]
  • Raitakari OT, Porkka KV, Taimela S, Telama R, Rasanen L, Viikari JS. Effects of persistent physical activity and inactivity on coronary risk factors in children and young adults. Am J Epidemiol. 1994;140:195–205. [PubMed]
  • Sallis JF, Alcaraz JE, McKenzie TL, Hovell MF, Kolody B, Nader PR. Parental behavior in relation to physical activity and fitness in 9-year-old children. Am J Dis Child. 1992;146:1383–1388. [PubMed]
  • Katzmarzyk PT, Janssen I. The economic cost associated with physical inactivity and obesity in Canada: An update. Can J Appl Physiol. 2004;29:90–115. [PubMed]
  • Chandrashekhar Y, Anand IS. Exercise as a coronary protective factor. Am Heart J. 1991;122:1723–1739. doi: 10.1016/0002-8703(91)90290-X. [PubMed] [Cross Ref]
  • Morris CK, Froelicher VF. Cardiovascular benefits of physical activity. Herz. 1991;16:222–236. [PubMed]
  • Smith Jr SC, Blair SN, Criqui MH, Fletcher GF, Fuster V, Gersh BJ, Gotto AM, Gould KL. Preventing heart attack and death in patients with coronary disease. Circulation. 1995;92:2–4. [PubMed]
  • Wenger NK, Froelicher ES, Smith LK, Ades PA, Berra K, Blumenthal JA, Certo CM, Dattilo AM, Davis D, DeBusk RF. Cardiac rehabilitation as secondary prevention: Clinical practice guidelines #17. Rockville, MD; 1995.
  • Lee IM, Paffenbarger RS. Associations of light, moderate, and vigorous intensity physical activity with longevity: The Harvard Alumni Health Study. American Journal of Epidemiology. 2000;151:293–299. [PubMed]
  • Villeneuve PJ, Morrison HI, Craig CL, Schaubel DE. Physical activity, physical fitness, and risk of dying. Epidemiology. 1998;9:626–631. doi: 10.1097/00001648-199811000-00011. [PubMed] [Cross Ref]
  • Seeman TE, Berkman LF, Charpentier PA, Blazer DG, Alberta MS, Tinetti ME. Behavioural and psychosocial predictors of physical performance: MacArthur studies of successful aging. Journal of Gerontology: Biological Science & Medical Sciences. 1995;50A:M177–M18365. [PubMed]
  • Stuck AE, Walhert JM, Nikolaus T, Bula CJ, Hohmann C, Beck JC. Risk factors for functional status decline in community-living elderly people: A systematic literature review. Soc Sci Med. 1999;48:445–469. doi: 10.1016/S0277-9536(98)00370-0. [PubMed] [Cross Ref]
  • Unger JB, Johnson CA, Marks G. Functional decline in the elderly: Evidence for direct and stress-buffering protective effects of social interactions and physical activity. Ann Behav Med. 1997;19:152–160. doi: 10.1007/BF02883332. [PubMed] [Cross Ref]
  • Vaillant GE, Mukamal K. Successful aging. Am J Psychiatry. 2001;158:839–847. [PubMed]
  • Centers for Disease Control. Promote lifelong physical activity among young people: Risk behavior surveillance - United States. MMWR. 1997;46:1–36. [PubMed]
  • Freedson PS, Rowland TW. Youth activity versus youth fitness: Let's redirect our efforts. Res Q Exerc Sport. 1992;63:133–136. [PubMed]
  • Raphael D, Bryant T. The state's role in promoting population health: Public health concerns in Canada, USA, UK, and Sweden. Health Policy. 2006;78:39–55. doi: 10.1016/j.healthpol.2005.09.002. [PubMed] [Cross Ref]
  • Dobbins M, Ciliska D, Cockerill R, DiCenso A. A framework for dissemination and utilization of research evidence for health care policy and practice. The OnLine Journal of Knowledge Synthesis for Nursing. 2002. p. 9. [PubMed]
  • Horsley JA, Crane J, Crabtree MK. Using research to improvie nursing practice: A guide. New York: Grune & Stratton; 1983.
  • Titler MG, Klieber C, Steelman V. Infusing research into practice to promote quality care. Nursing Research. 1994;43:307–313. doi: 10.1097/00006199-199409000-00009. [PubMed] [Cross Ref]
  • Stetler CB. Refinement of the Stetler/Marram model for application of research findings to practice. Nursing Outlook. 1994;42:15–25. doi: 10.1016/0029-6554(94)90067-1. [PubMed] [Cross Ref]
  • McCormack B, Kitson A, Rycroft-Malone J, Titchen A, Seers K. Getting evidence into practice: The meaning of 'context'. Journal of Advanced Nursing. 2002;38:94–104. doi: 10.1046/j.1365-2648.2002.02150.x. [PubMed] [Cross Ref]
  • Nutley S, Walter I, Davies H. From knowing to doing: A framework for understanding the evidence-into-practice agenda. Research Unit for Research Utilisation: University of St.Andrews; 2002.
  • Landry R, Amara N, Ouimet M. Mapping the three worlds of knowledge transfer. University of Laval; 2002.
  • Rogers EM. Diffusion of Innovations. New York: The Free Press; 1995.
  • Dobbins M. PhD Thesis. Graduate Department of Health Administration, University of Toronto; 1999. Characteristics of the innovation, organization, environment and individual that facilitate the utilization of five systematic reviews among public health decision-makers in Ontario.
  • Dobbins M, Cockerill R, Barnsley J, Ciliska D. Factors of the innovation, organization, environment, and individual that predict the influence five systematic reviews had on public health decisions. Int J Technol Assess Health Care. 2001;17:467–478. doi: 10.1017/S0266462300105069. [PubMed] [Cross Ref]
  • Dobbins M, Cockerill R, Barnsley J. Factors affecting the utilization systematic reviews: A study of public health decision-makers. Int J Technol Assess Health Care. 2001;17:203–214. doi: 10.1017/S0266462300105069. [PubMed] [Cross Ref]
  • Dobbins M, Robeson P, Ciliska D, Hanna S, Cameron R, O'Mara L, DeCorby K, Mercer S. A description of a knowledge broker role implemented as part of a randomized controlled trial evaluating three knowledge translation strategies. Implementation Science. in press . [PMC free article] [PubMed]
  • Burns LR, Wholey DR. Adoption and abandonment of matrix management programs: Effects of organizational characteristics and interorganizational networks. The Academy of Management Journal. 1993;36:106–138. doi: 10.2307/256514. [PubMed] [Cross Ref]
  • Champion VL, Leach A. Variables related to research utilization in nursing: Am empirical investigation. Journal of Advanced Nursing. 1989;14(9):705–710. doi: 10.1111/j.1365-2648.1989.tb01634.x. [PubMed] [Cross Ref]
  • Cockerill R, Barnsley J. Innovation theory and its applicability to our understanding of the diffusion of new management practices in health care organizations. Healthc Manage Forum. 1997;10:35–38. [PubMed]
  • Dean JW, Sharfman MP. Procedural rationality in the strategic decision-making process. Journal of Management Studies. 1993;30:587–610. doi: 10.1111/j.1467-6486.1993.tb00317.x. [Cross Ref]
  • Kaluzny AD. Innovation in health services: Theoretical framework and review of research. Health Serv Res. 1974;9:101–120. [PMC free article] [PubMed]
  • Kimberly JR, Evanisko MJ. Organizational innovation: The influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. Acad Manage J. 1981;24:689–713. doi: 10.2307/256170. [PubMed] [Cross Ref]
  • Funk SG, Champagne MT, Wiese RA, Tornquist EM. The BARRIERS to research utilization scale. Appl Nurs Res. 1991;4:39–45. doi: 10.1016/S0897-1897(05)80052-7. [PubMed] [Cross Ref]
  • Landry R, Amara N, Lamari M. Utilization of social science research knowledge in Canada. Research Policy. 2001;30:333–349. doi: 10.1016/S0048-7333(00)00081-0. [Cross Ref]
  • Campbell K, Waters E, O'Meara S, Kelly S, Summerbell C. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews. 2002. p. CD001871. [PubMed]
  • Ciliska D, Miles E. Effectiveness of community-based interventions to increase fruit and vegetable consumption. Journal of Nutrition Education. 2000;32:341. doi: 10.1016/S0022-3182(00)70594-2. [Cross Ref]
  • Dishman RK, Buckworth J. Increasing physical activity: A quantitative synthesis. Medicine & Science in Sports & Exercise. 1996;28:706–719. [PubMed]
  • Hardeman W, Griffin S, Johnston M, Kinmonth AL, Wareham NJ. Interventions to prevent weight gain: A systematic review of psychological models and behaviour change methods. Int J Obes. 2000;24:131–143. doi: 10.1038/sj.ijo.0801100. [PubMed] [Cross Ref]
  • Kahn E, Ramsey LT, Brownson RC, Heath GW, Howze EH, Powell KE, Stone EJ, Rajab MW, Corso P. Task Force on Community Preventive Services. The effectiveness of interventions to increase physical activity. Am J Prev Med. 2002;22:73–107. doi: 10.1016/S0749-3797(02)00434-8. [PubMed] [Cross Ref]
  • Pratt BM. Wool: Interventions for preventing eating disorders in children and adolescents. Cochrane Database of Systematic Reviews. 2002. p. CD002891. [PubMed]
  • Effectiveness of physical activity enhancement and obesity prevention programs in children and youth. http://old.hamilton.ca/phcs/ephpp/Research/Summary/2004/HealthyWeightsFull2004.pdf
  • Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–1278. doi: 10.1016/0895-4356(91)90160-B. [PubMed] [Cross Ref]
  • Thomas H, Ciliska D, Dobbins M, Micucci S. A process for systematically reviewing the literature: Providing the research evidence for public health nursing interventions. Worldviews on Evidence-Based Nursing. 2004;1:176–184. doi: 10.1111/j.1524-475X.2004.04006.x. [PubMed] [Cross Ref]
  • Pinheiro J, Bates D. Mixed-Effects Models in S and S-PLUS. New York: Springer; 2000.
  • Grimshaw J, Eccles M. Is evidence-based implementation of evidence-based care possible? MJA. 2004;180:S50–S51. [PubMed]
  • Manske SR. PhD Thesis. University of Toronto; 2001. Explaining knowledge use among clients of the program training and consultation centre.
  • Ross L, Nisbett R. The person and the situation: perspectives on social psychology. New York: McGraw-Hill; 1991. Applying social psychology.
  • Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L. Toward evidence-based quality improvement: Evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966-1998. J Gen Intern Med. 2006;21:S14–S20. [PMC free article] [PubMed]
  • Hawkins RP, Kreuter M, Resnicow K, Fishbein M, Dijkstra A. Understanding tailoring in communicating about health. Health Educ Res. 2008;23:454–466. doi: 10.1093/her/cyn004. [PMC free article] [PubMed] [Cross Ref]
  • McGregor M. Impact of TAU reports. Montreal, Quebec, Canada; 2008.
  • Thompson GN, Estabrooks CA, Degner LF. Clarifying the concepts in knowledge transfer: A literature review. J Adv Nurs. 2006;53:691–701. doi: 10.1111/j.1365-2648.2006.03775.x. [PubMed] [Cross Ref]
  • Forsetlund L, Bradley P, Forsen L, Nordheim L, Jamtvedt G, Bjorndal A. Randomised controlled trial of a theoretically grounded tailored intervention to diffuse evidence-based public health practice. BMC Medical Education. 2003. p. 3. [PMC free article] [PubMed]
  • Lomas J. The in-between world of knowledge brokering. BMJ. 2007;334:129–132. doi: 10.1136/bmj.39038.593380.AE. [PMC free article] [PubMed] [Cross Ref]
  • Stetler C, Legro M, Rycroft-Malone J, Bowman C, Curran G, Guihan M, Hagedorn H, Pineros S, Wallace C. Role of "external facilitation" in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science. 2006;1:23. doi: 10.1186/1748-5908-1-23. [PMC free article] [PubMed] [Cross Ref]
  • Hargadon AB. Brokering knowledge: Linking learning and innovation. Research in Organizational behavior. 2002;24:41–85. doi: 10.1016/S0191-3085(02)24003-4. [Cross Ref]
  • Zook MA. The knowledge brokers: Venture capitalists, tacit knowledge and regional development. International Journal of Urban and Regional Research. 2004;28:621–641. doi: 10.1111/j.0309-1317.2004.00540.x. [Cross Ref]
  • Ellis I, Howard P, Larson A, Robertson J. From workshop to work practice: An exploration of context and facilitation in the development of evidence-based practice. Worldviews on Evidence-Based Nursing. 2005;2:84–93. doi: 10.1111/j.1741-6787.2005.04088.x. [PubMed] [Cross Ref]
  • Harvey G, Loftus-Hills A, Rycroft-Malone J, Titchen A, Kitson A, McCormack B, Seers K. Getting evidence into practice: the role and function of facilitation. J Adv Nurs. 2002;37:577–588. doi: 10.1046/j.1365-2648.2002.02126.x. [PubMed] [Cross Ref]
  • Kitson A, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. Evaluating the successful implementation of evidence into practice using the PARiHS framework: Theoretical and practical challenges. Implementation Science. 2008;3:1. doi: 10.1186/1748-5908-3-1. [PMC free article] [PubMed] [Cross Ref]
  • Cillo P. Fostering market knowledge use in innovation: The role of internal brokers. European Management Journal. 2005;23:404–412. doi: 10.1016/j.emj.2005.06.008. [Cross Ref]
  • Mercer SL, DeVinney BJ, Fine LJ, Green LW, Dougherty D. Study designs for effectiveness and translation research: Identifying trade-offs. American Journal of Preventive Medicine. 2007;33:139–154. doi: 10.1016/j.amepre.2007.04.005. [PubMed] [Cross Ref]

Articles from Implementation Science : IS are provided here courtesy of BioMed Central

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...