• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Int J Obes (Lond). Author manuscript; available in PMC Sep 23, 2009.
Published in final edited form as:
PMCID: PMC2749285
NIHMSID: NIHMS136832

Rationale, design and methods for process evaluation in the HEALTHY study

Abstract

The HEALTHY study was a multi-site randomized trial designed to determine whether a 3-year school-based intervention targeting nutrition and physical activity behaviors could effectively reduce risk factors associated with type 2 diabetes in middle school children. Pilot and formative studies were conducted to inform the development of the intervention components and the process evaluation methods for the main trial. During the main trial, both qualitative and quantitative assessments monitored the fidelity of the intervention and motivated modifications to improve intervention delivery. Structured observations of physical education classes, total school food environments, classroom-based educational modules, and communications and promotional campaigns provided verification that the intervention was delivered as intended. Interviews and focus groups yielded a multidimensional assessment of how the intervention was delivered and received, as well as identifying the barriers to and facilitators of the intervention across and within participating schools. Interim summaries of process evaluation data were presented to the study group as a means of ensuring standardization and quality of the intervention across the seven participating centers. Process evaluation methods and procedures documented the fidelity with which the HEALTHY study was implemented across 21 intervention schools and identified ways in which the intervention delivery might be enhanced throughout the study.

Keywords: school-based, process evaluation, adolescent, diabetes, diet, physical activity

Introduction

Responding to the dramatic increase in obesity and type 2 diabetes among children and adolescents, in 2002 the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health sponsored a collaborative agreement to develop and conduct both a treatment and a prevention trial of type 2 diabetes in children and adolescents. After a series of pilot studies, the prevention arm of this effort culminated in the development and conduct of the HEALTHY primary prevention trial. Seven centers across the country recruited 42 middle schools, and half were randomly assigned to receive a multicomponent intervention. The study followed students through the 3 years of middle school (sixth grade to eight grade) from 2006 to 2009.1

The HEALTHY intervention was designed to make changes in the school nutrition and physical education (PE) environments, provide opportunities for behavior change and education, and disseminate communications campaigns and promotional messages, events and activities. The four components—nutrition, PE, behavior and communications—interacted synergistically within a single integrated intervention. The HEALTHY study group recognized the importance of documenting delivery of the intervention components to address problems and make modifications, as well as for use in the interpretation of study findings. Data collected during an intervention study, either for the purpose of analyzing factors that may explain the intervention's success or failure or for the purpose of fine-tuning intervention delivery, are generally referred to as process evaluation data.2 Experts from across the study group and representing each of the intervention components formed a Process Evaluation Committee to develop the methods as well as to monitor and report findings on an ongoing basis.

This paper describes the design and methods of the process evaluation of the HEALTHY study. The procedures used to obtain both qualitative and quantitative data are described, and the methods for rapidly summarizing the data to fine-tune intervention delivery are delineated. Valuable lessons learned are shared on the basis of our experience in conducting process evaluation in a study that was complex not only because of its multifaceted intervention goals but also because of its implementation across widely diverse geographical locations with multiple schools within each center and the inherently challenging delivery environment—middle school.

Background and rationale

Large multi-site public health interventions are challenging to implement and difficult to evaluate. Many such efforts have omitted from their planned data collection any systematic documentation or evaluation of how the intervention was delivered and/or received.3 Only three out of ten long-term (that is, at least 12 months) randomized childhood obesity prevention intervention studies reported information that might be used to explain the success or failure of the intervention.4

Process evaluation data collection and procedures are being increasingly incorporated into public health intervention studies.5 Process evaluation of a school-based intervention designed to increase physical activity and reduce bullying reported that only 39% of planned intervention events were implemented and pointed to institutional barriers that impeded the implementation of the intervention.6 Three other recently reported community-based interventions designed to alter health behaviors also reported rigorously designed process evaluation components.79 In all three cases, the process evaluation data offered valuable insights into reasons why the programs may have been more or less effective in achieving their goals.

Process evaluation adds value to the analysis of multi-component, community-based interventions by documenting characteristics of the intervention and by eliciting information about barriers to and/or facilitators of the intervention components. These functions are particularly salient in a multi-site trial such as HEALTHY, where the intervention may be implemented and received in different ways.10 In terms of documenting the extent to which an intervention is delivered as intended, process evaluation is typically described as encompassing three dimensions: fidelity, dose and reach.2 The fidelity of intervention delivery includes the extent to which the intervention is delivered as intended. The intervention dose refers to how much of the intended intervention is delivered. The reach is the proportion of intended recipients who actually participate in an intervention. These three dimensions can be tracked using various forms of monitoring, including observations, surveys and program documentation. The elicitation of barriers to and facilitators of successful intervention implementation is often accomplished using qualitative research methods such as interviews and focus groups. The HEALTHY study made use of all these methods to document program administration and identify factors that enhanced or impeded the intervention implementation.

Preliminary studies

The study group conducted a series of pilot studies to inform the final design of the HEALTHY study. Each of these preliminary studies yielded information that provided valuable input to the final design of the intervention and the process evaluation plan. For example, data from the pilot studies showed the critical importance of ongoing mentoring relationships between the HEALTHY study staff and the school PE teachers. In the main trial, therefore, each center study staff included an expert responsible for implementation of the PE intervention component. The PE expert observed and visited with PE teachers at the intervention schools repeatedly throughout the intervention. This modification was a shift away from the original plan, in which study staff would train the teachers and then refrain from intervening. Similarly, the pilot and formative studies revealed that local school food service managers had minimal control over the foods served in their cafeterias, and it was therefore critical to cultivate a partnership with food service directors at the district level. Subsequently, the nutrition intervention component incorporated national summits to which all the school district food service directors were invited. These meetings provided opportunities for nurturing a partnership with district-level food service directors and facilitating agreement and support for study-mandated school food service changes. Pilot studies of the behavior intervention component in-class educational sessions revealed that students misreported their behavior to be eligible for contest prizes. This resulted in minimizing performance-based competition in the main trial.

Perhaps the greatest challenge to implementing the intervention as designed was the enormous range of institutional characteristics across schools in terms of size, policies and procedures, and relative receptivity to an externally imposed intervention. To maximize the study's chances for success, the HEALTHY study team developed a flexible planning model, in which intervention and research methods could be adapted to local circumstances while maintaining overall standardization of methods and procedures.

The study group also used the pilot studies to develop formative and process evaluation methods and procedures for the main trial. With the primary emphasis being placed on implementation of the intervention and collection of outcome data, collecting process evaluation data had to minimize the level of burden. The frequency and length of interviews with school faculty and staff were trimmed, and replaced with interviews of study staff to monitor barriers and facilitators to program implementation during the main trial. Data collection forms and procedures were revised and streamlined through the use of checklists. The end result was more effective design of the process evaluation methods for the main trial.

The HEALTHY intervention

To understand the process evaluation methods and procedures, it is important to understand the HEALTHY intervention and its delivery. The HEALTHY study incorporated four distinct, yet closely integrated, intervention components. The PE component targeted school PE by introducing a curriculum of HEALTHY PE lesson plans and classroom management to increase student levels of participation in moderate-to-vigorous physical activity.11 The nutrition intervention component was designed to modify the quantity and quality of food and beverage offerings throughout the total school food environment, including cafeteria meals and after school snacks, as well as a la carte and vending machines.12 The behavior intervention component provided students with knowledge and skills that would facilitate healthy lifestyle choices both inside and outside the school environment.13 The communications campaign intervention component used social marketing principles to design and place promotional materials in the intervention schools, to motivate students and build excitement within the student population around each semester's behavior goals, and to integrate the disparate elements of the intervention into a cohesive whole. To support and promote HEALTHY intervention activities in the school, a group of students from the target grade were recruited each year to take on leadership roles as student peer communicators (SPCs).14

HEALTHY process evaluation

Design of the process evaluation followed the steps and principles delineated by Steckler and Linnan2 and summarized briefly by Saunders et al.5 The first step was the finalization of a clear description of the intervention itself, together with the specification of what constituted complete and acceptable program delivery. Next, the Process Evaluation Committee proposed methods and considered them in relation to program resources, characteristics of the school environment and qualities of the intervention participants. Proposed methods were revised repeatedly in response to feedback from the study group until consensus was achieved.

Process evaluation typically uses both quantitative and qualitative methods, including structured observations, questionnaires, semi-structured interviews, focus groups and logs. Quantitative methods have the advantage of being amenable to quick analyses, brief reports and relatively straightforward interpretation, but often are not capable of answering questions about why and how, for example, a particular intervention component was not being received as intended. Qualitative methods have the advantage of being able to elicit unanticipated information, suggested solutions and/or innovations that address these sorts of questions, as well as the diverse perspectives of different groups participating in the intervention (that is, teachers and students). For this reason, the HEALTHY process evaluation plan incorporated both quantitative and qualitative methods.

The schedule of process evaluation data collection was tied to the local school calendar. Procedures for collecting the data were established relative to semesters for each school year of the study. Data collected in one semester were quickly processed and reported back to the study group for consideration of the status of intervention delivery successes and failures.

In addition to the study-wide reports of process evaluation data, the study's Procedures Oversight Committee reviewed quantitative process evaluation data at the end of each semester and engaged in communications with the study staff to understand lapses in intervention delivery or adherence. Resolution could be specific to a school or center, or could lead to study-wide modifications and improvements. This process provided an additional means of fine-tuning the intervention delivery and helped maintain consistent implementation of the intervention activities across centers.

Process evaluation staffing and training

To minimize bias and maximize the quality of the process evaluation data, personnel responsible for collecting these data were rigorously trained and enjoined to maintain strict segregation from the intervention activities. Specifically, the research assistants (RAs) who collected the process evaluation data were prohibited from participating in any intervention implementation activities. In a multi-year, labor-intensive and complex trial such as the HEALTHY study, in which some communication between evaluators and implementation staff is necessary for the successful execution of the protocol, it is a constant challenge to foster the neutrality of process evaluation staff. In addition to the rigorous training required of all RAs, in which the potential for bias was emphasized and strategies for reducing bias and error were discussed, each center hosted a site visit each semester at which, among other things, the degree of potential ‘contamination’ between evaluation and implementation was assessed and, if necessary, addressed.

Each site used a small number of permanent RAs and identified a process evaluation coordinator from among the staff to coordinate the process evaluation data collection. RAs retained primary responsibility for collecting all process evaluation data, whereas the process evaluation coordinator represented the center on the study-wide Process Evaluation Committee. The committee retained primary responsibility for determining and designing process evaluation methods, instrumentation and training sessions for center RAs. In addition, the Process Evaluation Committee functioned as a problem-solving body to modify and interpret the process evaluation procedures.

Research assistants attended a biannual central training session at which they received an overview of the purpose of process evaluation, instruction in data gathering skills (for example, interviewing techniques, focus group strategies), orientation to each of the data collection forms, and training on procedures for entering or summarizing data to be forwarded for central processing; the latter included how to condense the focus group and interview information into key point summaries. The importance of practice with the scripts before conducting interviews or focus groups was emphasized. Particular attention was also placed on performing the structured observations consistently with regard to global ratings of student engagement in activities. Central training on observations was followed by opportunities for practice before actual study data collection.

Procedures and measures

Table 1 shows the different pieces of process evaluation data collected throughout the study. In designing the process evaluation plan, the choice of methods was strongly influenced by considerations of feasibility, including the limitations of available resources, likely respondent burden, acceptability of methods and instrumentation to local school and study staff, and the likelihood of obtaining information of the same quality through alternative (for example, less intrusive and burdensome) methods. An early study decision was made to devote the largest share of evaluation resources to assessing fidelity, barriers and facilitators with the goals of making the intervention as robust as possible throughout the 3-year intervention, maximizing the consistency in intervention delivery across the 21 intervention schools, and rigorously documenting intervention implementation. Owing to feasibility and resource limitations, the Process Evaluation Committee opted to forgo the assessment of reach, although an indirect assessment was obtained through administrative documents used to manage and track the intervention throughout the course of the study. Similarly, a formal plan for documenting intervention dose was not included in the process evaluation design because the intervention was delivered under controlled conditions to the entire grade that was targeted each year. That is, students did not have the ability to ‘opt out’ of the intervention, which was delivered through changes to the school environment.

Table 1
HEALTHY process evaluation data collection, with actual (and expected) counts

Structured observations of PE classes

Research assistants conducted structured observations of a sample of PE classes at each intervention school throughout each semester of the intervention. The total number of observations per school varied depending on the allocation of classes to teachers (minimum of three randomly selected class observations per PE teacher each semester). Observations were spread throughout the semester to obtain data across the time period. Items on the observation form assessed whether the class was delivered in a manner that reflected the intent and design of the HEALTHY lesson plans, whether a majority of students engaged in activity quickly at the beginning of the class, whether the PE teacher assistant from the HEALTHY study staff was present during the class, and whether the RA observed any barriers that might have prevented or discouraged the students from engaging in moderate-to-vigorous physical activity.

Structured observations of the nutrition environment

Research assistants conducted structured observations of the total school food environment at each intervention school twice each semester throughout the intervention. To complete the structured observation, the RAs directly observed all food and beverage points of service, that is, breakfast, lunch, a la carte, vending machines and school stores. To supplement direct observation, the RAs reviewed the menus, work production sheets and specification sheets in the food service manager's file, as well as the product information notebook compiled by the study staff nutrition interventionist. This notebook contained nutrition information about any new food items introduced to the total school food environment during the study.

Observations of the total school food environment were designed to determine whether the school was meeting the food service and nutritional goals established by the HEALTHY study. The RAs determined which of the 12 unique core strategies of the nutrition intervention were being met at the time of the observation.12

Structured observations of behavior intervention classes

Research assistants conducted structured observations of a sample of classes administering the behavior education program at each intervention school in each semester of the intervention. Each teacher selected and trained to deliver the educational program was observed at least once during each intervention semester. The class to be observed was selected randomly. The RA assessed whether the session was delivered as intended, whether the majority of students were able to follow the instructions, and whether there were any observed barriers to implementation.

Structured observations of the communications campaign

Much of the information needed to track the communications portion of the intervention was obtained using logs completed by study staff responsible for implementing campaign events (for example, the intervention launch) and distributing campaign materials (for example, posters, t-shirts). Study staff completed a worksheet that documented the placement and removal of campaign elements. In addition, the RAs conducted three to four structured observations each semester during which they observed and documented the number, nature and condition of communications materials displayed throughout the school.

Structured interviews with intervention staff

Toward the end of each semester of the intervention, a structured interview was conducted with each HEALTHY intervention staff member responsible for training and supporting the school staff who implemented the intervention components. These interviews were used to explore and record the interventionists’ perceptions of the degree to which each intervention component was being implemented as planned within each school, as well as to identify barriers to and facilitators of the intervention. For all these interviews, the interview guide included prompts designed to elicit information about differences among schools and/or across teachers within schools, barriers to intervention implementation, and facilitators of intervention success.

Structured interviews with students

Each semester during the intervention, RAs interviewed a sample of 20 consented students (or 10% of consented students at the school, whichever was greater) at each intervention school. Consented students were those who had provided written parental consent and personal assent to participate in data collection for the HEALTHY study. These interviews assessed the students’ recollection of intervention components, their perceptions of specific study activities and their overall opinions about the HEALTHY study.

SPC focus groups

Toward the end of each semester during the intervention, all SPCs at each school were invited to participate in a focus group discussion. Depending on the number of SPCs at each school, RAs conducted up to three focus groups to elicit information regarding student perceptions of the HEALTHY intervention. The process also provided an opportunity for SPCs to comment on their own role in the study and to make suggestions for the coming semester.

Structured interviews with school administrators

At the end of each intervention year, an RA, the process evaluation coordinator, or a trained co-investigator conducted an interview with the primary administrative liaison at each intervention school. Open-ended questions were designed to collect information about whether and how the study had affected the regular functioning of the school (for example, school scheduling, student comportment, school employee morale) and how the study was perceived by the school administration.

Structured interviews with school staff

During the bulk of the intervention, we refrained from collecting information from school staff involved in implementation to avoid excessive respondent burden and potential alienation of school staff. In the final year of the intervention, however, interviews were conducted with two PE teachers, one food service manager and two behavior class teachers per intervention school, as well as all relevant district-level food service directors. Questions were designed to ascertain perceptions of the study as a whole, including opinions about the parts of the intervention with which they were most familiar, and suggestions for enhancing future implementation of similar interventions.

Behavior class teacher feedback forms

Additional information about the behavior in-class educational program was obtained using a brief form that was completed by the classroom teacher each time an educational session was delivered. This form provided a means of tracking the date that each session was delivered, the time that it took to deliver each session, and whether there were any barriers to implementation.

SPC tracking logs

Throughout the intervention, study staff maintained a log of each time an SPC received study-related training or participated in a study-related activity. This log provided a means of ensuring that no SPC devoted more than 1 h per week out of class to study-related activities, and enabled the study staff to document that SPCs had assisted at study-sponsored activities and had received the relevant and required study trainings.

Analysis plan and methodological approaches

Data obtained using structured observation forms were entered through a custom web-based data entry system and accumulated centrally. Data obtained from focus groups and interviews were summarized into a key point summary (KPS) by trained facilitators at each center. The KPS provided major points from the interviews and focus groups and salient respondent quotes. Individual KPSs were composed for each study and school staff interview, whereas student interviews were aggregately summarized into a single KPS per school each semester. Each KPS was then forwarded to the qualitative data core (QDC) located at the University of North Carolina at Chapel Hill.

Summary tabulations of forms data were provided to the QDC for further analysis of descriptive statistics and report compilation. To provide timely recommendations, the QDC designed the KPS methodology for rapid data analysis while maintaining the integrity and quality of the data. At the end of each interview transcript, the interviewer added a brief summary page with interview responses for specific questions. When analyzing the data, researchers at the QDC primarily analyzed these summary points, referencing the full interview transcript as needed. On the basis of internal, QDC-operated analysis testing, when gathering information on strengths and challenges of the intervention, the KPS methodology provided similar data as full transcript analysis. The QDC coded the KPSs using ATLAS.ti 5.2 (Scientific Software Development GmbH, Berlin) and analyzed each set of interview and focus group data (grouped by semester) for emergent themes.15 This analysis of aggregate data focused primarily on the strengths of and barriers to different intervention components. Program recommendations were made on the basis of these data analyses.

Results

Table 1 provides a simple numerical report of the quantity of data collected during the HEALTHY study. Numbers of completed process evaluation data collection activities were compared with the number of planned or expected data collection activities. Completion was near or at 100% of intended data collection activities throughout the study, and at times exceeded 100% when sites opted to collect additional interviews or observations. One reason for this high adherence to the process evaluation design was the impact of the periodic process evaluation reports summarizing the data from the previous semester. These scheduled reviews of the process evaluation data ensured that any difficulties that emerged with collecting the process evaluation data were identified and resolved in a timely manner.

Discussion

The process evaluation methods and procedures used in the HEALTHY study combined qualitative and quantitative methods to document the intervention as it was delivered, identify areas where delivery was suboptimal or problematic, and contribute to the development of strategies to address barriers to and problems with delivery of the intervention components. The data collected were used to make ‘mid-course’ adjustments in intervention delivery and will be used to interpret trial outcomes. As the study progressed, many lessons were learned about the challenges and benefits of conducting a rigorous process evaluation during a complex multi-site intervention trial.

The existence of the Process Evaluation Committee as a continuous monitoring and problem-solving body proved crucial in sorting through the many divergent school settings in which the study was conducted and the natural evolution of the intervention itself. Differences among the sites in the ways that PE, behavior classes and school food services were structured had an impact on the implementation of the process evaluation procedures, and required the committee to modify the process evaluation data collection and instruments to most accurately capture intervention implementation in a generalized manner, while taking into consideration variations in locations and circumstances. The design of the communications intervention component evolved across the course of the trial in response to the preferences and trends that characterized our target population as they matured from sixth grade to eight grade. Again, the ongoing operation of the Process Evaluation Committee provided a mechanism for response to these changes and ensured that the evaluation tools remained appropriate to the intervention as delivered.

The plan by which process evaluation data were quickly analyzed and used to provide feedback to the intervention teams had some unexpected side effects over the course of the study. One positive outcome of the effort to provide rapid and timely feedback was the quick identification of data collection methods that were not performing adequately and required modification. We also discovered, however, that the mechanisms put in place to accommodate the rapid feedback system had the unfortunate effect of making study staff feel occasionally threatened by the process that asked them to quickly respond to performance criteria and then took those responses into a central arena for review and evaluation. The system felt more judgmental than deliberative and problem solving. Some study staff members developed a hesitancy to describe implementation problems or concerns occurring at their site. To address this, the Procedures Oversight Committee, which was representative across study sites, was directed to talk about the purpose of and need for the rapid feedback system. Discussions were also held with the Steering Committee and with project coordinators, who then spoke with their local team members to further their understanding of the nature and importance of process evaluation procedures in a multicentered trial. These measures, combined with regular site visits by study leadership, overcame initial reluctance as rapport was established across the study group and members came to appreciate that the system improved not only the delivery of the intervention but their own efforts as well.

Despite the apparent utility of the rapid feedback system that was established to monitor and fine-tune the intervention delivery as the study progressed, the translation of recommendations into changes in intervention implementation may have been hampered for several reasons. First, the timing was problematic. Analysis of the process evaluation data commenced as quickly as possible at the end of each semester and reports were issued early in the following semester, but by that time it may have been too late to implement some changes for that intervention period. Thus, the implementation of recommended changes may have been delayed or even neglected as the intervention moved ahead. Another reason that changes may have failed to be integrated into the intervention delivery was a lingering skepticism about the utility of the process evaluation data among interventionists. As a result, considerable effort went into educating study staff about the value of process evaluation, with the effect that the data generated by the process evaluation had an increasingly influential role as the study progressed.

The need to maintain a separation between evaluation and implementation staff was a constant challenge to the integrity of the process evaluation, especially as some knowledge of and familiarity with the intervention was beneficial to RAs as they attempted to process the information they obtained. Moreover, communication between the evaluators and the interventionists was at times necessary to support the successful completion of the evaluation plan. There were times, for example, when an RA was informed by an interventionist about special events occurring in the school that would be likely to interfere with the process evaluation data collection. At some sites where schools could be located hours of driving time away from the study offices, such information was extremely useful.

The process evaluation of the HEALTHY study used both quantitative and qualitative methods of data collection. In the health sciences, qualitative methods are often viewed as less reliable or valid compared with quantitative methods, and debate continues concerning whether validity and reliability are appropriate terms to use in describing qualitative research, or whether they should be replaced by terms such as trustworthiness and rigor.1518 This perception stems in part from the lack of research validating the use of qualitative methods in health research. Unlike standardized assessment tools, the type of qualitative data collection procedures used in this and similar studies is tailored to the particular intervention and can only be evaluated for validity within the context of the setting for which they were designed. When qualitative process data have been used to examine possible mediators and/or moderators of intervention impact, they have been found to be useful for interpreting study outcomes.19,20 More research is needed, however, to document the utility of these methods.

The use of mixed methods to obtain information about the intervention implementation was extremely valuable, given the complexity of the study, with its multiple sites and multicomponent intervention. By triangulating the data collected from different sources (for example, observations, interviews, focus groups), we were able to create a fairly accurate picture of what actually transpired in the schools. Naturally, it is virtually impossible to eliminate socially desirable responses in qualitative research, but by obtaining multiple readings on each intervention component, we believe that we have documented accurately how the intervention was delivered throughout the HEALTHY study. This information will contribute toward interpreting the outcome results of the study, and will be invaluable in providing a template of the intervention for future dissemination.

Acknowledgements

Past and present members of the Process Evaluation Committee are Allan Steckler (chair), Tom Baranowski, Stan Bassin, Steve Bruecker, Wendy Burd, Ashanti Canada, Lynn DeBar, Eileen Ford, Debby Ford, Stacy Grau, Natasha Greene, Will Hall, Joanne Harrell, Jill Hartstein, Art Hernandez, Katie Hindes, Ann Jessup, Sean Kolmer, Chung-Hui Lin, Mariam Missaghian, Griselle Montez, A. Gayle Moore, Mia Morris, Patricia Pearce, Trang Pham, Kimari Phillips, Amanda Phillips-Martinez, Leigh Rosen, Margaret Schneider, Sandy Sherman, Linda Simkin, Adriana Sleigh, Debbe Thompson, Victoria Thompson, Thang Trinh, Beth Venditti, Stella Volpe, Alyssa Voss, Maihan Vu, Lisa Wald, Alissa Wheeler, Mamie White and Abby Zeveloff. We certify that all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during this research.

Footnotes

Conflict of interest

The authors declare no conflict of interest.

References

1. The HEALTHY Study Group HEALTHY study rationale, design and methods: moderating risk of type 2 diabetes in multi-ethnic middle school students. Int J Obes. 2009;33(Suppl 4):S4–S20. [PMC free article] [PubMed]
2. Steckler A, Linnan L. Process Evaluations for Public Health Interventions and Research. Jossey-Bass; San Francisco: 2002.
3. Armstrong R, Waters E, Moore L, Riggs E, Cuervo LG, Lumbiganon P, et al. Improving the reporting of public health intervention research: advancing TREND and CONSORT. J Public Health. 2008;30:103–109. [PubMed]
4. Brown T, Summerbell C. Systematic review of school-based interventions that focus on changing dietary intake and physical activity levels to prevent childhood obesity: an update to the obesity guidance produced by the National Institute for Health and Clinical Excellence. Obes Rev. 2008;10:110–141. [PubMed]
5. Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6:134–147. [PubMed]
6. Bowes D, Marquis M, Young W, Holowaty P, Isaac W. Process evaluation of a school-based intervention to increase physical activity and reduce bullying. Health Promot Pract. 2008. doi:10.1177/1524839907307886. Available on-line at http://hpp.sagepub.com/cgi/content/abstract/1524839907307886v1. [PubMed]
7. Feathers JT, Kieffer EC, Palmisano G, Anderson M, Janz N, Spencer MS, et al. The development, implementation, and process evaluation of the REACH Detroit partnership's diabetes lifestyle intervention. Diabetes Educ. 2007;33:509–520. [PubMed]
8. Sy A, Glanz K. Factors influencing teachers’ implementation of an innovative tobacco prevention curriculum for multiethnic youth: project SPLASH. J Sch Health. 2008;78:264–273. [PMC free article] [PubMed]
9. Young DR, Steckler A, Cohen S, Pratt C, Felton G, Moe SG, et al. Process evaluation results from a school- and community-linked intervention: the Trial of Activity for Adolescent Girls (TAAG). Health Educ Res. 2008;23:976–986. [PMC free article] [PubMed]
10. Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332:413–416. [PMC free article] [PubMed]
11. McMurray RG, Bassin S, Jago R, Bruecker S, Moe EL, Murray T, et al. for the HEALTHY Study Group Rationale, design and methods of the HEALTHY study physical education intervention component. Int J Obes. 2009;33(Suppl 4):S37–S43. [PMC free article] [PubMed]
12. Gillis B, Mobley C, Stadler DD, Hartstein J, Virus A, Volpe SL, et al. for the HEALTHY Study Group Rationale, design and methods of the HEALTHY study nutrition intervention component. Int J Obes. 2009;33(Suppl 4):S29–S36. [PMC free article] [PubMed]
13. Venditti EM, Elliot DL, Faith MS, Firrell LS, Giles CM, Goldberg L, et al. for the HEALTHY Study Group Rationale, design and methods of the HEALTHY study behavior intervention component. Int J Obes. 2009;33(Suppl 4):S44–S51. [PMC free article] [PubMed]
14. DeBar LL, Schneider M, Ford EG, Hernandez AE, Showell B, Drews KL, et al. for the HEALTHY Study Group Social marketing-based communications to integrate and support the HEALTHY study intervention. Int J Obes. 2009;33(Suppl 4):S52–S59. [PMC free article] [PubMed]
15. Patton M. Qualitative Evaluation and Research Methods. Sage Publications; Newbury Park, CA: 1990.
16. Collingridge DS, Gantt EE. The quality of qualitative research. Am J Med Qual. 2008;23:389–395. [PubMed]
17. Kitto SC, Chesters J, Grbich C. Quality in qualitative research. Med J Aust. 2008;188:243–246. [PubMed]
18. Porter S. Validity, trustworthiness and rigour: reasserting realism in qualitative research. J Adv Nurs. 2007;60:79–86. [PubMed]
19. Resnicow K, Davis M, Smith M, Yaroch AL, Baranowski T, Baranowski J, et al. Wang OT How best to measure implementation of school health curricula: a comparison of three measures. Health Educ Res. 1998;13:239–250. [PubMed]
20. McGraw SA, Sellers DE, Johnson CC, Stone EJ, Backman KJ, Bebchuk J, et al. Using process data to explain outcomes. An illustration from the Child and Adolescent Trial for Cardiovascular Health (CATCH). Eval Rev. 1996;20:291–312. [PubMed]

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...