Format

Send to

Choose Destination
Eval Program Plann. 2016 Apr;55:155-62. doi: 10.1016/j.evalprogplan.2016.01.001. Epub 2016 Jan 7.

Why so many "rigorous" evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute.

Author information

1
6295 S.W. Elm Avenue, Beaverton, OR 97005, USA. Electronic address: jmichaelbamberger@gmail.com.
2
UNESCO Evaluation Office (Suite 6.063), 7 Place de Fontenoy, Paris 75007, France. Electronic address: mitarsi@hotmail.com.
3
Department of Sociology, Boston College and Director of the Women's and Gender Studies Program, McGuinn Hall 419, 140 Commonwealth Avenue, Chestnut Hill, MA 02467, USA. Electronic address: sharlene.hesse-biber@bc.edu.

Abstract

Many widely-used impact evaluation designs, including randomized control trials (RCTs) and quasi-experimental designs (QEDs), frequently fail to detect what are often quite serious unintended consequences of development programs. This seems surprising as experienced planners and evaluators are well aware that unintended consequences frequently occur. Most evaluation designs are intended to determine whether there is credible evidence (statistical, theory-based or narrative) that programs have achieved their intended objectives and the logic of many evaluation designs, even those that are considered the most "rigorous," does not permit the identification of outcomes that were not specified in the program design. We take the example of RCTs as they are considered by many to be the most rigorous evaluation designs. We present a numbers of cases to illustrate how infusing RCTs with a mixed-methods approach (sometimes called an "RCT+" design) can strengthen the credibility of these designs and can also capture important unintended consequences. We provide a Mixed Methods Evaluation Framework that identifies 9 ways in which UCs can occur, and we apply this framework to two of the case studies.

KEYWORDS:

Evaluation design; Mixed-methods; Randomized control trials; Uninintended consequences

[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Elsevier Science
Loading ...
Support Center