Presented by Christoph Spurk, Zurich University of Applied Sicences (Switzerland)

Measuring the impact of media projects has been and still is a challenge for many organisations working for media development. Using the original “theory of change” ideas, developed by evaluation practitioners some years ago, the workshop demonstrated the advantages of a result chain oriented evaluation approach when it comes to learning, compared to the widely used logframes (which have other advantages).

A study for Swiss Development cooperation shows that formulating detailed results chains for media projects can be not only insightful for planning, but also used in evaluation as a tool to see what works and what doesn’t. It offers more learning than other quantitative evaluation approaches (such as before-after designs with treatment and control groups). This new approach looks very appropriate to a sector like media development, which has struggled to develop meaningful evaluations.

The workshop also provided an example of realistic evaluation in a radio project in Mali, demonstrating that media projects have achieved interesting results at concrete levels, like raising the discussion within families about conflicts and their causes, but also showing that some of the high level and very ambitious goals (such as democratisation or contributing to the end of violence) were not achieved.

What makes this experience relevant to others?

Measuring impact is a challenge for everyone working in development cooperation. Many donors request serious evaluations, but big quantitative ones are very costly and demanding, and therefore still in short supply. One of the reasons for this shortage is that the results of those quantitative impact evaluations cannot tell why projects have worked; they can just state how much they have contributed towards achieving a goal. Now, the new approach suggested here, tells a lot about why projects have worked (or not) in the way foreseen (by the result chain), and where the result chain broke or stopped.

Interested in further resources?

• Weiss, Carol. H. (1995): Nothing As Practical As Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families, in: Connell, James P.; Kubisch, Anne C.; Schorr, Lisbeth B., and Carol H. Weiss (Ed.) New Approaches to Evaluating Community Initiatives. Concepts, Methods, and Contexts. Aspen Institute for Humanistic Studies, New York, pp. 65–92
• Schoemaker, E. and Stremlau, N. (2014): The Media and Conflict: An Assessment of the Evidence, in Progress in Development Studies 14, 2 (2014) pp. 181–195
• Open training resources on evaluation and impact assessment from the CAMECO literature database.