The

Building Capacity to Use Research Evidence (BCURE) programme, was a £15.7 million initiative aiming to improve the use of evidence in decision making in low and middle-income countries.

I led the evaluation team that explored how and why capacity building within government systems could lead to greater use of evidence in policy and planning. The BCURE evaluation ran from 2014-2017, and was implemented by

Itad, who I often team up with.

Using realist evaluation

We used a realist evaluation design - a specific theory-based design, one of a family of theory-based evaluation approaches, e.g. contribution analysis, theory-of-change based approaches, and process tracing. We chose realist evaluation over the others because it's particularly useful in pilot programmes where it is not clear exactly how and why the intervention will work; where programmes are implemented across multiple settings and are likely to lead to different outcomes for different groups in diverse contexts; and where commissioners are interested in learning lessons about how to scale up or roll out an intervention (see

Gill Westhorp 2014 for an introduction to realist evaluation).

Where do mechanisms come in?

At the heart of realist evaluation lies the concept of a causal mechanism. 'Mechanism' is quite a conceptual and hotly contested thing in evaluation circles, drawing on much underlying theory and philosophy.

However, when you start to think about it, it's quite intuitive - we use the idea of mechanisms a lot in everyday life. For example, we talk about ideas gaining a 'critical mass', and reaching a 'tipping point.' We use 'carrots' and 'sticks' to describe the way different behaviour change incentives work. Economists on the news talk about the 'trickle down' effect (which is an example of a mechanism that doesn't work, or at least the conditions haven't ever been in place). These are all 'mechanisms' - a metaphor to describe a process that triggers change.

For the BCURE evaluation, we framed this definition:

“Mechanisms are the causal forces, powers, processes, or interactions that generate change within an intervention – including the choices, reasoning, and decisions people make as a result of the resources the programme provides.” (

→ www.itad.com/wp ᠁

Vogel & Punton, 2018, p.5)

In our framing, mechanisms are real, although usually invisible. They often occur at the level of individual behaviour, including the choices, reasoning, and decisions that people make as a result of the resources provided by a programme – for example, an intervention may seek to incentivise ('carrot') or deter ('stick') particular behaviour among its participants. However, mechanisms can also include the interactions between people or groups and the powers that people or institutions have as a result of their position in a group or society - 'influence' is another example of a mechanism in this situation.

What's useful about a 'mechanism' way of thinking is that you can link a particular mechanism up to broader theoretical ideas (e.g. from psychology or behaviour change evidence) so that the change process you seen one setting can be translated to another. You can then start to dig into why it works here and differently there, and understand the contextual factors that are needed to spark the mechanism. The central point is that you can then design and tailor programme activities so that they hopefully optimise the right conditions for the mechanism to work and spark change.

What did we learn?

• Realist evaluation is useful but can be hard to implement

Realist evaluation proved to be the right design for BCURE, but it wasn't easy to implement. Over three years of work, we faced many thorny challenges, including how to link up mechanisms working at an individual level to those working at a team level and then to an institutional level - the capacity to use evidence for decision making turned out to be a system-wide capacity.

You can read more on how we navigated these challenges and more in our reports, methods and findings from the three year evaluation. Together with

Mel Punton, we have written a number of reflection papers as well. All our reports and papers are published on the Itad website here and are free to download.

• Clear and intuitive communication is key

If you dive into realist evaluation, you'll find a lot of abstract concepts and jargon. Though important, we learned that this can get in the way of evaluation commissioners and users being able to take up the findings and apply them.

We tried many different ways of communicating insights. By framing our mechanisms in everyday labels, and pinpointing the contexts and intervention factors necessary for mechanisms to ‘fire,’ we eventually succeeded in generating practical, intuitive and nuanced insights that fed directly into the design of a £17m follow-up programme (discussed further in

Punton, Vogel, Leavy, Michaelis, & Boydell, 2020).

• Mechanisms can travel and be applied elsewhere

Once you start thinking in 'mechanisms', you start to see them everywhere! The seven mechanisms we uncovered are continuing to inform our own and others’ work on institutional capacity change in a wide range of fields (see the

infographic).