Opening remarks delivered by Tom Parks at Monday’s JSRP / Asia Foundation research workshop.

This meeting marks the end of the first year of collaboration between The Asia Foundation (TAF) and the Justice and Security Research Programme (JSRP).  Together we’ve been investigating the empirical grounding and application of our Theories of Change within TAF’s country offices in Sri Lanka, Nepal, Phillippines and Timor-Leste, as part of a larger debate within the development community about Theory of Change and the role that evidence should play in shaping development policy.

The partnership has led to the publication of three papers so far, with eight more coming over the next few months. On the TAF side, we’re thrilled with the collaboration so far; it’s fantastic to have this level of academic engagement with the ideas and realities we work with in the field every day.

Within TAF, we’ve just finished a major programme on subnational conflict which asked: how much can aid plausibly affect conflicts that have been going on for four or five generations?  The truth is, development agencies have very little understanding of the long-term effect of this kind of work on conflict dynamics.  And we found very little evidence that development organizations understand their impact on conflict dynamics, particularly at the local level.

This lack of understanding around realistic baselines of evidence and impact has led to dramatic and widespread over-claiming on the effects that development work can have on conflict over time.

It’s a cycle that leads organizations like us to be more and more ambitious in the Theories of Change we develop for donors. Our research found that development programs tend to have vague and aspirational theories that link their program interventions – however small scale – to much larger processes of political change.  And then if we can’t provide tangible evidence that we’re having an impact on those lofty ToCs, or our indicators are hard to track, we find ourselves in a bind.

We see this collaboration with the JSRP as a way to address that larger problem.  There’s a growing push everywhere for development actors to show clear evidence of results. It started with DFID a few years ago, but has rippled out to AusAid, USAID, and the World Bank, all of whom are now much more serious about results and evidence.

This leads to a real conundrum for us at TAF.  Internally, we’re having discussions about hierarchies of evidence, what kind of evidence matters most, and the role of randomized control trials, metrics, and data.  The assumption among many donors is that these tools constitute “real evidence”, and others aren’t.

That assumption is threatening for TAF, because the the work we do can’t really be measured through those types of metrics.

So we recognize that this shift towards evidence claims is where the field is going, and that we need to get in front of the debate.  We want qualitative evidence and mixed method analysis to be part of the discussion.

Through this partnership, we have the possibility to shape the discourse of the evidence debate in the coming years, and create a broader view of what constitutes evidence.  We need to engage with this work and this debate, which allows us to unpack and qualify the kind of impact we really think we’re having.

There’s always going to be two groups of people within a development organization.  On one hand there the “true believers”: people who believe that development is a good thing as a truism.  The other side — most of us at TAF — are much more skeptical and critical.  We think there is an urgent need for organizations working on conflict issues to be more grounded and  humble in our claims of impact. Furthermore, there a serious problems when traditional development models are applied in conflict-affected areas that are just being recognized.  And we want to be on the cutting edge of addressing these challenges.

So, we’re looking forward to continuing this conversation and collaboration with the JSRP.

Thomas Parks is The Asia Foundation’s Regional Director for Conflict and Governance, based in Thailand. In this role, he shapes the strategic and intellectual direction for Foundation programs and research in the areas of subnational conflict, statebuilding and peacebuilding, state fragility, security and development in Asia.