The holy grail of evidence has become all pervasive in the global HIV response. The drive for universalising knowledge on what works, derived from the emergence of ‘high quality’ evidence for biomedical prevention technologies, is now so entrenched in policy that sensitive appraisals of the ways in which interventions are engaged with and made to work in different contexts are being overlooked.

Nowhere is this more apparent than in debates around community mobilisation. Two studies which colleagues and I conducted in collaboration with the International HIV/AIDS Alliance fundamentally contest the desire to want to evaluate community mobilisation as if it is a simple technical component that either works or does not.

Working across 14 countries, our first study – a reflection on current practices in Linking Organisations of the International HIV/AIDS Alliance – found community mobilisation to be many things. Indeed, its strength is in its flexibility and adaptability. In Uganda, the Community Alliance for Health (CHAU) employs people living with HIV to link other positive people to health services, act as their medical advocate and provide adherence support. In Indonesia, Rumah Cemara supports people who are homeless and use drugs to take part as a team in the Homeless World Cup. In Latin America and the Caribbean, REDTRASEX brings sex workers together to fight for their human rights. With representatives on Colombia’s and Costa Rica’s Global Fund Country Coordinating Mechanisms, and regular consultation with UNAIDS and governments, they are making a significant impact.

In their different ways, each of these organisations is building the kind of social networks that help people to be healthy. Moreover, being so entrenched in these communities allows, or indeed forces, these organisations to understand, and respond to, local contexts and needs.

The findings of our second study, a systematic review of the evidence on community mobilisation’s efficacy, were inconclusive. This was particularly true of our review of general population studies although we were more positive about studies on most at risk groups, particularly sex workers. This is not surprising perhaps, given the diverse nature of community mobilisation – but often, what is being evaluated is, at best, a weak version. For instance, some studies labelled support groups as community mobilisation, a hugely different intervention from the diverse social networks being built by the organisations in our first study.

Crucially, a systematic review of the evidence cannot answer the crude “what works” question. It cannot tell us whether, if sex workers form an action committee, local police will follow the rule of law and local officials will allow them to access welfare. This appears to be a good strategy in parts of India and Latin America but in South Africa it has proved almost impossible to forge a sense of collectivity among sex workers who compete desperately for needed custom, and none of whom feel entitled to state benefits.

Although the evidence-based policy movement is founded on claims of the value of science, its use of evidence is often unscientific and unhelpful. For instance, in their book, Evidence Based Policy, Nancy Cartwright and James Hardie, deplore the tendency to assume that because “it worked there” it will “work here”. They argue that recourse to evidence cannot be used as a substitute for the hard work of thinking through what is needed to make something work in a particular context. In the medical field, Trisha Greenhalgh and colleagues, recently called for “real evidence-based medicine” which emphasises ethical treatment, individualised care and expert judgement rather than mechanical rule following. Analogies can easily be made for ‘real evidence based policy/practice’ for community health interventions with an emphasis on ethics, local responsiveness, and careful judgement of what might work and how, given the context. Anthropologist David Mosse, through his ethnographic studies of development programmes in India, has shown us that successful interventions are not simply ‘discovered’ but interventions are made successful through hard work and careful attention to how that work is presented.

Evidence of technologies’ effectiveness will not be sufficient to make a difference to the epidemic alone. Technologies are not just ‘rolled out’ – unfurled like a carpet to smoothly cover an affected area with a uniform surface. Technologies can be provided, or offered, but they have to be engaged with, appropriated, demanded, adopted, committed to, and made to work. A Randomised Control Trial can tell us that, under ideal conditions, PrEP can reduce HIV transmission by 50% or 90% but it can’t tell us how to achieve these ideal conditions so that people take their medicines consistently.

To make anti-retrovirals work, families have to source and cook nourishing food. Counsellors have to help people figure out how to fit the medicines into their everyday routines. CHWs, nurses and doctors have to be sufficiently motivated and paid to actually turn up to work.  Advocates have to keep up the pressure on hospitals and departments of health to make sure that the medicines are actually in stock. Managers have to take responsibility for the reputation of their clinic or hospital for trustworthiness, discretion and reliability. TV show writers and school teachers have to convey that being HIV positive and taking medicines is not something to be ashamed of or to mock. And so on.

Interventions do not just work automatically, they have to be made to work – and people have responsibility for making them work. And herein lies the rub – for the question we should ask is not “what works?” but “what are we committed to making work?” To be able to answer that we must be prepared for the hard work of gathering evidence in context, and mobilising the personal and political will to make interventions work against the backdrop of adverse circumstances and complicated lives.

Dr Flora Cornish is Associate Professor in the Department of Methodology at LSE.

This blog is a longer version of one previously posted by the International HIV/AIDS Alliance. It is adapted from my presentation at a panel debate held at the London School of Economics in October 2014.  Watch the podcast & video.