In a standard class on research methods, you will learn about biases that the researcher can introduce into the research. Researchers, we are taught, sometimes unconsciously influence respondents to give answers that make for results that are convenient to them. At the same time, respondents may give different answers to a question depending on the age, gender, ethnicity or nationality etc. of the researcher.

During my studies, I took notice of the fact that these ‘research biases’ exist, but was never too bothered about them. I happily let loose my econometric methods on data gathered by others. Other researchers did not seem bothered either; the closest they came to acknowledging with these biases was by assuring the reader, for example, that sensitive questions (e.g. about sexuality) were asked to women by women and to men by men.

Some of my colleagues, in anthropology or political science,  use different research methods than the ones I have been trained in.  They told me that they planned to study both “the environment that they are in and themselves as a researcher in that environment”. That idea seemed rather ‘out there’ to me.

Seeing bias in action: South Sudan

Earlier this year, I went to South Sudan as part of a team from the Justice and Security Research Programme. My main task was to lead a household survey on a number of topics, including how much people trust various local leaders.

The dice-throw of our random sampling sent us to one village where we met with a ‘Boma Chief’, a leader at the village level considered to be a ‘traditional leader’. During our first visit, during which we did little apart from introducing ourselves and explaining our purpose, this Boma Chief acted unlike any of the other chiefs we had encountered.  He did not take the lead in the conversation and remained seated in the background. From this interaction and later conversations, we got the sense that this particular Boma Chief was not very well-liked by ‘his’ community.

As the next day was Sunday, we spent it in a larger town some miles away. The Boma Chief we met earlier turned up in town too, and, knowing that we were returning to ‘his’ village on Monday, came asking for a ride. As we had an interest in him being there too, and did not see a reason for him to ride his bike all the way while we were driving anyway, we took him along. We spent three more days in this village gathering our survey data and left with the job done.

Of course we were aware of the risks of appearing to be too closely aligned with any leader, especially if we were asking people questions about the degree of trust in local leaders afterwards. But it is impossible to avoid all such association; doing research in a village without the local leader’s permission would be a serious mistake. In my mind, I had already written a disclaimer for our report, stating that the degree of trust in local leaders was likely biased upwards because of this.

It turned out we had an impact on the situation in this village rather more directly.  Some time and two villages later, I met with the Boma Chief again, this time in the county capital. He sounded angry.

Through a translation, I understood that rumours had surfaced in ‘his’ village that we had given him money intended for the community, but that he had kept it for himself. Already mistrustful of the Boma Chief, the community took the fact that he had spent a Sunday in the same town with us, and had arrived in our car, as evidence of this. In reality, we had never given him anything of any kind. Nevertheless the Boma Chief said he was afraid to go back to ‘his’ village, fearing community members would turn violent.

We tried to ‘fix’ the situation as best we could; we gave the Boma Chief an official-looking letter stating we had never given him anything, and sent a contact in the area back to the village to explain this in person. Some weeks later, our contact reported that things had calmed down.

Our initial concerns that our behaviour might bias our data seem quite trivial now, seeing how our behaviour changed (or at least catalysed changes in) the reality we were studying so directly.

One of the questions in our survey was: “How often do you trust the Boma Chief?” The answers that we got suggest that people in this village trust their Boma Chief significantly less than the ‘average village’  (although the same can be said about some of the other villages we visited). However, I can imagine a carbon-copy of our research team revisiting this village a week after our original visit, and finding trust in the Boma Chief to be even lower compared to the data our actual research team collected.

This blogpost does not offer a better solution to ‘researcher bias’ than that of the standard methods class: that it is important to be aware of possible biases in the data. But I cannot say things would have gone differently, had I been more ‘aware’ (whatever that means in practice) in the field . Neither do I mean to devalue the worth of all data gathering exercises, let alone our own.

To me, this experience instead shows that biases are unavoidable and come at us from unexpected directions — so it’s important to live with and be open about them with each other.

Anouk S. Rigterink is a PhD candidate at the London School of Economics. Her research interests include the relationship between natural resources and violence and how violent conflict is quantified.