Last week I participated in an expert meeting entitled “Sharing experiences and finding practical solutions regarding the implementation of the United Nations Guiding Principles’ effectiveness criteria in grievance mechanisms”. The event was organised by ACCESS Facility, a global non-profit organisation that supports rights-compatible, interest-based problem solving to prevent and resolve conflicts between companies and communities.
The meeting was extremely thought-provoking, and this post offers a few preliminary reflections on two recurring questions:
- Why is measuring the effectiveness of grievance mechanisms important?
- How can effectiveness be measured?
The relevance of the topic cannot be underestimated. Grievance mechanisms (GMs) are flourishing: according to a recent report from CSR Europe, 87% of its members already have a GM in place that deals with complaints coming from the workforce, and 40% have started addressing complaints from communities in a systematic way.
In addition, measuring the effectiveness of GMs is widely recognised as a fundamental ambition in the business and human rights field. In its 2013 report to the UN General Assembly, the UN Working Group on business and human rights stressed that “the development of performance indicators that can be used by stakeholders to encourage proper functioning of grievance mechanisms is important”. The report by CSR Europe confirmed that “setting out key performance indicators (KPIs) to measure the performance of grievance mechanisms is what all companies point to as a current challenge”.
Against this background, it is surprising that practical guidance is almost non-existent. David Vermijs recently commented that “both the policy debate and practice would be highly enriched if there were clear and tested metrics for measuring effectiveness of grievance mechanisms. Unfortunately, … the literature is limited”.
Why is measuring the effectiveness of GMs important?
The production of valid measures of GMs’ effectiveness is consequential for several reasons. At the corporate level, managers need indicators in order to track and improve performance. As the adage goes, “what gets measured, gets managed”. A valid measurement framework is necessary to detect what works and what does not.
KPIs can also be used to incentivize implementation by staff members. For instance, companies could link complaints-related metrics to personal performance indicators and assess senior managers on the basis of their cooperation in gathering information and their ability to meet internal deadlines.
For external stakeholders, valid measures of effectiveness are important to overcome a diffuse lack of confidence. GMs are criticized for being biased (companies are often involved in their administration) or futile (they have little enforcement powers). Robust evidence of their effectiveness can help modify this opinion.
In addition, NGOs and trade unions show increasing frustration because of their inability to make informed choices between different GMs. Valid metrics can be used to guide strategic decisions. Which GMs are most likely to produce a satisfactory outcome?
How to measure effectiveness
According to the United Nations Guiding Principles on business and human rights (UNGPs), GMs should meet eight criteria in order to be effective. They should be legitimate, accessible, predictable, equitable, transparent, rights-compatible, a source of continuous learning and (in particular operational-level mechanisms) based on engagement and dialogue with stakeholders.
These criteria have proved incredibly useful to draw the boundaries of the debate on what “good” GMs look like. Yet, they are still rather generic. For instance, legitimacy is defined as “enabling trust from the stakeholder groups for whose use they are intended, and being accountable for the fair conduct of grievance processes”. How can one measure levels of trust? How to assess fairness of conduct?
The landscape of existing KPIs on GMs is varied: total number of grievances, number of grievances pursued through other (judicial and non-judicial) mechanisms, number of solved grievances, hours of staff training on GMs, level of awareness of the mechanism among intended users, number of access points, etc.
All these indicators offer invaluable evidence of short- and long-term trends. Yet, as highlighted by IPIECA, “caution is needed when interpreting data on complaints”. For instance, a reduction in the number of grievances is good news only if one can show that the trend is due to stakeholders having fewer grievances and not due to a lack of trust in, or access to, the mechanism. Similarly, the number of “solved” grievances risk being invalid because employees often provide false attestations when in the presence of management.
How to proceed? I distilled three general lessons from the discussion at the expert meeting.
First, more guidance is needed on how to operationalise effectiveness.
The abovementioned report by CSR Europe represents one of the first attempts to publicly clarify what the UNGPs criteria practically mean in a business context. The result is the Management of Complaints Assessment (MOC-A) tool, a promising set of 21 concrete process requirements derived from the UN document.
Martijn Scheltema followed a different path and argued against the over-reliance on procedural features. Assessment of effectiveness should focus not only on process, but also on ‘outcomes’ (such as enforceability and rights-compatibility).
Second, general KPIs and process requirements (such as those offered by CSR Europe) are useful only as a starting point in view of more locally-owned and fine-grained measurement frameworks.
GMs vary widely in terms of location, procedure and institutional set-up: OECD National Contact Points are different from hotlines; the Compliance Advisor Ombudsman (CAO) of the International Finance Corporation has almost nothing in common with Ghana’s Commission for Human Rights and Justice. In addition, GMs need to respect traditional cultures and adapt to local contexts. For instance, some individuals may prefer anonymous hotlines than direct contacts with in-line managers.
As highlighted by the CAO, “creating and implementing effective grievance mechanisms is not an exact science”. Verité agrees that “the design of grievance mechanisms is a tailored process that depends importantly on many factors”. There is no reason to use the same set of metrics for different mechanisms and environments.
Third, Emma Wilson and Emma Blackmore rightly argued that the true test of effectiveness is in the eyes of intended users.
Most of the criteria listed in the UNGPs are inherently subjective. There is no legitimate or accessible GMs in general terms. Legitimacy and accessibility are concepts that make sense only in relation to specific individuals and communities. Something which is legitimate for Indian workers may not be legitimate for an Amazonian community.
If this is the case, measuring the effectiveness of GMs cannot but be based on regular engagement with potential victims at the local level. As stressed by Caroline Rees, “feedback from the intended user groups [is] extremely valuable in interpreting numerical indicators and need to include both individuals who had used the mechanism and some who had not, including groups or populations who might feel excluded, such as indigenous peoples and women”.
Actually, affected stakeholders should be involved not only during the measurement process. Deanna Kemp and Nora Gotzmann are right to argue that a “decide together” approach should be used to decide which KPIs and measurement framework to use. An advanced step in this respect is suggested by the International Council on Mining and Metals: the creation of a multi-stakeholder grievance advisory committee – that is, an oversight body composed of company, workers and community representatives who monitor performance and provide strategic advice about the GM.
Conclusion
Research by SOMO on the electronics industry shows that there is still much to do to improve the effectiveness of existing GMs.
A few companies are taking the lead in measuring the effectiveness of their mechanisms. For instance, Barbara Linder, Karin Lukas, Astrid Steinkellner report that Anglo American started “measuring the effectiveness of the group-wide mechanism on the basis of key performance indicators … in 2011”.
This is good news. As seen above, valid measures of the effectiveness of GMs are fundamental to convince those who are sceptical about non-judicial remedies. The risk is that the very idea of GMs will be written off because so many of those that exist are not trusted.
In an article in the American Political Science Review, Christopher Blattman, Alexandra C. Hartman and Robert A. Blair showed that Liberian towns receiving training in alternative dispute resolution mechanisms had higher resolution of land disputes and lower violence. Scholars and non-profit organisations (such as ACCESS Facility and SOMO, which runs a specific Human Rights & Grievance Mechanisms Programme) should consider producing similar research on corporate-related GMs.
This is important to know that GMs work. This is important to know which GMs work better.
Damiano de Felice
Damiano de Felice is one of the co-Directors of MB&HR, a PhD Student at the London School of Economics, and a Board Member of the European Society of International Law Interest Group on business and human rights. In the past year, Damiano delivered a TEDx talk on the corporate responsibility to respect human rights, conducted the baseline study in view of the adoption of the Italian National Action Plan on business and human rights, and published an article on business and human rights indicators on Human Rights Quarterly.
