In this blog post, Sebastian Rodriguez Perez, an Executive Master of Public Administration student, summarizes his work in encouraging East Londoners to vote for the Liberal Democratic party in the 2016 Mayor of London and London Assembly elections. He carried out a Randomised Evaluation Experiment which measured the effect of personal canvassing and leafleting appeals on voter turnout when a partisan message is conveyed.  

Do political campaigns make a difference?
To answer this question, I carried out a Randomised Evaluation Experiment (REE), which involved 1,600 registered voters. The REE was conducted on behalf of the Liberal Democrats during the 2016 Mayor of London & London Assembly elections. I worked as part of a volunteer group formed by supporters of Ciudadanos, the Liberal Democrats’ sister party in Spain.

Experiment Design
The original study was designed to measure the effect of personal canvassing and leafleting appeals on voter preferences. However, time and resource limitations forced a moderation of our goals. The aim of the REE was to measure the effect of personal canvassing and leafleting appeals on voter turnout when the message is conveyed in a partisan manner (see Table 1).

Table 1: Hypothesis formulation

This field experiment was conducted in two wards of the London’s Borough of Tower Hamlets: Limehouse and Poplar, which combined together, have a population of approximately 14,800. This was not a haphazard choice of wards as the selection process took several factors into account to ensure a representative sample. These factors were:

• relative size
• demographic and ethnic composition
• voting history in national and local elections

Having obtained a complete list of the registered voters in Limehouse and Poplar, we created a data set of all households. By selecting one individual per household, we were left with 1,598  voters whose participation in the 2016 Mayoral election could be determined from public records. Through a series of random assignments, the sample was divided into control and experimental groups (see Table 2). Overall, 662 people were assigned to intervention and 936 to control.

Table 2: Assignment of Persons to Treatment and Control Conditions

Note: The effectiveness of the randomization was validated using a Chi-square test method for independence (p > .10)

Personal Canvassing and Leafleting Procedure
Each weekend for five weeks in the run-up to the elections, a group of 10 canvassers was sent to contact randomly selected registered voters. The profile of the canvassers was very particular. They:

• were Spanish
• spoke fluent English
• declared themselves sympathizers of the Liberal Democrats
• belonged to Liberal Democrats’ sister party in Spain, Ciudadanos
• participated in this experiment voluntarily and were not paid

A volunteer group canvassing in Limehouse and Poplar

Personal canvassing consisted of a visit from the volunteer to a registered voter’s doorstep. Having identified him/herself, the canvasser delivered a leaflet with information about the candidate, conducted a pre-defined survey which included questions about local issues and voting intentions, amongst others; and encouraged the voter to vote for the Liberal Democrat candidate for London Mayor and City & East constituency, respectively Caroline Pidgeon and Elaine Bagshaw.

Training on effective canvassing was provided in the form of a set of instructions, which included best practices and the uninterrupted supervision of an experienced canvasser.

Leafleting on the other hand, was conducted without having direct contact with voters. These voters would receive the official campaign leaflet and the free-to-post-back survey in their mailboxes.

One important feature of the leafleting technique is that volunteers had to write down by hand the name and address of the target voter. In doing so, we intended to increase the likelihood of engaging with the  individual.

The group of canvassers successfully contacted with 100 (21%) out of the 469 voters in the personal canvassing treatment group; they also delivered 106 leaflets (55% of the total). These contact rates are in line with others observed across similar experiments.

Results
As shown in Table 3, the effect of personal canvassing on turnout was within the range of 0, i.e. no effect, and +2%; whilst leafleting’s  ranges from 0 to + 7% (see column C in Table 3).

Table 3: Summary of Turnout and Effectiveness Rate by Technique

Table 3 shows the effect of face to face contact and leafleting on turnout in more than one way:

• Section A includes the turnout rate registered in each of the original control (A.1) and treatment (A.2) groups.
• Section B compares the turnout rate of those who were contacted face-to-face or received the campaign materials (B.2) with those who were not contacted or did not receive the materials (B.1). The latter group, however, is a combination of the control group – whom we made no attempt to reach – and those in the treatment group whom the canvassers were unable to reach.

There are limitations to the approach used in Section B as it does not respect the original randomization assignment, and also overestimates the effect of each technique. The reason being is that voters who have been contacted are different from those whose houses could not be accessed by volunteers e.g. those living in council estates blocks with difficult access for canvassers may have different voting behaviour from those living in detached houses whose doors and mailbox are accessible from the street.

Despite the limitations of the approach used in Section B, it is used in the academic literature either as the only calculation (Eldersveld, 1956; Adams and Smith, 1980) or alongside other calculations which take into account the probability of voting among those easier to contact (Gerber and Green, 2000). In this experiment, we have used the figure resulting from Section B to set the upper bound of the Effectiveness Range, i.e. to set the maximum effect that could possibly be measured.

Conclusion
After conducting this randomized evaluation experiment in Limehouse and Poplar, I came to the following conclusions:

1. Registered voters showed similar voting behaviors regardless of the group they were assigned to. This unexpected result can be explained by:
   • The low contact rates achieved in this type of experiments: 21% and 55% for personal canvassing and leafleting, respectively
   • The fact that the canvassers were European citizens, did not live in the local neighborhood and were unaccompanied by the candidate
2. When canvassers are able to directly talk to the voters, this generates participation in the electoral process. In our experiment, turnout increased by 2% amongst those who were contacted in this way.
3. Leafleting also increased voter turnout up to 7% amongst those who received the campaign materials. This is partly due to the campaign materials addressed the voters in a personalized and a handwritten fashion, thus increasing the chance of successfully engaging the target individual.

Sebastian Rodriguez Perez is in his second year of the Executive Master of Public Administration. He would like to offer his thanks to the following:
Canvassers: Maria Aguilar, Eugenia Bilbao-Goyoaga, Lluis Sanchis, Guillermo Giralda, Miguel Timoner, Jose Otero, Nicolas Balcell, and Samuel Abeijon
Liberal Democrats: Elaine Bagshaw, Benjamin Sims, Stephen Clark, Mark Pack, and Tim Pollard

This post gives the views of the author, and not the position of the Institute of Public Affairs, nor of the London School of Economics.