Do scholars produce and reproduce a biased representation of the academy when compiling their taught course reading lists? Following a year-long mapping exercise of the university’s entire international relations curriculum by a group of PhD students at the LSE, Gustav Meibauer, Kiran Phull and Gokhan Ciflikli found that male authors continue to significantly outnumber their female counterparts, with little discernible progress over the last two decades. Visualisations of the research data have been made available online, and the authors invite the academic community to contribute to discussions on their project, its implications, and the disciplinary and pedagogical questions it provokes.
Following scholarly interest in gender in the international relations (IR) discipline (e.g. Maliniak et al, 2013; Colgan, 2017, Colgan, 2016; Teele and Thelen, 2017) as well as increasing concerns voiced by student bodies (e.g. the “Why is My Curriculum White?” YouTube video; the “Decolonising SOAS” campaign), a group of PhD students at the Department of International Relations at the London School of Economics and Political Science (LSE) embarked on a year-long mapping exercise of the 2015/16 IR curriculum. Existing approaches to dissecting IR syllabi often lack in large-enough datasets or employ very different methodologies between them, both of which limit comparability and replicability as well as inhibiting broader disciplinary debates. They also often speak to important stakeholders, especially students, only indirectly. For these reasons, we decided to visualise our data for easy access on a dedicated website – https://gokhan.shinyapps.io/gender/ – and we would like to invite the academic community to have a look, play around with the data, and join us in conversation on the project, its implications, and the disciplinary and pedagogical questions it allows us to conceptualise and analyse.
The website is based on an export of Moodle data containing syllabi for each BA, MA, and PhD-level IR course on offer at the LSE’s IR department during the academic year 2015-16, and provides, as far as we can tell, the largest sample to date for this type of pedagogical study. A total of 43 courses (18 undergraduate, 23 postgraduate, and 2 PhD-level) render 12,358 non-unique textual sources listed as both essential and background reading material. These include journal articles, books and book chapters, reports, treaties and other documents, webpages, and more. To tackle questions of gender bias and female inclusion as it relates to authorship, we code (male=M/female=F) for sex of author(s). Of our readings, 2,574 (20.8%) have at least one female author. Conversely, male-only contributions – i.e. those written by one or more men without female co-authors – make up 9,784 items (79.1%) of the reading lists. This means that, excluding co-ed teams, the female to male inclusion ratio is 1:5.64 (for every 1 female included, 5.64 males are included).
Since we have course-level information, we can disaggregate the overall dataset into different subsets. The IR department is organised into research clusters; i.e. issue areas or disciplinary subfields in which its faculty has specific expertise. We break down the 43 courses into five such subfields (statecraft/security studies, area/regional studies, international political economy, IR theory, and international organisations/law). Some subfields clearly do better than others when it comes to including female contributors on their reading lists, as indicated by the different percentages. Around half of all the courses have less than 20%:80% female-to-male author ratio. In the below graph, we also indicate the level of the course (UG, MA, PhD).
Figure 1: Female inclusion ratios ordered per sub-field, courses in the sub-fields, and level of courses offered. Click to enlarge.
Contrary to past endeavours that have focused solely on either journal articles, or doctoral-level courses, or core IR courses, our approach captures the full spectrum of the taught discipline, drawing data from the entire 2015-2016 curriculum at the LSE IR department. Our data speaks primarily to taught or instructed IR; that is, we analyse the state of the discipline as it exists in syllabi across LSE IR courses. Through analysing the taught discipline as it appears in reading lists composed of published books, journal articles, blogs, and more, we can also uncover some (more limited) insights into the published discipline (also: Evans and Moulder, 2011; Mitchell, Lange and Brus, 2013).
For example, we have coded for author sequence, and for multiple author incidences (co-authorships). This allows us to illustrate and think about co-authorship patterns as they appear in our reading lists. In our data, male authors get to be first author in co-ed publication teams 57% of the time. First position of authorship in a publication is usually less significant in the social sciences than in the natural sciences. Oftentimes, author order is assigned alphabetically rather than by rank of the respective co-author or importance to the project. Still, as we can’t assume men have last names that are systematically higher in the alphabet than women, something else must be going on to explain this phenomenon. Does this pattern come down to course convener selection? Or does it tell us something about bias in authorship decisions more generally?
Figure 2: An illustration of co-authored works featuring up to three authors; author order preserved. Click to enlarge.
As a key benefit of using Moodle-exported syllabi, all reading list data arrive in a standardised format and with additional classification variables, such as dates of publication, type of textual source (book or article), and publication information (specifically the name of the press or journal to publish each item). Combining this information with the gender breakdown and co-authorship patterns allows us to map “publication pathways” for female authors (at least as reflected in our reading lists). We can disaggregate items written by female authors by their publication characteristics in the last decades (the “2010s” only include data until 2015). Unsurprisingly, given the gender imbalance, female-female collaborations are the least likely occurrences.
Figure 3: Characteristics of reading list items written by female scholars. Click to enlarge.
In absolute terms, we see a steady increase in the number of female authors included in the IR reading lists the more recent the date of publication. That is, a book from 2010 on our reading lists is more likely to be from a female author than one from 1990. However, works by male authors display a similar pattern, as reading lists are modernised throughout the years and newer items replace older ones more generally. While we cannot compare across reading lists from different years (see below), relative inclusion patterns by date of publication remain steadily low.
Figure 4: Number of publications included in LSE IR reading lists per publication year of the item. Click to enlarge.
A few words on two obvious limitations: one, the dataset captures a snapshot of the IR curriculum in one year alone. However, given that the lifecycle of a standard course syllabus can extend for a long duration and that many of the courses included in the curriculum are core disciplinary courses and/or have been offered to students for years, it could be hypothesised this dataset closely resembles the curriculum offered for about two to five years either side of the 2015-16 academic year. Also, our dataset is limited to one institution. However, the LSE is seen (both by itself, as well as within the discipline of IR) as a cutting-edge, research-driven university that seeks to be (and frequently has been and continues to be) at the vanguard of both research and education in the discipline. It might therefore be interesting prima facie as an exemplar into what an up-to-date reading list in the discipline may look like. The LSE IR department is also, importantly, located firmly within European traditions of historical, sociological, and often qualitative inquiry into the discipline’s core questions, which makes for interesting comparisons with its American or continental counterparts.
We believe the data allow for a wide range of interesting observations and insights into the discipline (some of which we outline and try to explain in further detail in articles currently under review), invite us to think critically about how we as scholars produce and reproduce a possibly deeply biased representation of the discipline in our syllabi, and finally consider what policies the academic community as a whole, the LSE as a major institution, and we as individual teachers and researchers should pursue to further improve the inclusion and representation of female scholars in the taught discipline. This is a conversation we hope we can contribute to with this project, the blog post, and our website. We also hope it may serve as a benchmark for studies in other departments and universities.
Update, 8 October 2018: the authors have recently published this research in the European Journal of International Relations – “Gender and bias in the International Relations curriculum: Insights from reading lists” (DOI: 10.1177/1354066118791690).
Visualisations of the research data can be seen at https://gokhan.shinyapps.io/gender/
Featured image credit: Panel by Alper Çuğun (licensed under a CC BY 2.0 license).
Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
About the authors
Gustav Meibauer is a postdoctoral fellow at the LSE’s Department of International Relations. His research focuses on US foreign policy, coercive diplomacy and intervention. His ORCID iD is 0000-0002-5574-7270.
Kiran Phull is a PhD student at the LSE’s Department of International Relations. Her research examines the development and presence of a public opinion and polling industry in the Arab world.
Gokhan Ciflikli is a PhD student at the LSE’s Department of International Relations. He undertakes various computational social science projects at the LSE, and specialises in data science, predictive modelling, and data visualisation.
We’ve been having the same discussion on International Development – see blog on http://oxfamblogs.org/fp2p/the-perils-of-male-bias-alice-evans-replies-to-yesterdays-sausagefest/ and this Google Doc where lecturers/students are uploading their course breakdowns https://docs.google.com/document/d/1fxDk3Qn6LCZo118YNE11n0G84KMsRVUq4sSllx63LHw/edit
What is the distribution of male and female authors in the general literature, and how does that compare to the reading lists?
In terms of publications, Teele & Thelen (2017) find F/M ratio to be around 26% in ten prominent political science journals (n of ~6000), which we use as a natural benchmark. Given the sample sizes, the difference between that and the syllabi ratio (n ~13000, 20%) comes out as statistically significant at conventional levels (95%).
Thank your for the information.
Presumably, reading lists not random samples of all the literature out there. The papers and books are chosen by instructors because of their importance (i.e, impact) to the respective fields.
Is there some way of combining that sex ratio (26%) with some measure of impact?
How would THAT measure compare with the sex ratio of reading lists?
We don’t have any indicators for impact, no. However, impact alone is a biased metric as female authors cite themselves less and men cite other men more:
However, our data is available online so feel free to replicate our findings with a variable capturing impact and let us know the results!
So, the reading lists are not chosen at random from the literature, but rather they just happen to be limited to the most important books and papers ever written. Is that not what these reading lists are supposed to do? The null hypothesis stands firm.