Mashal Ijaz, BSc. Politics and International Relations ’21
Examples of data and measurement
There are two types of data: quantitative and qualitative. An example of quantitative data would be statistics (a summary of a variable for a set of units), usually shown in a table. Qualitative data can include documentary analysis, such as archival research or text analysis. Surveys can be used for qualitative evidence but can also be coded to provide quantitative data. Good examples include the European Social Survey, British Social Attitudes Survey, British Election Study, Eurobarometer and World Value Survey. Qualitative research can also be direct observation of participants in a designed experiment. Examples of measurements that are often used in political science include Freedom House ranking, Democracy Index, Polity project etc.
Challenges in measurement
There are several challenges with regards to measurement. Firstly, you can have random errors which introduce noise in your estimates i.e. it makes them less precise; these can occur, for example, if you input data incorrectly. On the other hand, systematic errors can occur when your estimator is pushing all our estimates either above or below the estimand. This is more problematic as it threatens the internal and external validity of the study, introducing bias and potentially resulting in incorrect conclusions.
Latent variables also pose problems in measurement. These are variable that are difficult to measure directly and include many of the most common outcomes of interest in social science. Examples include political knowledge, political efficacy or prejudice. We therefore have to record or score observable indicators assumed to be related to the latent outcome so as to infer characteristics of the latent variable.
Another measurement issue can occur when you are unable to measure your independent or dependent variable at the same level. An example would be if you are studying voter registration trials on social media; however, the treatment is assigned at the postcode sector level while the smallest level of measurement is the polling station.
Lastly, attrition is one of the most common problems with measurement. This occurs when units are missing from the sample, and is particularly harmful if they are missing from the outcome sample. If this occurs randomly, this might not be a large issue; however, if it is systematic, for example if people with higher values are more likely to have data missing than those with lower values, it can introduce bias. An example of attrition can be if subjects refuse to be interviewed after a study or you are unable to locate them because they have moved away. This can also harm the generalisability of your study, for example, if you find that women are most likely to attrite than men and therefore you are unable to generalise your results to the female population. A potential solution would be to keep track of subjects in the study. A better solution would be to make sure your intervention does not induce differential attrition. If your intervention makes it more likely that either the treatment or control group will attrite, this is likely to introduce bias. If the attrition is random, you can use a strategy called bounding whereby you make assumptions about the values that missing people would have taken on had they not been missing. However, this is only relevant if there is not a lot of missing data and ensure that you make your assumptions explicit and transparent.
Key definitions
- Estimator: the procedure we apply to data to generate a numerical result e.g. the linear regression estimator e.g. OLS estimator / difference in means
- Estimate: the numerical result arising from the application of our estimator to a specific set of data
- Estimand: the unobserved quantity we are trying to learn about with our data analysis or the ‘true’ correlation between the independent and dependent variables.
Image credit: https://learn.g2.com/hubfs/what-is-data.jpg
