The existing literature is laden with contradictory hypotheses and findings that suggest this potential relationship can be positive or negative, stronger or weaker, differentiated or non-existent and can vary across and within countries and time periods. However, fundamental methodological and empirical limitations of analysis do not allow us to make such claims robustly, to some extent because the process of democratisation and changes in levels of equality are highly nuanced, idiosyncratic and heterogeneous and thus difficult to capture econometrically. Some of the most prominent authors in this literature claim that high levels of inequality decrease the likelihood of democratisation, and they also talk about “causal effects” and “the impact of democracy” on outcomes. Such conclusions presuppose a number of very demanding assumptions and requisite premises that cannot be rigorously met.
In fact, thousands of academic papers analyse the potential relationship between political variables like democracy and economic variables like inequality by gathering their data, selecting their methods and then going forward with their analysis, interpreting their findings and potentially informing policy, with many other steps along the way that involve making important implicit methodological assumptions. My recent paper1 instead goes backwards to analyse whether the data and methods that are applied by the leading authors in this literature are able to produce the robust results that they claim. It emphasises that how we as researchers generate our correlational (or ‘causal’) claims cannot be viewed independently from how we make everyday, typically unreflective decisions, such as what we decide to analyse, how we construct our variables, how we collect and use our data, which methods we choose to apply, how we interpret our statistical results, and so forth.
Better understanding the methodological and empirical limits of analysing the potential relationship between phenomena like inequality and political regimes is important for both research and policy because leading economists and other social scientists misguidedly claim to establish causal relationships but at times still inform public policy and thus can bring about adverse social outcomes.
Contrary to the existing literature, the paper argues that ‘causal mechanisms,’ or even a robust correlation, that may potentially link the distribution of economic wealth and different political regimes cannot be identified due to a number of critical scientific constraints. Some of the main methodological and empirical limitations that are outlined include aggregate macro-level analysis using a single empirical observation per country or per year; creating a uniform and meaningfully comparable measure of democracy; a multitude of non-measurable factors that may simultaneously influence the independent and dependent variables; different time lags in the potential effects of the influencing variables; important assumptions behind correlational claims derived from statistical analysis; and trying to make meaningful comparisons across and within countries over different time periods despite very different degrees and types of democracy and inequality as well as country-specific policies and tax structures.
It is important to stress that dynamic social phenomena like democracy do not have an “intrinsic nature” nor do they abide by “social laws,” and so the data and methods used to measure democracy do not allow us to say anything about causality. Using new data sources, analysing different time periods, or employing new data analysis techniques cannot resolve this question or provide robust, general conclusions about this potential relationship across countries. Because researchers are restricted to exploring rough correlations over specific time periods and geographic contexts with imperfect data, they need to be more critical and transparent in explicitly outlining the limitations of the data and methods they apply, and about the precision and interpretation of their results. The hope of this paper is to possibly be a useful warning for researchers against overly ambitious research aims and the overselling of their estimated results.
- 1. Krauss, Alexander. 2015. The scientific limits of understanding the (potential) relationship between complex social phenomena: the case of democracy and inequality. Journal of Economic Methodology
