How to measure a crisis?
Remarks on the problem of measurement in the science on democracy
Abstract
In this paper we investigate the problem of a measurement of democratic backsliding through rankings. We indicate various theoretical and practical problems of measurement and conceptualisation of the phenomenon of democracy. We argue that democratic backsliding should be viewed as an opportunity to verify if rankings are an accurate method of measuring democracy. We are concerned particularly with the case of Poland, as the situation in that country in recent years has been universally described by experts and popular press as an example of the crisis of democracy. Having analysed popular democracy rankings (Democracy Index – The Economist Intelligence Unit, Global State of Democracy – International Institute for Democracy and Electoral Assistance, Liberal Democracy Index – Varietes of Democracy Institute, Democracy Matrix – University of Würzburg), we argue that they produce the expected results – consistently indicating the deterioration of Polish democracy in the period of 2015-2023.
Downloads
References
R. Adcock, D. Collier, “Measurement validity: A shared standard for qualitative and quantitative research,” American Political Science Review 95, no. 3 (2001): 529–546.
E. Bakke, N. Sitter, “The EU’s enfants terribles: Democratic backsliding in central Europe since 2010,” Perspectives on Politics 20, no. 1 (2022): 22-37.
N. Bermeo, “On Democratic Backsliding,” Journal of Democracy 27, no. 1 (2016): 5-19, https://doi.org/10.1353/jod.2016.0012.
K.A. Bollen, P. Paxton, “Subjective measures of liberal democracy,” Comparative Political Studies 33, no. 1 (2000): 58-86.
H. Chang, Inventing temperature: Measurement and scientific progress (Oxford University Press, 2004).
D. Chakrabarty, “Postcoloniality and the Artifice of History: Who Speaks for “Indian” Pasts?”, Representations 37, Special Issue: Imperial Fantasies and Postcolonial Histories, http://www.jstor.org/stable/2928652.
D. Collier, S. Levitsky, “Democracy with Adjectives: Conceptual Innovation in Comparative Research,” World Politics 49, no. 3 (1997): 430-451, https://doi.org/10.1353/wp.1997.0009.
M. Coppedge, J. Gerring, C.H. Knutsen, S.I. Lindberg, J. Teorell, D. Altman, M. Bernhard, “Conceptualizing and measuring democracy: A new approach,” Perspectives on Politics 9, no. 2 (2011): 247-267.
P. Craig, S.V. Katikireddi, A. Leyland, F. Popham, “Natural experiments: An overview of methods, approaches, and contributions to public health intervention research,” Annual Review of Public Health 38 (2017): 39-56.
R.A. Dahl, Polyarchy: Participation and Opposition (New Haven: Yale University Press, 1971).
R.A. Dahl, Democracy and its Critics (New Haven: Yale University Press, 1989).
S. Dahlberg, U. Mörkenstam, “Exploring popular conceptions of democracy through media discourse: analysing dimensions of democracy from online media data in 93 countries using a distributional semantic model,” Democratization (2024): 1-32, https://doi.org/10.1080/13510347.2024.2342485.
T.G. Daly, “Illiberal Democracy: Time to Stop Using a Problematic Term?,” Percorsi costituzionali, no. 2: Ossimoroni costituzionali. Constitutional Oxymorons (Naples: Jovene editore, 2020).
L. Diamond, “Facing Up to the Democratic Recession,” Journal of Democracy 26, no. 1 (2015): 141–155.
R.M. Dworkin, “The Model of Rules,” University of Chicago Law Review 35, iss. 1, article 3 (1967).
D. Giannone, “Political and ideological aspects in the measurement of democracy: The Freedom House case,” Democratization 17, no. 1 (2010): 68-97.
S.D. Hyde, “Democracy’s backsliding in the international environment,” Science 369 (2020): 1192–1196, https://doi.org/10.1126/science.abb2434.
L. Kaftan, “Lip service to liberal democracy in Western Europe?,” European Political Science Review 16, no. 4 (2024), https://doi.org/10.1017/S1755773924000079.
A. Kendall-Taylor, N. Lindstaedt, E. Frantz, Democracies and Authoritarian Regimes (Oxford: Oxford University Press, 2019).
M. Klinowski, R. Smoleń, “The march of the brutes. Remarks on agential causes of the crisis of liberal democracy in Poland,” De Legibus – Revista de Direito da Universidade Lusófona Lisboa 4 (2024): 119-140, https://doi.org/10.60543/dlb.vi7.9680.
C.H. Knutsen, K.L. Marquardt, B. Seim, M. Coppedge, A.B. Edgell, J. Medzihorsky, D. Pemstein, J. Teorell, J. Gerring, S.I. Lindberg, “Conceptual and Measurement Issues in Assessing Democratic Backsliding,” PS: Political Science & Politics 57, no. 2 (2024), https://doi.org/10.1017/S104909652300077X.
A. Kozłowski, K. Skelnik, “Disinformation as a Tool Aimed at Weakening Consolidated Democracies,” Public Governance. Administration and Finances Law Review 5, no. 1 (2020): 73-85, https://doi.org/10.53116/pgaflr.2020.1.5.
I. Krastev, S. Holmes, The Light that Failed: A Reckoning (Penguin Books, 2020).
K. Lenaerts, “The principle of democracy in the case law of the European Union Court of Justice,” International and Comparative Law Quarterly 62, no. 2 (2013): 271-315, https://doi.org/10.1017/S0020589313000080.
A.T. Little, A. Meng, “Measuring Democratic Backsliding,” PS: Political Science & Politics 57, no. 2 (2024), https://doi.org/10.1017/S104909652300063X.
A. Lührmann, S.I. Lindberg, “A third wave of autocratization is here: what is new about it?,” Democratization 26, no. 7 (2019): 1095-1113.
R. Markowski, Demokracja i demokratyczne innowacje. Z teorią w praktykę (Warszawa: Instytut Obywatelski, 2014), https://instytutobywatelski.pl/pliki/pdf/demokracja_calosc_www.pdf.
M.G. Marshall, T.R. Gurr, K. Jaggers, Polity IV Project: Political regime characteristics and transitions, 1800-2018 (Center for Systemic Peace, 2019).
G.E. Moore, Principia ethica (Mineola, N.Y.: Dover Publications, 1903).
G.L. Munck, J. Verkuilen, “Conceptualizing and measuring democracy: Evaluating alternative indices,” Comparative Political Studies, 35, no. 1 (2002): 5-34.
P. Norris, R. Inglehart, Cultural Backlash. Trump, Brexit, and Authoritarian Populism (Cambridge: Cambridge University Press, 2019).
A. Polyakova, T. Taussig, T. Reinert, K. Kirişci, A. Sloat, J. Kirchick, M. Hooper, N. Eisen, A. Kenealy, The anatomy of illiberal states: Assessing and responding to democratic decline in Turkey and Central Europe (Washington: The Brookings Institution, 2019), https://www.brookings.edu/research/the-anatomy-of-illiberal-states, accessed January 3, 2025.
K. Popper, The Logic of Scientific Discovery (London: Hutchinson, 1959).
A. Przeworski, “Who Decides What Is Democratic,” Journal of Democracy 35, no. 3 (2024).
J. Reykowski, Rozczarowanie demokracją. Perspektywa psychologiczna (Sopot: Smak Słowa, 2019) (published originally in English: Disenchantment with democracy, Oxford University Press, 2020).
W. Sadurski, “How Democracy Dies (in Poland): A Case Study of Anti-Constitutional Populist Backsliding,” Sydney Law School Research Paper 18/01 (2018), http://dx.doi.org/10.2139/ssrn.3103491.
G. Sartori, The theory of democracy revisited, vol. 2: The classical issues (Chatham: Chatham House Publishers, 1987).
M. Saward, Democracy (Cambridge: Polity Press, 2003).
S.-E. Skaaning, “Waves of autocratization and democratization: a critical note on conceptualization and measurement,” Democratization 27, no. 8 (2020): 1533-1542, https://doi.org/10.1080/13510347.2020.1799194.
E. Tal, “Measurement in science,” in The Stanford Encyclopedia of Philosophy, ed. E.N. Zalta (Stanford: Stanford University, fall 2020).
J. Teorell, M. Coppedge, S. Lindberg, S.-E. Skaaning, “Measuring Polyarchy Across the Globe, 1900–2017,” Studies in Comparative International Development 54, no. 1 (2019): 71-95.
Ch. Tilly, “Inequality, Democratization, and De-Democratization,” Sociological Theory 21, no. 1 (2003): 37-43.
D. Treisman, “What have we learned about the causes of corruption from ten years of cross-national empirical research?”, Annual Review of Political Science 10 (2007): 211-244.
D. Waldner, E. Lust, “Unwelcome change: Coming to terms with democratic backsliding,” Annual Review of Political Science 21 (2018): 93-113.
F. Wolkenstein, “What is democratic backsliding?,” Constellations 30 (2022): 261-275, https://doi.org/10.1111/1467-8675.12627.
The Treaty of Lisbon (Official Journal of the European Union, C 306, 17 December 2007).
Interpellation No 15878 to the Minister for Development on the decrease of Poland’s place in the World Economic Forum ranking, https://www.sejm.gov.pl/Sejm8.nsf/InterpelacjaTresc.xsp?key=509908BF, accessed March 7, 2025.
Judgement of 24 November 2010 of the (Polish) Constitutional Tribunal on the compliance of the Lisbon Treaty with the 1997 Constitution of the Republic of Poland (K 32/09).
Judgment of 17 January 2023 of the Grand Chamber of the European Court of Human Rights in Strasbourg in the case of Fedotova and Others v. Russia, https://hudoc.echr.coe.int/fre?i=001-222750.
Office of the High Commissioner for Human Rights, Further measures to promote and consolidate democracy, Commission on Human Rights resolution 2002/46, https://www.ohchr.org/en/about-democracy-and-human-rights, accessed March 8, 2025.
Rankings of democracy and other materials prepared by the Economist Intelligence Unit, the International Institute for Democracy and Electoral Assistance, the Varietes of Democracy Institute, the Democracy Matrix Research Project (rankings’ results, reports comprising of rankings’ results and discussion, information on rankings’ methodology), available at the relevant websites: https://www.eiu.com/n/global-themes/democracy-index, https://www.idea.int/gsod, https://www.v-dem.net, https://www.democracymatrix.com.
Latana, Alliance of Democracies, Democracy Perception Index for 2022 and 2024 – survey’s results and discussion, available at https://staging.latana.com/democracy-perception-index-report-2022 and https://www.allianceofdemocracies.org/wp-content/uploads/2024/05/DPI-2024.pdf.
European Social Survey’s results: www.europeansocialsurvey.org
Copyright (c) 2025 De Legibus - Law Journal
This work is licensed under a Creative Commons Attribution 4.0 International License.
Free and open access to the editorial content is guaranteed for all readers, and its full or partial reproduction is permitted under the CC BY 4.0 (Creative Commons Atribuição 4.0 Internacional) license.
