A new measure of issue polarization using k-means clustering: US trends 1988-2024 and predictors of polarization across the world.

Abstract

Political issue polarization worries scholars and the public alike. To understand what drives political issue polarization, longitudinal analyses and cross-national comparative research are necessary, but difficult to implement using current measures. We propose a new technique for measuring political issue polarization which is well suited to longitudinal and comparative analyses, using a machine learning algorithm called k-means clustering, which identifies coherent groups of politically-like-minded citizens from the bottom up. We analyse the between-cluster separation, within-cluster cohesion and size parity of the clusters to quantify a society’s political issue polarization. Using American National Election Studies data, we find that polarization increased in the USA from 1988 to 2024, driven by a period of rising separation between clusters from 2008 to 2020. Using World and European Values Survey data, we find that across the world, mass issue polarization is driven primarily by disagreement over cultural issues, but manifests differently depending on a society’s level of Human Development Index (HDI), with lower-HDI countries seeing culturally conservative clusters account for a majority of citizens, and higher-HDI countries having more culturally liberal and equally sized clusters. Different societal-level predictors, including ethnic fractionalization, wealth inequality and HDI, are associated with different aspects of polarization.