Computers, coders, and voters: comparing automated methods for estimating party positions

Finding reliable and valid positions for political actors is of key importance to political scientists. In this paper we compare estimates obtained using the automated Wordscores and Wordfish techniques with estimates from the widely-used Comparative Manifesto Project (CMP) as well as voter and expert placements. We estimate the positions of 254 manifestos across 33 elections in Germany and Denmark, two cases with very different textual data available. The paper contributes to the literature on automated content analysis by providing a comprehensive test of convergent validation, both in terms of number of cases analyzed and number of validation measures. In both cases, Wordscores approximately replicates the CMP, voter and expert assessments of party positions, whereas Wordfish replicates the positions in the German manifestos. The results demonstrate that automated methods can produce valid estimates of party positions, but also that the appropriateness of each method hinges on the quality of the textual data. Additional analyses suggest that Wordfish requires both longer texts and a more ideologically charged vocabulary in order to produce estimates comparable to Wordscores.