Multilevel modelling of country effects:a cautionary tale

Country effects on outcomes for individuals are often analysed using multilevel (hierarchical) models applied to harmonised multi-country datasets such as ESS, EU-SILC, EVS, ISSP, and SHARE. We point out problems with the assessment of country effects that appear not to be widely appreciated, and develop our arguments using Monte-Carlo simulation analysis of multilevel linear and logit models. With large sample sizes of individuals within each country but only a small number of countries, analysts can reliably estimate individual-level effects but estimates of parameters summarising country effects are likely to be unreliable. Multilevel modelling methods are no panacea.