Which indicators are most useful for comparing citizenship policies?
- Which indicators are most useful for comparing citizenship policies?
- David Reichel: We need different indicators for different research questions.
- Maarten Vink: Naturalisation rates and rejection rates measure different phenomena, and have different problems
- Jan Willem Duyvendak, Rogier van Reekum, Peter Scholten, Christophe Bertossi: The of/for distinction
- Thomas Janoski: What we need citizenship indicators for depends on who are “we”
- Thomas Huddleston: From politics to impact: How citizenship really works
- Anita Manatschal: On the relevance of comprehensive comparative analyses at the subnational level
- Concluding remarks by Marc Helbling
- All Pages
Kickoff contribution by Marc Helbling
September 24, 2010
Over the last decade there has been an important new development in the citizenship literature. After a long period of case study research and small-N comparisons, various scholars and projects have started to compare a relatively large range of cases (Waldrauch and Hofinger 1997; Koopmans et al. 2005, 2010; MPG 2006; Helbling 2008; Howard 2009; Janoski 2010). This new research approach has inevitably led to a quantification of the data under study, however, not yet to the creation of a widely accepted citizenship indicator. On the contrary, there are almost as many indices as there are large-N studies.
The emerging pioneer spirit has left surprisingly little space to the question of how useful these indicators are to compare citizenship policies. Do we need so many indicators? Do they all measure the same or different concepts? Do they really measure what they are supposed to measure? While hardly anybody will disagree that we need a more thorough discussion on these questions, it is all the more astonishing that such a debate has been absent so far.
Besides the more general claim that we urgently need such a debate, I like to make three further points in this contribution: (1) There is evidence that most of the existing indicators measure the same concepts, irrespective of how sophisticated the indicators are. It would therefore be more efficient to stick to simple indicators and avoid projects that require a huge amount of resources for the data collection. (2) Moreover, while some indicators combine information on policy aspects with naturalisation rates, keeping apart these two aspects would allow us to make a difference between the output and the outcome of citizenship politics. Isolating policy outcomes is particularly important to study policy effects. (3) While policy outcome studies most of-ten use naturalisation rates and define these as the ratio between the yearly number of naturalisations and the number of foreign residents in a country, I claim that we should focus on rejection rates, i.e. the ratio between the rejected and the submitted applications as a more valid way to study policy effects.
Let us first see whether all indicators that focus on policy aspects measure the same concept (Waldrauch and Hofinger 1997; Koopmans et al. 2005, 2010; MPG 2006; Howard 2009). Both Howard (2009: 32-35) and Koopmans et al. (2010: 12-13) use correlation tests to compare their own indicators with each other and in addition make a comparison with the Migration Integration Policy Indices (MIPEX) (MPG 2006). Koopmans et al. additionally compare their indicator with the Legal Obstacles to Integration-Index (LOI, Waldrauch and Hofinger 1997). Since all but Howard’s indicator also include information on integration regulations, only the components that are directly related to naturalisation regulations are accounted for in the two tests. By doing so, Howard (2009) and Koopmans et al. (2010) run reliability tests and assess the consistency of their codings. Although they run slightly different correlation tests, both come to the conclusion that all indicators are highly related to each other and thus seem to measure the same concept in a consistent way.
In addition to these authors’ calculations, I have run correlation and factor analysis tests of the overall indicators including all components and thus have analysed their convergent validity. I found that even if additional aspects concerning integration are included the four indices correlate at a relatively high level between 0.7 and 0.9 and load highly on the same factor. Such a finding is all the more astonishing as the four studies created their indicators in different ways that required different amounts of resources. Howard’s Citizenship Policy Indicator (CPI), which is based on data from the NATAC project (The Acquisition of Nationality in EU Member States: Rules, Practices and Quantitative Developments) and integrates three elements (ius soli, naturalisation requirements, dual citizenship), is certainly the most straightforward indicator as it is based on information that is clearly specified in national laws.
Waldrauch and Hofinger (1997) included almost 80 items in their index. Koopmans et al.’s (2005; 2010) 42 sub-indicators do not only involve legal but also cultural aspects that depend on jurisprudence, administrative decrees and local implementation practices – information that is more difficult to find and code. Finally, the MIPEX-indicator, which is based on a large range of over 100 sub-indicators, relies on an at-tempt to measure policies in the realm of immigrant integration against a standard of “best practice” drawn from Council of Europe Conventions or European Community Directives. Data are collected by means of expert surveys in which the respective na-tional legislations are evaluated.
In the light of such findings one might wonder why we do not simply stick to indicators that require relatively few resources to be built. More generally, we also need to ask why so many resources are invested in large projects, such as the EUDO citizen-ship project. What is the added value of such a project? Would it not be better to heavily reduce the number of items that are covered in such a project and instead collect data over time and for countries outside the Western world? Such questions not only concern naturalisation, but also other domains, such as the research projects that measure policy positions of political parties. Various studies have shown that different methods based on newspaper and party manifesto codings as well as expert and population surveys lead to similar results (Marks et al. 2007; Ray 2007; Helbling and Tresch 2010).
A second problem in the current citizenship literature concerns the confusion between policy outputs and outcomes. Whereas one group of researchers focuses on citizen-ship policies (Waldrauch and Hofinger 1997; Koopmans et al. 2005; 2010; MPG 2006; Howard 2009), another group seeks to explain the number of people affected by such policies (i.e. naturalisation or rejection rates) (Helbling 2008; Janoski 2010). This differentiation is, however, seldom made explicit, and it is often not clear why one indicator is preferred over another one and whether or not they are used to answer the same research questions.
The latter question can be illustrated by a construct validity test that “assesses whether a measure relates to other observed variables in a way that is consistent with theoretically derived predictions” (Bollen 1989: 188). In this short test, I compare the three studies by Helbling (2008), Howard (2009) and Janoski (2010) that have developed a very similar theoretical framework (see also Koopmans et al. 2010: 25). All three focus on the politics of citizenship and show that political parties and their mobilisation play an important role; they do so, however, to explain, at least at first sight, three different aspects, namely rejection rates, citizenship policies and naturalisation rates.
In my own study, I argue that local political struggles lead to specific national self-understandings within a municipality, which in turn explains to a high degree local rejection rates (Helbling 2008). This argument is supported by both a large-N study including 103 municipalities and a comparison of 14 case studies. Howard (2009) asks whether widely existing anti-immigrant sentiments become activated politically by the far right, which in turn might dampen liberalisation processes. In his comparison of 15 countries and their development between 1980 and 2008 he finds indeed a quite strong relationship between the strength of far right parties and citizenship liberalisation. Janoski (2010) compares 18 countries and their evolution between 1970 and 2006. He reveals that increasing left party power in the post-World War II period leads to higher naturalisation rates. While all three studies find evidence for their main argument, they show at the same time that socio-structural and socio-economic factors such as immigration and unemployment rates fail to explain the respective de-pendent variables.
Not only is it often unclear why output or outcome variables are put forward. It even happens sometimes that the two aspects are combined. Both Howard (2009) and Koopmans et al. (2005; 2010) use naturalisation rates as part of their policy indicators. Howard (2009: 24) uses naturalisation rates as a “correction” that helps him “to account for the potential problem of a country appearing to have a very inclusive naturalisation policy, but in reality – whether due to administrative “discretion” or other barriers or disincentives – being much more restrictive in practice.” For exactly the same reason, Koopmans et al. (2005: 38) have included naturalisation rates in their indicator. They argue that there are many informal factors that might affect ac-cess of migrants to the nationality of the host country. By including naturalisation rates both studies are at the same time pointing to and blurring the crucial differentiation between policy outputs and outcomes. It thus becomes once more unclear what these indicators are actually measuring.
To measure the outcome of citizenship politics, naturalisation rates are most useful. The same argument holds in other domains such as for example immigration politics. Money (1999) relied on immigration ratios and not on the policies themselves to study immigration politics in her large-N study. She argues that looking exclusively at the formal regulations leaves out important aspects such as the control, interpretation and implementation of laws as well as the consequences of formal regulations that are in this case the actual numbers of admitted immigrants (Money 1999: 22).
Besides the classic naturalisation rates that measure the ratio between the yearly num-ber of naturalisations and the number of foreign residents in a country, there are alter-native ways to evaluate policy effects that might be more valid. Janoski (2010) pro-poses a new and, as he claims, a more comprehensive way of measuring naturalisation rates. Instead of just looking at the number of people who have passed the official naturalisation procedures, he also accounts for different naturalisation regimes, some of which allow some people to "circumvent" the naturalisation processes. For example, in countries with a ius soli policy citizenship is automatically conferred to children of resident immigrants. Janoski considers this as an alternative form of naturalisation and accordingly adjusts his indicator by adding up the number of naturalised people and the number of people who receive the nationality at birth.
Although Janoski accounts for important additional aspects of policy effects, I claim that it is better to use rejection rates. In the same paragraph where Howard (2009: 24) explains why he takes into account naturalisation rates he states that they “should not be relied on too closely, since the rates may depend significantly on the provenance of the immigration population, as well as the demographic patterns of immigration […].” Howard points here to the crucial content validity problem of naturalisation rates that depend on both the demand and the supply side. After all, a low naturalisation rate might be explained by a restrictive naturalisation policy, and/or by the fact that there is low demand on the part of the foreign population. Moreover, since the ratio is measured within the overall foreign-resident population, changes of the naturalisation rate might tell us more about immigration flows (the denominator in the calculation) than about the desire of foreign residents to become naturalised (Ludvig 2004: 509-510).
We need, however, an indicator that depends on the supply side only, since the rea-sons why somebody applies for citizenship are not relevant for explaining the conditions for doing so. Indeed, we would rather like to know why citizenship policy is more restrictive in some settings than in others. For these reasons I propose to use rejection rates, that is, the ratio between the rejected and the submitted applications (Helbling 2008). In contrast with the naturalisation rate, the rejection rate almost exclusively depends on the supply side. Once the applications are handed in, naturalisation candidates no longer have control over them; whether they are rejected or not depends entirely on legal regulations and administrative authorities. While the validity of this indicator is much better for measuring policy effects, it is not perfect for two key reasons. The rejection rate does not consider, first, that certain candidates with-draw their applications during the procedure and, second, that some alien residents might not apply because they do not expect to be naturalised in settings with a restrictive naturalisation policy (deterrence effects).
I conclude with the paradoxical diagnosis that existing citizenship indicators seem to measure the same theoretical concept but that it is unclear what this concept is. Ac-cording to the convergent and construct validity tests, all indicators measure pretty much the same thing. They correlate with each other at a relatively high level and can be explained by the same factors. There are, however, a lot of conceptual inconsistencies and the various scholars seldom specify what they really seek to explain. For a young research field this is not uncommon as one important step at such a stage is to bring together different ideas and approaches before a generally accepted measure of a certain concept emerges.
Nonetheless, every project needs to specify what exactly is under study. Otherwise it is almost impossible to build on existing work and to advance knowledge. Moreover, such clarifications are also necessary to make research more efficient. If it is really the case that the existing indicators measure the same concept, then one might wonder why we should use indicators whose coding is rather time-consuming instead of just taking already available data on the number of naturalised immigrants or even better on the ratio of rejected applications.
Bollen, Kenneth A. (1989), Structural Equations with Latent Variables, New York: John Wiley & Sons.
Helbling, Marc (2008), Practising Citizenship and Heterogeneous Nationhood. Natu-ralisations in Swiss Municipalities, Amsterdam: Amsterdam University Press (IMIS-COE Series).
Helbling, Marc and Anke Tresch (2010), ‘Measuring Party Positions and Issue Sali-ence from Media Coverage: Discussing and Crossvalidating New Indicators’, Ms.
Howard, Marc Morjé (2009), The Politics of Citizenship in Europe, New York: Cam-bridge University Press.
Janoski, Thomas (2010), The Ironies of Citizenship. Naturalization and Integration in Industrialized Countries, New York: Cambridge University Press.
Koopmans, Ruud, Paul Statham, Marco Giugni and Florence Passy (2005), Contested Citizenship: Immigration and Cultural Diversity in Europe, Minneapolis and London: University of Minnesota Press.
Koopmans, Ruud, Ines Michalowski and Stine Waibel (2010), Citizenship Rights for Immigrants: National Paths and Cross-National Convergence in Western Europe, 1980-2008, WZB Discussion Paper SP IV 2010-703, Berlin.
Ludvig, Alice (2004), ‘Why should Austria be different from Germany? The two re-cent Nationality Reforms in Contrast’, German Politics, 13(3): 499-515.
Marks, Garry, Liesbet Hooghe, Marco R. Steenbergen and Ryan Bakker (2007), ‘Crossvalidating data on party positioning on European integration’, Electoral Studies, 26(1): 23-38.
Money, Jeannette (1999), Fences and Neighbors. The Political Geography of Immigration Control, Ithaca and London: Cornell University Press.
MPG (2006), The Migration Integration Policy Index (MIPEX), Migration Policy Group British Council.
Ray, Leonard (2007), ‘Validity of measured party positions on European integration: Assumptions, approaches, and a comparison of alternative measures’, Electoral Studies, 26(1): 11-22.
Waldrauch, Harald and Christoph Hofinger (1997), ‘An Index to Measure the Legal Obstacles to the Integration of Migrants’, New Community, 23(2): 271-285.
- Next >>