ws 09/10© prof. dr. friedrich schneider, university of linz, austria1 6. left-right party ideology...

WS 09/10 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 1

6. Left-right party ideology and government policies: A meta-analysis

Kurs Public Choice

WS 2009/10


6. Left-right party ideology and government policies: A meta-analysis

1. Introduction

2. Data and Methods

3. Empirical Results

4. Overview and Discussion


1. Introduction

(1) The question of the impact of political parties on policy has been the object of much research and debate in the past decades. The main theoretical debate concerns about the relative weight of socio-economic and political variables in determining policy outputs.

(2) Advocates of the so-called ‘convergence’ school of thought argue that industrialized societies of the twentieth century have become increasingly similar, facing the same kind of problems and applying the same kind of solutions.

(3) Consequently, the argument goes, political, institutional and cultural differences do not matter much when it comes to explaining variations in policy outputs.


1. Introduction

(4) In reaction to the convergence argument, policy scholars have suggested that indeed politics does matter. Without denying the importance of economic factors, advocates of the ‘politics matters’ school of thought argue that there is a correlation between partisan variables and policy outputs.

(5) Much empirical research on the policy impact of political parties has been published in the form of comparative analyses based on precise quantitative definitions and measurements within the well-established scientific framework of partisan theory (Hibbs 1977, 1992).

(6) Ceteris paribus, changes in the left-right party composition of government are hypothesized to be related to changes in policy. This is expressed by a general equation of the form: Policy outputs = f (left-right variations in the party composition of government, variations in controlling factors).


1. Introduction

(7) This general equation has been applied, in a large number of studies comparatively to measure partisan influence on government actions in a variety of policy domains across different countries.

(8) The quantitative literature on partisan theory displays a high level of accumulation but it has not achieved a comparable level of scientific aggregation. By accumulation one means that one has a large literature (with publications numbering in the hundreds).

(9) The goal in this lecture is to contribute to this debate by summarizing how the partisan influence literature assesses the relationship between the leftright party composition of government and changes in public policy through a meta-analysis of empirical tests of partisan theory in OECD countries.


1. Introduction

(10)A meta-analysis is an analysis of analyses. It is the statistical analysis of a large collection of test results coming from individual studies for the purpose of integrating the findings:

i. In meta-analysis, ‘the findings of multiple studies should be regarded as a complex data set, no more comprehensible without statistical analysis than would be hundreds of data points in one study’.

ii. The statistical methods used to accomplish this task are similar to the ones used in primary analyses of the original data.

The major difference is that each case in a meta-analysis is an estimate obtained from the original data whereas each case in a primary analysis is obtained directly from the original data.


1. Introduction

iii. A meta-analysis is useful not only to draw out general patterns from a wide variety of empirical sources but also because it enables researchers to resolve disputes in the literature, to determine which factors have contributed to systematic differences across studies, and to identify areas that have been neglected and warrant further investigation.

iv. The meta-analysis helps identify both substantive influences on research findings and artefacts resulting from particular research methods and data analysis techniques.


2. Data and methods

(1)Although they share a common scholarly purpose, meta-analyses (MAs) and literature reviews (LRs) are quite different from each other.

(2)LRs examine how a particular phenomenon has been studied by scholars in published works.

(3)MAs deal with specific empirical hypotheses and try to investigate, on the basis of the empirical work so far published, whether there is an impact of some independent variables upon dependent variables, and, if so, what the magnitude of this impact is.


2. Data and methods

(4) Like any other empirical research, MAs are vulnerable to various threats to validity:

i. The main threat to validity in this MA is heterogeneity among the studies incorporated in the analysis.

ii. One tries to respond to this threat to validity; first, by establishing explicit criteria for the selection of individual studies in order to ensure that the tests within each study are comparable; and second, by coding relevant published information in the form of variables most susceptible to influencing the predictive performance of each test.

iii. The criteria for the selection of cases and the rules for constructing the variables in the MA were organized in a codebook which served as guidance for researchers to identify individual parameter estimates and their relevant attributes in each study.


2.1. Selection of cases

(1) One concern in selecting individual cases for the meta-analysis was comprehensiveness and/or comparability. To have confidence in measures, one needs to be sure that the selection process was representative of the available material. To achieve this goal, one tries to minimize discretion in selection by considering everything we could find in the published record.

(2) Over 600 studies (mainly journal articles and book chapters) were identified, containing empirical results.

(3) In this MA we are interested in reporting cross-national comparisons and include results from cross-sectional studies and times-series cross-sectional studies only.


2.1. Selection of cases

(4)The validity of the meta-analysis may also be threatened by the wide variety of measures of party ideology used by partisan theorists to explain policy outputs in individual studies (the explanans). To limit heterogeneity in party ideology variables, one considers only tests in which ideology is defined as either left or right.

(5)Another possible threat to validity in MA is the lack of sample homogeneity in the estimation methods. One of the objectives of this MA is to extract overall measures of the relationship between the party composition of government and policy outputs from single coefficients found in individual studies.


2.2. Factors in the meta-analysis

(1) The dependent variable in this MA is the predictive performance of individual tests of partisan theory, measured on the basis of parameter estimates of party impact.

(2) These estimates are operationalized in terms of three attributes:

(i) estimates that support the partisan theory hypothesis (the coefficient is significant and in the predicted direction) are recorded as ‘successes’;

(ii) estimates that fail to confirm the hypothesis (the coefficient fails the test of statistical significance) are reported as ‘failures’;

(iii) estimates that contradict the hypothesis (the coefficient is significant and in the wrong direction) are reported as ‘anomalies’.



(3) Individual tests of partisan impact on welfare policies typically postulate that, ceteris paribus, welfare intervention by the state is positively correlated with the presence of leftist parties in government and negatively correlated with the presence of rightist parties in government (denoted L+, R−).

(4) The approach postulates that spending on development assistance is positively correlated with the presence of the left in government and negatively correlated with the presence of the right in government (L+, R−).

(5) The hypothesis is reversed for military spending and military alliance membership: military spending and state involvement in military alliances correlates positively with the presence of the right in government and negatively with the presence of the left in government (L−, R+).



(6) One important variant of this approach is the political business-cycle hypothesis, which states that the standard relationship (L+, R−) holds for state intervention in macro-economic policy, public debt as a percentage of GDP, and progressivity of taxes. The reversed relationship (L−, R+) holds for privatisation, and budget balance as a percentage of GDP.

(7) A final approach examines how partisan factors affect the overall size of government. The approach postulates that, ceteris paribus, government size correlates positively with the presence of the left in government and negatively with the presence of the right in government (L+, R−).



(8) Individual tests of partisan theory also vary in terms of how they operationalize observed government activity. Most studies operationalize government activity in terms of public spending or revenues from taxes (often expressed as a percentage of GDP).

(9) In order to assess the impact of variation in the definition of policy domains on our conclusions, we record whether individual parameter estimates apply

i. to social welfare policies,

ii. the size of the state,

iii. economic policies, and

iv. or foreign and defense policies.

We also record whether the measure of government activity in each test is based on

a) financial data (e.g., public spending) or

b) non-financial data (e.g. regulation or administrative action).



(10)Individual authors often calculate the presence of the left or the right in government in the form of sophisticated indices. However, so far we do not have a standardized normalized measure.

(11)There is some disagreement about the range of parties to be included in the measure. Most studies examined in this MA report the presence of the left and/or the right in government on the basis of either leftist parties, a left-right scale, conservative parties, or major parties of the right.

(12)Individual measures of the party composition of government also differ in their definitions of relative party strength. The most frequently used measure is the party composition of the cabinet.

(13)This is justified on the grounds that in advanced liberal democracies, political power is basically exercised through the cabinet and that governmental policy is directly related to the allocation of ministerial portfolios among different parties.



(14)Another factor that may affect the validity of our meta-analysis is the variation in the specification of the model in each individual study, i.e. the number and nature of explanatory or control variables that partisan theorists explicitly include in their datasets.

(15)There is evidence that the statistical significance and the direction of estimates of partisan impact may be affected by institutional factors such as the division of power between parties in and out of government,

the frequency of electoral competition, or the time a government has been in power.


3. Empirical Results

(1) Out of over 600 studies identified in the bibliographic search, 43 proved to correspond to our selection criteria.

(2) The studies are listed in chronological order in Table 3 along with information (i) about the period, (ii) the number of countries over which the tests were run, (iii) the relevant policy domain, and(iv) the left-right attribute used to define the party composition of government variable.

(3) Table 1 reports only the summary information for each study.

Individual tests within each study may involve different time periods, countries, policy domains and different measures of party ideology.


Table 1: Characteristics of quantitative studies on the ideology-policy relationship


Table 1: Characteristics of quantitative studies on the ideology-policy relationship (cont.)


3.1. Overall left-right partisan effects

(1) Table 1 also presents the number of relevant tests in each study and the frequencies with which these tests are reported as ‘successes’ (a coefficient associated with a right or a left partisan variable is significant (p < 0.10) and in the correct direction), ‘anomalies’ (a coefficient associated with the right or the left is significant and in the opposite direction to the one hypothesised) or ‘failures’ (a non-significant coefficient) in this study.

(2) Once the results of each individual test have been sorted into successes, failures and anomalies, we can use the simple ‘vote-counting’ meta-analytic procedure to assess the overall relationship between the left-right party composition of government (the explanans) and policy outputs (the explanandum).


3.1. Overall left-right partisan effects

(3) Looking at the overall distribution of tests (bottom row of Table 1), we see that, out of the 693 tests, only 154 (22 percent) are reported as successes and 48 tests (7 percent) are reported as anomalies. The remaining 491 tests (71 percent) fail the statistical significance test (at 10 percent) and are reported as failures.

(4) The success rates appear remarkably stable over time: 86 out of 385 tests published in or before 1986 are reported as successes (22 percent); 68 out of 308 tests published after 1987 are reported as successes (22 percent).


3.2. Mediating factors

(1) Clearly, these numbers cannot be interpreted as evidence in support of the left-right party impact hypothesis. Applying the vote-counting technique to the data suggests that, since the ‘failure’ category is the modal category, we cannot reject the null (convergence) hypothesis of no party impact based on the average predictive power of individual tests.

(2) Table 2 presents a breakdown of individual tests by policy domain. We see from the table that 69 (24 percent) out of 288 tests in the welfare policy domain are reported as successes.

(3) The difference in predictive performance across specific areas within the welfare domain (from a low of 19 percent to a high of 27 percent) is not statistically significant. Next we see that 59 (37 percent) out of 161 tests of the size of the state are reported as successes.


Table 2: Success rate of individual tests by policy domain



(3) As in the welfare domain, the deviations from the mean in the size of the state domain are not statistically significant.

(4) The overall success rates are much lower in the two remaining policy domains. There are 20 successes (13 percent) out of 148 tests in the foreign affairs and defence policy domain and only 6 successes (6 percent) out of 96 tests in the economic policy domain. The deviations from the mean success rate are statistically significant in both the foreign affairs and the economic policy domains.

(5) Tests of left-right party impact on the level of state intervention in the economy have a (relatively) very high degree of success (39 percent). On the other hand, the success rate in the public budget and taxation areas is virtually zero.


Table 3: Success rate of individual tests across other variables



(6)We see from Table 3 that there are twice as many tests based on the presence of the left (472) as compared with tests based on the presence of the right (221) in our sample.

(7)Tests that report the presence of the right generate a higher rate of success on average (25 percent) than tests that report the presence of the left (21 percent).

(8)Looking at the party strength variable of Table 3, we see that using the percentage of cabinet portfolios, although the preferred method of calculating party strength, has no statistical impact on success when compared with alternative methods.



(9) Next, we see that 79 (28 percent) out of 279 multivariate tests are reported as successes against 75 (18 percent) out of 414 bivariate tests.

(10)The paired difference for the model specification variable is statistically significant, as is the paired difference for sample size (the proportion of success is 20 percent for smaller samples against 28 percent for larger samples) and for the measure of government activity (the rate of success is 23 percent for financial data compared with 12 percent for non-financial data).

(11)The paired difference for the historical period variable is also statistically significant: 92 (27 percent) out of 339 tests based on the post-1973 period are reported as successes against 64 (18 percent) out of 354 tests based on the pre-1973 period.


Table 4: Success rate of individual tests by historical period and by policy domain



(12)An interesting pattern emerges when we crosstabulate the historical period variable with policy domains (Table 4).

(13)Post-1973 tests of party impact on the size of the state, foreign affairs and economic policy have significantly higher average success rates than pre-1973 tests of party impact in these policy domains.

(14)However, individual tests of party impact on welfare policy follow the opposite direction.

The average success rate of estimates of party impact on welfare is smaller for the post-1973 period than for the pre-1973 period.



(15)Table 5 presents a multivariate logistic regression model in which the dependent variable takes the value of 1 when a test is reported as a success and 0 otherwise. There were too few anomaly cases to include them as a distinct variable in this analysis.

(16)Three policy domain variables are included in the model to assess the predictive performance of tests of party impact on the size of the state (total public spending and taxation), foreign affairs, and the economy as compared with the predictive performance of tests of party impact on welfare (the reference policy domain).

(17)The next two variables in the model are dichotomous variables intended to assess whether the predictive performance of individual tests is affected by the use of non-financial policy data (as opposed to financial policy data) and by the inclusion of a left (instead of right) partisan variable.


Table 5: Multivariate logit analysis of individual tests’ predictive performance



(18)From Table 5, we see that, ceteris paribus, the success rate of individual tests is not significantly affected by how partisan impact is operationalized.

(19)Whether individual tests use the left or the right to measure partisan impacts does not affect predictive performance, and neither does the definition of party strength (although measures of party strength based on the percent of popular votes perform better than measures based on cabinet portfolio allocation).

(20)The data also rule out sample size and the use of financial (as opposed to non-financial) data as causes of variation in predictive performance.



(21)Thus, keeping the other variables in the model constant, the predicted logit of success for a multivariate, post-1973 test of the left-right division of the popular vote on the size of the state is L = −1.17 + 0.56 + 0.67 + 0.50 + 0.67 = 1.23.

(22)The predicted probability of success for such a test is therefore P = 1/[1 + e−(1.23)] = 0.77. By comparison, the probability of success of a bivariate, pre-1973 test of partisan impact on welfare using a mixed measure of party strength is P = 1/[1+e−(−1.17)] = 0.24.

(23)This demonstrates that there are clearly identifiable conditions under which the probability of support for the left-right party influence hypothesis can be substantially increased.


4. Overview and discussion

(1) We started this review of 693 published cross-section estimates of left-right party impact on policy with a series of questions that have not been addressed in a meta-analytic format before:

i. how often do estimates support the left-right party impact hypothesis?

ii. What is the average magnitude of the effect size of left-right party impact?

iii. What (if any) are the determinants of variations in statistical support?

iv. Are partisan cycle explanations more sensitive to substantive factors (e.g., the policy domain of government intervention; the historical period of analysis) or to issues of measurement and methodology?

v. How do our results contribute to the advancement of knowledge in the field?



In response to the first two questions:

154 (22 percent) estimates support the left-right party impact hypothesis,

48 (7 percent) estimates contradict the hypothesis, and

491 (71 percent) estimates fail to support the hypothesis.

Applying the ‘vote-counting’ meta-analytic technique to these numbers leads us to conclude that the null hypothesis of no left-right party impact cannot be rejected.



We reach a similar diagnosis when we use a revised ‘combined tests’ technique to generate an ‘average effect size’ of left-right partisan impact on policy.

The 0.28 average correlational effect between party and policy indicates that approximately 8 percent of the variation in policy output is accounted for by the variation in the left-right party composition of Government.

Literature:

Louis M. Imbeau, Francois Pétry and Moktar Lamari (2001), Left-Right Party Ideology and Government Policies: A-Meta-Analysis, European Journal of Political Research 40/1, 2001, pp. 1-29.