Evaluating the impact Evaluating the impact –– what what do we need to know?do we need to know?

Rose Ryan

Heathrose Research Ltd

What is the evidence base for labour and employment relations policies?

• Evaluations of specific policy interventions (eg; WLB, trial employment periods)

• Small scale case studies

• Analysis of outcomes (eg; collective agreements, work stoppages, ERA determinations and mediations)

• Submissions and consultative processes with representative groups

• Anecdotal evidence

“There is limited robust or relevant data in respect of (Parts 9 and 10) of the Act. In respect of the review of the role of the Department’s Labour Inspectors, the Department has anecdotal evidence to support the analysis of policy options”

(Regulatory Impact Statement, Amendments to the Employment Relations Act, 2000, 18/10/20100).

Consequently .....Consequently .....

“The analysis of the policy options under consideration is constrained by the limited availability of objective evidence of problems with the current holidays legislation, both on the type and extent of any problems. There is a lack of robust data available to quantify the potential impacts of the policy options considered.”

(Regulatory Impact Statement, Review of the Holidays Act 2003, 4/6/2010).

What do we need to know What do we need to know and why?and why?

Did they achieve their policy aims and intent?

How are they contributing to the overall operation of the system?

Are they contributing to achieving the overall objectives of the Employment Relations Act and the Holidays Act?

Operation of the 90-day provisionsCollective bargaining processesDynamics of union-management interactions

at the workplaceCompliance with minimum conditionsDispute resolution

Issues that we need to know aboutIssues that we need to know about

Understanding the changes in Understanding the changes in contextcontext

How have the changes been understood at the workplace?

What is the

impact on behaviour?

For exampleFor example

Operation of the 90-day provisions

Collective bargainingDynamics of union-

management interactions at the workplace

Compliance with minimum conditions

Dispute resolution

• Impact on recruitment and selection processes, and on employment rates

• What happens?• In what areas is union

access problematic and for what reason?

• What is the level of non-compliance and why?

• Are things being resolved at the right level and what is the impact of the changes?

Some gaps .......Some gaps .......

Workplace Employment relations practicesEmployee engagement and representationFlexible work arrangements and working

timeJob quality and employee well-beingWorkplace climate.

Need for some large-scale and longitudinal data bases

Better understanding of issues from a range of perspectives – multiple respondents◦And the differences within groups as much as

between themInput from social partners

Implications for methodologyImplications for methodology