A Participatory Consultation Process Model

Reflection on, Description of;

A Participatory Consultation Process Model

Reflection on, Description of;

The European University Association, Quality Review of the

Dublin Institute of Technology

2005

Prepared for the Irish Evaluation Network,

TCD Policy Institute, June 2006

Aims of presentation

1. Introduction1. Introduction

2. Quality Review Context2. Quality Review Context

3. DIT emergent response 3. DIT emergent response

4. The process and methodology 4. The process and methodology

5. Utilisation5. Utilisation

1. Introduction1. Introduction

• Aidan Kenny BA (Hons) & MSc. DCU, student D.Ed. TCD

• DIT roles (96-06)• Project Manager, Skills research

Initiative.• Qualifications Framework

Development Officer.• Quality Review, Consultation

Process Facilitator.• Lecturer, School of Construction

faculty of the Built Environment.• Staff representative on the

Academic Council.• TUI Branch Chair.• TUI Branch Equality Officer• Subject Matter Expert ITAC• Chief Examiner NSC DoES

• Other roles (80-00)• Community Development Officer

Clondalkin partnership.• Resource Teacher Youthreach• Partner Dublin In-depth

Photography.• Partner All Surface Plastering Ltd.• Varity of voluntary community work,

– CAFÉ,

Belfast Exposed, Clondalkin Travellers Development Group,

Pavee Point,

Ballymun Co- op,

SOAL drugs project etc.

• Dublin Institute of Technology (DIT)– Origin 1887 – DIT Act 1992– Awarding body (level 6-10 NFQ) – 4000 Major Awards annually, – 22,000 students, annual intake 9695 undergraduates, 1272

postgraduates 4000 apprenticeship, remaining part time & short courses.

– 2100 staff, 1500 academic, 500 non academic, 100 management & other.

– Six Faculty’s, Applied Arts, Business, Engineering, Built Environment, Science, Tourism and Food.

– 35 locations in Dublin City Centre– 250 undergraduate programmes, 30 taught postgraduate

programmes, MPhil & PhD researchers (353), apprenticeship programmes (20/25 Designated Trades), junior music and a variety of part time courses.

1. Introduction1. Introduction

• Macro• European Commission

– Sorbonne 1998 • Harmonisation of the European HE system

– Bologna 1999 • European Higher Education Area

– Comparable degrees, Two cycles, ECTS, Mobility, Quality Assurance co-operation, European dimensions.

– Prague 2001• European Network of Quality Assurance in HE (ENQA)

– Berlin 2003• ENQA & European University Association (EUA)

2. Quality Review Context2. Quality Review Context

• Mesco• National• 1997 Universities Act,

– Conference of Heads of Irish Universities CHIU (now called the Irish University Association IUA) carried out research on Quality assurance procedures for the universities.

– The Act required quality review to be undertaken on a 5-7 year cycle (EUA carried out this work from 2004-2005)

• 1999 Qualifications (Education & Training) Act, – Required the DIT and the two awards councils FETAC & HETAC

to Quality Review on a 5-7 year cycle. Reports to be sent to the NQAI for consideration and made public.

2. Quality Review Context2. Quality Review Context

• Micro• 1992 DIT Act, required an annual report on the

functioning of the Institute• 1996-98 Academic Council developed and

implemented DIT Quality Assurance – Procedures, based on the concept of subsidiary

(Programme teams – Faculty Boards – Academic Council – Governing Body)

– Note student involvement in all levels.

• 2004 DIT & NQAI agree to request the EUA to carry out a Quality Review of the Institute.

2. Quality Review Context2. Quality Review Context

• EUA approach– Questions for the Quality review;

• What is the Institute trying to do?• How is the Institute trying to do it?• How does the Institute know it works?• How does the Institute change in order to improve?

– Focus;• Quality Assurance Procedures• Mission & Strategic Plan• Learning & Teaching• Research• Organisational Structures• Resources & Facilities

2. Quality Review Context2. Quality Review Context

– Quality Review team• Four international academic experts, one student (ESU)

– Process• Quality review guideline document• Establish a Steering Committee• Produce a Critical Self Evaluation Report• First Visit of Review team (3 days)• Additional information• Second visit of Review team (4 days)• Oral feedback• Final report

12 m

on

ths

2. Quality Review Context2. Quality Review Context

• Governing Body, President and Directorate agree to establish a Steering Committee (SC)

• SC composition, – 10 members, Chair Director of Academic Affairs,

Secretary Academic Registrar, member for each faculty (different grades, academic and non academic), Research and enterprise representative and student DITSU representative.

– Supported by, Consultation Process Facilitator, 2 Quality Assurance Officers and administrative staff.

• SC was responsible for, information dissemination, gathering data and compiling the Self Evaluation Report (full autonomy).

3. DIT emergent response 3. DIT emergent response

– SC considerations;• Complexity (of the task)• Politics (inherent in large organisations)• Trust (needed to be established)• Collaborative participation (SC, staff, students)• Stakeholders participation (trade unions, DITSU)• Ethics (social research)• EU Consultation Directive (in draft form at the time) • Time frame (project management)• Communications (effective channels)• Truth (enquiry and reporting)

3. DIT emergent response 3. DIT emergent response

• My approach:My approach:• ParadigmParadigm; ;

– Interpretative some cross over with Critical theoryInterpretative some cross over with Critical theory– Naturalistic informed (Guba, Lincoln 89) 4th Generation EvaluationNaturalistic informed (Guba, Lincoln 89) 4th Generation Evaluation

• Methodology;Methodology; – Informed by Yin 93, Stake 95, Hamel 93, Guba & Lincoln 89, academic Informed by Yin 93, Stake 95, Hamel 93, Guba & Lincoln 89, academic

disciplines of sociology, psychology, management and education practice and disciplines of sociology, psychology, management and education practice and theory,theory,

– Case study type, Descriptive (Yin 93) Intrinsic study (Stake 95), Case study type, Descriptive (Yin 93) Intrinsic study (Stake 95), – Applications (Yin 93) Explain complex, real-life, intervention, explore situations.Applications (Yin 93) Explain complex, real-life, intervention, explore situations.– Unit of analysis DIT communities (staff, student, stakeholders).Unit of analysis DIT communities (staff, student, stakeholders).

• Research designResearch design;;– ‘‘Multi method’ (Morgan 97)- qualitative (focus groups x 10, submissions x6)-Multi method’ (Morgan 97)- qualitative (focus groups x 10, submissions x6)-

quantitative (online surveys x2), link with Guba & Lincoln (89, p 44) 4th quantitative (online surveys x2), link with Guba & Lincoln (89, p 44) 4th Generation Evaluation.Generation Evaluation.

• Data analysisData analysis– Triangulation (Denzin 84, Yin 84).Triangulation (Denzin 84, Yin 84).

• ProcedureProcedure;;– Develop protocol,Develop protocol,– Conduct study,Conduct study,– Analyse evidence,Analyse evidence,– Develop conclusions/recommendations.Develop conclusions/recommendations.

4. The process and methodology 4. The process and methodology

• Initial three stage process

AnalysisPreparation

OrganisationReadiness

AnalysisPreparation

OrganisationReadiness

InformationEngagement

ListeningRefine

InformationEngagement

ListeningRefine

Data gatheringAnalysisFeedbackReporting

Data gatheringAnalysisFeedbackReporting

4. The process and methodology 4. The process and methodology

Consultation process

Aims And

objectives

Readiness for

Consultation

Consultation

Collect data

Consultation Team

Develop process,

Tools, Strategy

Run Pilot

Evaluate and refine tools and process

Collate data

Evaluate data May need to

adapt tools or process

Draft data Feedback to target

group. Recommendations

are feed back. A plan of action is

moved forward

Evaluate outcome against

aims and objectives

May need to re-

engage process

Mixed method social research

Focus groups6 thematic

3 stakeholder

Online surveysStudent

Staff

Faculty Board

submissions

Personal submission

4. The process and methodology 4. The process and methodology

Staff Focus groups sample profile,

12%

9%

16%12%

12%

4%

4%

3%

4%

3%1%1%1%1%1%

16%

Built Environment 8

Tourism & Food 6

Business 10

Science 10

Engineering 8

Applied Arts 8

Researcher 3

Research students 3

Administration 2

Technicians 3

Library 2

Porters 1

ICT 1

HR 1

Careers 1

LTC 1

Staff Focus sample profile.

Mixed group method.

, Six thematic

focus groups.

Excluded members of

Faculty Board’s

N=68.

4. The process and methodology 4. The process and methodology

Stakeholder Venue Time Male Female Date Moderator

TUI Bolton St. 3.00 6 1 15/12/04 2

DITSU Aungier St. 12.00 6 1 13/1/05 2

Amicus Pembroke St. 11.00 4 1 17/1/05 2

IMPACT/SIPTU Aungier St. 2.30 5 0 19/1/05 2

Faculty Boards Venue Time M F Date Presenters

Built Environment Linenhall 2.30 18 3 17/11/04 2

Applied Arts Mountjoy Square 2.30 14 9 23/11/04 2

Engineering Kevin St., 3.30 16 2 23/11/04 2

Tourism and Food Cathal Brugha St., 3.00 6 6 25/11/04 2

Business Aungier St 12.00 13 4 02/12/04 2

Science Kevin St. 2.30 21 9 02/12/04 2

Total number of staff at presentations 88 33 N=121

Stakeholder Focus Group

N=24

Presentations made to Faculty Board'sN=121

Student response rate

807(86%)

59(6%) 46(5%) 12(1%) 2(0%) 16(2%)0

100

200

300

400

500

600

700

800

900

1

Category, total number of responses 942

Act

ual n

umbe

rs

Full time u/g Full time p/g Part time ug/pg Short courses Apprenticeship No category

Reponses rate per faculty

135(14%)

351(38%)

80(8%)

136(14%) 151(16%)

73(8%)

16(2%)

0

50

100

150

200

250

300

350

400

1

Act

ual n

umbe

r

F. Applied Arts F. Business F. Built Environment F. Engineering

F. Science F. Tourism and Food No faculty

Student Response rate

N=942(3 days)

Staff response rate N=479

(5 days)

• Analysis

• Data (qualitative & quantitative)– Processed– Coded– Clustered– Triangulated

Emerging themes priority order

Human resource management Research

Teaching Standards

QA procedures Committees

Faculty structuresPower/partnershipExternal contactAccommodation

4. The process and methodology 4. The process and methodology

• Findings from the Self-evaluation process were used for;– The Self-evaluation Report– The construction of an action plan to address issues and

concerns that emerged, see below example of one stream of action.

5. Utilisation5. Utilisation

TTHHEEMMEE

AACCTTIIOONN

AACCTTIIOONN PPLLAANN

Mission, strategic plan and positioning

President/Directorate to establish working party representative of the Institute’s stakeholders to:

Carry out a review and updating of the Strategic Plan, in the context of the planning for Grangegorman, and goals should be prioritised; Address the positioning of the Institute, especially in the context of changing demographics and competition from other HE institutions in Ireland; and, Take cognisance of this self-evaluation report and the recommendations of the EUA that will emerge from the current process; and, Consider whether there should be a re-application for university designation.

Thank youThank you

• In particular,– Dr Gerry McNamara DCU

– The Irish Evaluation Network

– TCD Policy Institute

– Further information– Aidan Kenny, 01 402 3312, aidan.kenny@dit.ie

• Article by A. Kenny on consultation process ‘case study’ available http://level3.dit.ie/html/issue3/kenny/kenny_abstract.html

• DIT EUA webpage (includes final EUA Report) available http://intranet.dit.ie/academicaffairs/EUA/documents.html