1

TAU PMIG Evaluation in Practice

“Flexible Methodology” 19 Oct 2010

By Barrie Kroukamp & Cliff Matthews

2

Or……..

3

Some hard data can never be hard enough

The key to flexible evaluation

4

It’s All in the Mind

• While it is critical to be uncompromising when doing evaluation it is realistic to be philosophically flexible.

• This applies especially to on going projects or mid term reviews• So get hard on data but soft on application• Try to be as developmental in approach as possible • Organizations value and respect interim feedback so that they can

take action immediately • Try to see it as a process rather then a moment in time balance

sheet• Allow for and encourage robust debate and even conflict within the

evaluation team• Apply a number of balancing “instruments” to check data• Ensure that personal or marketing agendas do not get in the way of

the truth.• Listen intently but don’t hold back with any feedback …just be nice.

5

The Q sort oka the 10 card trick

6

OCAT Methodology

B. MANAGEMENT PRACTICES 1. Organisational structure and culture a. TAU has an organisational structure with clearly defined lines of authority and responsibility

0 1 2 3 4 5 6

b. Management policies are in place to ensure regular audits of organisational development

0 1 2 3 4 5 6

c. Management policies reflect equality as defined in the South African Constitution

0 1 2 3 4 5 6

d. Systems are in place for regular measure of congruence between stated mission and operating culture

0 1 2 3 4 5 6

e. Systems are in place to ensure appropriate involvement of all levels of staff in decision making

0 1 2 3 4 5 6

f. TAU has policies and procedures in place to ensure mutual accountability to key stakeholders

0 1 2 3 4 5 6

7

How we normalized the data

OCAT - External Responses

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

CLIENTS 6.1 6.4 6.2 6.1 6.6 6.1 6.6

STAKEHOLDERS 6.6 6.8 6.2 6.4 6.6 7 6.8

SERVICE PROV 5.9 5.6 5.9 6.4 6.8 6.3 6.6

TOTAL 6.1 6.2 6.1 6.2 6.7 6.2 6.7

GOVERNANCE MNG PRACTICEHUMAN

RESOURCESFINANCIAL

RESOURCESSERVICE

DELIVERYEXTERNAL

RELATIONSSUSTAINABILITY

EXTERNAL VIEW

INTERNAL VIEW

40

3939

27

2412

Role Clarity

Communication

Leadership

Cooperarion

Job Security

Perf Mng

10

SUMMARY OF FINDINGS

11

SUPPLY CHAIN MANAGEMENTTopic Rating Comments

Use and compliance with governance documents and Standard Operating Procedures (SOPs)

Record-keeping

Bidding process

Bid evaluation/ Contract award

Contract preparation and signing

Management of Contract Execution

Professional behavior and ethics

Anticorruption Measures

Appeal Mechanisms for bidders

3 = Good, 2 = Average, 1 = Insufficient

12

KEY SUCCESS FACTORS1. Leadership

2. Brand, niche

3. Loyal clients

4. Skills

5. Industry knowledge and “intimate” relationships

6. Accountability – Counterpart, MOA

7. Discernment with projects – Justification, diagnostic

8. Management – Project, SCM, Contract, Business, QA

9. Support Structures

10. Process & Methodologies (Technical) Balance

11. Trust – NT & Government

ORG. MATURITY STAGES

14

GREINER MATRIX

Category Phase 1

Creativity

Phase 2

Direction

Phase3

Delegation

Phase 4

Co-ordination

Phase 5

Collaboration

ManagementFocus

Development of process

Efficiency Growth of clients base

Consolidation Problem solving and innovation

OrganisationStructure

Informal Centralized Functional

Decentralized Line reporting and process groups

Matrix

Top Management Style

IndividualisticEntrepreneurial

Directive Delegation Oversight Participative

MonitoringSystems

Results and feedback

Standards and regulations

Reporting mechanisms

Plans and strategic focus

Mutual Goal setting

Recognition Emphasis

Commitment and ownership

RemunerationPerformance

Individual achievement

Commitment and ownership sharing

Team acknowledgement

15

THANK YOU