2004 minnesota council for quality training for team leads and judges 2004 award cycle

2004 Minnesota Council for Quality

Training for Team Leads and Judges

2004 Award Cycle

2004 Team Leader Judge Trainingb.ppt 2 2004 Minnesota Council for Quality

Today’s Agenda

Purpose and objectives

Stage 0: Preparation

Stage 1: Independent Review

Stage 2: Consensus

Stage 3: Site Visit

Feedback Report

Stage 4: Post Site Visit Activities

Improvement Planning

Judging

Evaluation of Today’s Learning


Today’s Purposes

Ensure that Team Leads and Judges understand the MQA process and deliverables and can teach and mentor evaluators

Improve understanding of roles, responsibilities, procedures, forms and expectations

Improve ability to guide a team

Increase awareness of resources for Team Leads

Convey changes in the process


Today’s ObjectivesTeam Leads will be able to:

Lead a team from formation through site visit

Ensure quality of the feedback report

Participate in improvement planning

Report to the judges

Help recognize the organization

Judges will be able to:

Mentor team/Team Leads from team formation through site visit

Review site visit issues

Edit the feedback report

Present the organization to the Panel of Judges and participate in the Judges’ deliberations



The Minnesota Quality AwardAssessment Process

Stage 0:Preparation

Stage 2:Consensus

Stage 3:Site Visit

Stage 4:Post-Site

Visit

Stage 1:Indpdnt.Review

Improvement PlanningMQA Judging


Stage 0: Preparation

NOTE: This stageof the process may begin at any time and overlaps withapplicant preparation

0.01Council

receives LOI

0.03Council notifiesChief Judges

Check COIConsider sector, Criteria knowledge,

time since last assignment, other factors

0.05Chief Judgesnotify Council

& judges of judgeassignments

0.02Council

builds team &initiates TeamTracking Sheet

0.06Standard Message #1:

Council notifiesteam membersof assignments

cc: Lead Judges &Chief Judges

0.07Team Leads

divide up TL/team dutiesvia phone or

e-mail


Team Leader sends e-mail,"Welcome to the Team,"

to all team membersw. cc to Lead Judges

& Council

0.10Team Lead schedules intro.

team meeting & e-mailsagenda to team members cc: Lead Judges & council

0.09Team members& lead judges

respond w/introductory

e-mails

0.11Team holds actual/virtual meeting

to get acquainted, set ground rules, establish computer compatibility. etc.

(see agenda)

0.12Team Lead

contacts ParticipantRepresentative

(see agenda)

GO TOSTAGE 1

0.13Team Lead finalizes

date/locationfor Consensus/SV

Plng Meeting

0.04Chief Judges

assign Lead Judges


Stage 0: PreparationTeam Lead Role

Team formation and communication

Establish liaison with applicant organization

Establish plans and schedules

Communicate performance expectations

Manage the team process

Report team progress to Lead Judge and Council

Coach individual team members


Team Dynamics

Forming Storming Norming Performing


Team Formation and Communication Team members introduce themselves by e-mail

CC: Brian, Kathryn, Lead Judges

Introductory Meeting (real or virtual)

Team Formation Team Lead role Team member role Lead Judge role

Ground rules

Team-specific timeline & deliverables

Site visit calendar


Setting Ground RulesSome suggestions:

Anyone can mediate. Respect other opinions. “Shadow” assignments. Assign evaluators to “shadow” evaluators in other roles. Writing rule owners. Assign one writing rule to each member of the team. Have that

team member check the feedback report for violations of that rule before sending the product to the Council.

Time start and end. Make up agendas for teleconferences as well as for face-to-face meetings. Appoint a time keeper for each meeting.

Have a “parking lot” for ideas. If something is being discussed for too long with no resolution, “park” it and move on to other things. Come back to it later.

Commitment to quality comments early in the process leads to less re-work when preparing the final feedback report.

Don’t be afraid to say, “I don’t know.” Don’t be afraid to ask the Council, Team Leads, or Judge Mentors for help during the process. Speak up – ask questions. Don’t wait. Do it now.

Set aside pride of ownership. If your comment is put “below the line,” it is no reflection on your abilities as an evaluator or your value to the team. Everybody makes mistakes!

Work at the big picture. Don’t get bogged down in the smaller details. Have fun!


Liaison with Applicant Organization

Team Lead contacts applicant representative: Introductions Exchange contact information Overview of the process and status Confirmation of site visit dates Initial dates for site visit training & planning Expectations for the applicant representative Applicant representative’s expectations

Inform team, Lead Judges, and Council of schedule


Yes

No

Yes

DAY 1

DAY 3

DAY 4

DAY 1

1.01Council emails

Evaluators application& materials

1.02Evaluators acknowledge

receipt to Council by e-mail

1.03Evaluators check COI

with Organizational Profile

1.04 Is there

a possibleconflict?

1.05Evaluator contactsCouncil to resolvepossible conflict

1.06Is there

a conflict?

DAY 1

Stage 1: Independent ReviewDAY 1

1.09Evaluators complete

Key Factors Worksheet& e-mails to designated

consolidator

1.10Consolidator combines

individual KF worksheets into one & e-mails

to Team & Lead JudgesASAP (usually 24 hours)

1.08Evaluators

skim application

No

DAY 4

1.12Evaluators e-mail

Item 1.1 to Team Leads& Lead Judges (TLs may

send theirs to model)

1.13Team Leads

& Lead Judgesreview all Item 1.1

comments

1.11Evaluators prepare

draft commentsfor Item 1.1

DAY 5

1.14Team Leads

& Judges confer

GO TOSTAGE 2

1.15 Do comments

conform w/Writing Rules?

No Yes

1.16Team Lead

notifies Evaluator

1.17Team Lead

coachesEvaluator(s)

1.18Evaluators prepare

Item Worksheets andExpectated Results Matrix

<= DAY 30

1.19Evaluators e-mail all

Worksheets &Expected Results Matrix

to full team

1.07Council replaces

Evaluator; Evaluatorreturns/destroys materials


Stage 1: Independent ReviewTeam Lead Role

Communicate performance expectations Manage the team process Report team progress to Lead Judge and Council Coach individual team members Preview Item 1.1 with Lead Judges

Teleconference with Lead Judges to determine which team members need coaching and who should provide it.

Appropriate coach follows through by contacting evaluators by e-mail or phone to provide coaching

Communicate evaluation of 1.1 comments to all team members


Stage 1: Independent ReviewEvaluator Role

1. Check for Conflict of Interest – Profile/Org. Chart – contact Council if necessary

2. Summarize Organizational Profile and send to fellow team members within first week Volunteer consolidates and

distributes to team

3. Write comments for Item 1.1

4. Send comments from Item 1.1 to Team Leaders and Judges immediately

5. Write comments for Items 1.2 through 6.2.

6. Compile Expected Results Matrix

7. Write comments for Category 7

8. Complete all comments & expected results table and distribute to team members


Reviewing Comments for Item 1.1 – Team Leaders & Lead Judges

1. Item 1.1 preview. Comments should Be tied to the criteria Follow the writing rules Have a consistent voice and style Be written on an appropriate number of Strengths and OFIs

(usually 3-5 Strengths and 3-5 OFIs)

2. Team Leaders and Lead Judges consult Identify need for coaching individual team members Decide who will do the coaching

3. Provide feedback/coaching to team members as needed


No

Yes

<= DAY 33

<= DAY 34

<= DAY 41

<= DAY 44

Stage 2: Consensus & Site Visit Planning

2.01Team Leads

assign Category / Itemleads & back ups


Team Leads CommunicateConsensus assignments& meeting arrangements

to team memberscc: Lead Judges &

Chief Judges &Council

2.04Team Leads

& Lead Judgesconfer

2.05 Are matrices

OK?

2.06Team Lead

coachesEvaluator(s)

2.07Team Lead

notifiesEvaluator

2.09Evaluators e-mail

completedConsensus Matrix

& Expected Results Tableto team

2.08Evaluators completeConsensus Matrices

& consolidatedExpected Results Tables

for assigned Items

2.03Evaluators completeConsensus Matrix for

first assigned Item& e-mails to Team Leads &

Lead Judges

2.11Team meets from8AM-3PM to reach

consensus on comments& draft KTs

2.10Evaluators review all

materials beforeConsensus meeting &

identify possiblediscussion points <= DAY 47<= DAY 45

<= DAY 44 2.14Council removes

Evaluator(s) to SVsize if needed;Council sendsSV material

2.17 Are comments

& SVIs OK?

<= DAY 48

SV Strategy WorksheetsExit presentation template

Judges presentation templateFeedback Report template

2.15Category/Item Leads

prepare SV Issuesheets & e-mail to team

& Lead Judges

No

Yes GO TOSTAGE 3

2.18Team Leads

revise Commentsand/or SVI Sheets 2.16

Team Leads& Lead Judges

confer on:comments, SVI's,

storyboardissues, KTs,walkarounds

2.12Participant rep.arrives (3PM);

trainers provideJIT SV training

2.13Category Leads

e-mail Consensuscomments to

team &Lead Judges

<= DAY 30


Stage 2: ConsensusTeam Lead Role

Communicate performance expectations Manage the team process Report team progress to Lead Judge and Council Assign Category Owners Preview each team member’s work Coach individual team members Lead the consensus meeting Establish plans and schedules for site visit with applicant

representative


Assign Category Owners

Assess knowledge, skills, abilities and interests of team members Use introductory e-mails each team member sent out

Divide work load

Ask for volunteers

Assign both Category Owners and backups (depending on team size)

Communicate assignments to Team, Council and Lead Judges using Standard Message #3


Consensus Matrix

Review information from training

Use the Basic, Overall, Multiple Guidelines to determine the appropriate comments for consolidation

Have each Category Owner send the first item consolidated to the Team Leads and the Lead Judges for review, feedback, and coaching as needed


Lead the Consensus Meeting Arrange for the facility, meals, equipment, parking, directions,

security, etc.

Set the agenda and timeline to conclude by 5:00 PM with the last 2 hours as site visit training

Inform the Council and applicant representative of site & date to arrange site-visit training

Set & communicate expectations for Category/Item Owners

Facilitate timely progress through the agenda

Assign follow-up actions

Have Category Owners send revised consensus matrices to team members, Lead Judges, and the Council


Stage 2: ConsensusJudges’ Responsibilities

Establish relationship with the Team Leads

Read the application

Review the team’s plans & schedules (provide feedback as needed)

Communicate performance expectations

Monitor team process & progress – coach team members as issues arise

Coach Team Leads as needed / requested

Review consensus report, draft key themes, and site visit issues and strategies and provide feedback as needed


A Word About the Baldrige Express

Use the same process as for the narrative

More emphasis on approach than deployment at the site visit


Site Visit Training & Planning Two-hour training session begins at 3:00 PM

Includes applicant representative Detailed review of people, process, products

Tentative site visit agenda

Share key site visit issues with applicant representative

Set expectations for team regarding completing Site Visit Issue Sheets

Send first Item SVI sheet to Team Leads

Provide feedback to team members

Share all SVI sheets with team members & Lead Judges


<= DAY 57

Stage 3: Site Visit

3.04Team arrives

at site

3.01Team arrives

at site visit hotel

<= DAY 583.03

Team preparespreliminary

Story Board *

3.02Team conducts

final prep meetingevening before(see agenda)

3.06Team conducts

preliminary caucus

3.05Participant KickOff presentation& introductions(optional tour)

3.07Team conducts site visit:

interviews, data review, walkarounds;frequent team caucuses; daily revision

of storyboard (reviewed with HRO at EOD)and comment worksheets/SVIs

<= DAY 60

3.09Team Leads/Council

meet w/Participant HRO

3.08Team prepares

exit presentations

<= DAY 60<= DAY 60<= DAY 60

3.10Team makes

exit presentation

3.11Team finalizes

Item comments and scores

3.12Team completes

Executive Summary(KTs), PGPs, Judges'

questions

3.14Team provides improvement

input for process, completes 360reviews, conducts final checklist,

& leaves site/hotel

<= DAY 60

GO TOSTAGE 4

* The story board process is coveredin the site visit training.

3.13Team Leads

appoint feedbackwriter (optional)


Stage 3: Site VisitTeam Lead Role

Assign feedback report writer Review Site Visit Issues with team members at night before site

meeting Co-lead opening meeting with Council representative Communicate performance expectations Manage the team process Coach individual team members when needed Lead team caucuses between interviews to identify which new issues

may have arisen, which issues have been resolved, and which issues are still open

Make sure all site visit issues are addressed before the third day Lead scoring Prepare executive preview Co-lead closing meeting with Council representative Prepare feedback report and provide to Judges Ensure that all team members know that they are to fill out and return

360° and MQA Process evaluations


Stage 3 – The Site Visit

Final preparation

Site Visit Opening Interviews Caucuses Executive Preview Closing Meeting Feedback Report and Scoring Presentation to judges 3600 Evaluation Process Evaluation


Team Leads and Judges Ensurethe Quality of the Feedback Report

1. Assign feedback editor

2. Review feedback report when complete Do comments follow writing rules? Are process and results comments appropriate? Are spelling & grammar correct? – use the tools Does report have a consistent voice/style? Are all item references correct? Do they follow the correct

item format (1.1a(1), 1.1a(1-3), 1.1a(1&3), 1.1a – NOT 1.1.a.1, 1.1a1, or 1.1a[1])?

Are Executive Summary and comments personalized? – avoid overuse of “the organization”

Are there any “it is not clear” comments? Are comments and scores aligned?


Stage 4: Post-Site VisitIMPROVEMENT PLANNING/JUDGING

4.02Council sends

post-site scheduleto TLs, Judges

<= DAY 62

4.03Feedback Writer completes

draft FR & e-mails TL(optional, if FW assigned)

4.04Team Leads editreport & e-mail to

Lead Judges

<= DAY 72

<= DAY 79

4.07Council sends final FRto TLs, Lead Judges, & Participant contact

(as appropriate)

<= DAY 81

4.08Council finalizes IPagenda/approach

w/Participant

4.09TLs/Council prepare

for IP Session

4.11Council sends thank youand satisfaction survey toParticipant (OCP, HRO)

4.01Council confirms

time/location for IPSession w/Participant& sends to full team

4.10TLs/Council participate

in IP Session: review FR,prioritize opptys, create

draft action plans(as appropriate)

4.05Lead Judge(s)work with TLsto create final

draft report

<= DAY 82

<= DAY 89

4.06Council makes

final edits

4.13TLs work withLead Judge(s)

to create Judges'Presentation; send to

Council

4.14Council sends agenda,Judges' presentation,

Org Profile, Cat 7,& FR to full POJ

4.15Council preparesscoring profiles &

loads PPT Judges'Presentation(s)

4.12Council confirmstime/location forJudging Session;

sends invite toEval. team(s)

4.18Panel of Judges

deliberates & reachesconsensus on level;if questions, contact

TLs by phone

4.17Lead Judges

present case to POJ& recommend

MQA level

4.16Team Leads

present to POJ(see agenda)

4.19Council communicatesrecognition level & key

messages to ParticipantHRO & Team Leads

END OFASSESSMENT

PROCESS

NOTE: Improvement Planning sessions arescheduled individually with each applicant;

Judging Sessions are batched throughout theyear. The process above is typical (IP before

Judging), though on occasion Judgingprecedes the IP Session.

END OFJUDGING PROCESS

4.20Team Leads

inform team membersof decision (not forpublic consuption)


Stage 4: Post-Site Visit ActivitiesTeam Lead Role

Aid the Council in scheduling Improvement Planning session with applicant rep

Participate in Improvement Planning Present Judges’ Presentation to the Panel of Judges Make sure that you are available to the Panel of Judges by

some means throughout the entire Judging Day. Provide contact information on the title page of the Judges’ Presentation.

Communicate thanks and the award level your applicant received to the team. Remind the team to complete and send in 360° and MQA process evaluation forms.

Participate in private award recognition ceremony (optional) Participate in public award recognition ceremony (optional)


Team Leads Participate in Improvement Planning

Identify vital few improvement strategies(It’s OK to be prescriptive.)

Discuss improvement planning agenda with Council staff/designate prior to the meeting

Lead feedback report discussion

Co-facilitate next steps improvement discussion

Thank the organization for participating


Team Leads Report to the Panel of Judges

Review the presentation template on site with the team

Prepare presentation immediately after completing feedback report

Send presentation to the Council for distribution to the Panel of Judges

Present to the Judges

Answer questions raised by the Judges Be available for follow up questions


Judges’ Presentation -Key Factors

Slide One:

Location

Products and services

Employee Profile

Slide Two:

Strategic Challenges


Judges’ Presentation –Key Themes: Process Strengths

What are the most important strengths or outstanding practices (of potential value to other organizations) identified in the process categories?

Include 3-5 bullets, which should align with the Executive Summary in the feedback report, on the above


Judges’ Presentation – Key Themes: Process OFIs

What are the most significant opportunities, concerns, or vulnerabilities identified in the process categories?

Include 3-5 bullets, which should align with the Executive Summary in the feedback report, on the above


Judges’ Presentation – Key Themes: Results Strengths and OFIs

What are the most important strengths or outstanding practices (of potential value to other organizations) and significant opportunities, concerns or vulnerabilities identified in the results category?

Include 3-5 bullets, which should align with the Executive Summary in the feedback report, on the above.


Judges’ Presentation –Significant Differences Within the Team

This slide in the Judges’ Presentation is optional. Use it only if the team disagreed. If you do not have anything to add to this slide, go ahead and delete it.


Judges’ Presentation -Commitment

The extent to which leaders are willing to invest time, resources and energy in making the journey.

An example of low commitment: leaders spend little time talking about quality or planning for improvements – few resources are made available for improving quality systems and approaches

An example of high commitment: leaders consistently encourage efforts to improve the organization – leaders plan for high levels of performance – leaders allocate resources to the effort


Judges’ Presentation - Sustainability

The extent to which the organization is likely to continue making progress on its journey

An example of low sustainability: The quality journey may be seen as just a management fad – “something the boss wants us to do”. If the leader of the quality effort were to disappear from the scene, the journey would stop

An example of high sustainability: “Managing quality is everyone’s job. We need to excel as an organization in order to survive and prosper. It’s not just the boss’ thing.” If there were a change in senior leadership, progress would continue.


Judges’ Presentation -Momentum

The direction and rate of progress on the quality journey.

An example of low momentum: There may be slow or no progress overall from year to year, although some pockets of rapid or steady growth may exist. The organization may be little aware of its opportunities for improvement or how to close the gaps. Perhaps, improvements in the past may have been lost and are no longer apparent.

An example of high momentum: Innovation and improvement in business practice is frequent and pervasive. Baldrige scores may have been rising each year through multiple cycle of self-assessment and improvements.


Guidelines for Judges’ Presentation

Be prepared – create the presentation AS SOON AS the team completes the Feedback Report

Follow the presentation template

Assist in understanding the comments of the report

Stick to the time limit

Use clear, concise, objective responses to questions

Do not advocate for the applicant


The Lead Judges Present the Organization and Deliberate

Introduce Team Leads and team members prior to their presentation

Thank Team Leads after their presentation

Team Leads leave

Facilitate Judges’ deliberation Summarize presentation Provide additional information as necessary Identify follow up questions for Team Leaders Suggest an award level


MQA Process Time Frame

Letter of Intent Arrives: Day 1

Stage 0: Preparation: Days 2-60

Stage 1: Independent Review: Days 61-91

Stage 2: Consensus and Site Visit Planning: Days 92-115

Stage 3: Site Visit: Days 116-143• Finalize Feedback Report: Days 144-152

Stage 4: Post-Site Visit Activities: Scheduled after the feedbackreport is received


Team Leads and Judges Are Invited to Help Recognize the Organization

Participate in the private recognition ceremony as defined by the organization.

Attend the Council’s public recognition ceremony.


Today’s Objectives:Are You Ready?

Team Leads are able to:

Lead a team from formation through site visit

Ensure quality of the feedback report

Participate in improvement planning

Report to the judges


Judges are able to:

Mentor team/Team Leads from team formation through consensus

Review site visit issues

Edit the feedback report

Present the organization to the Panel of Judges and participate in the judges’ deliberations


2004 Minnesota Council for Quality

Thank you for your participation.

2004 minnesota council for quality training for team leads and judges 2004 award cycle

Documents

team leader judge trainingb

ppt9 team formation

communication team members

minnesota council

ppt8 team dynamics

organization slide

process slide

quality training