1 2012 examiner training 2012 examiner training introduction to evaluating a baldrige application...

1

20122012EXAMINER TRAININGEXAMINER TRAINING

Introduction to Evaluating a Baldrige

Application

Presented byThe Granite State Quality Council

andThe Northern New England Alliance for Excellence

2

AgendaAgenda

• Stages of a Baldrige Evaluation• The Evaluation Process

– Before you begin

– Key Factors

– Key Themes

– Evaluating each Item• Process Item Demonstration

• Results Item Demonstration

• Review of the Assignment

3

What is a Baldrige Application?What is a Baldrige Application?

• A structured response to the questions posed by the Criteria– May be for internal use or for state/national award

• Contains:– Eligibility forms

– Glossary of Terms

– Organizational Profile (5 pages)

– Categories 1-7 (50 pages)

4

The Role of a Baldrige ExaminerThe Role of a Baldrige Examiner

• Provide objective feedback on Strengths and Opportunities for Improvement (OFIs)– First pass is based solely on the Application– A Site Visit may follow to verify and clarify

feedback from Application

5

The Goal of an Application The Goal of an Application EvaluationEvaluation

• Unbiased feedback on Strengths and Opportunities For Improvement (OFIs)

• Baldrige feedback is – Non-prescriptive

– Non-judgmental

– Grounded in the Criteria & Core Values

– Weighted on the Organization’s Profile

• Feedback may be used to determine award level

6

The Baldrige Application Evaluation Process The Baldrige Application Evaluation Process - Stages- Stages

Goal: Value Added Feedback

To the Applicant

ConsensusComments

IndependentObservations

Site VisitComments

Stage 1

Stage 2

Stage 3

7

Application Evaluation Application Evaluation ProcessProcess

10

The Application Evaluation Process – The Application Evaluation Process – Independent ObservationsIndependent Observations

Check for Conflict

of Interest

Assemble Materials and Read

Application

DraftInitial

Key Factors

Draft InitialKey ThemesObservations

Finalize Key ThemesObservations

Complete Checklist and

Assemble Scorebook

Evaluate Each Criteria Item

11

Contents of a Baldrige Application Case StudyContents of a Baldrige Application Case Study: : Tillingate LivingTillingate Living

• Eligibility Information– Pages i – x

• Additional Information– Org Charts, Glossary/Abbreviations (pages xi – xv)

• Responses to Baldrige Criteria Questions– Preface: Organizational Profile (pages xvi – xx)

– Categories 1 – 7 (pages 1-50)

12

The Application Evaluation ProcessThe Application Evaluation Process

Check for Conflict

of Interest


Application

DraftInitial

Key Factors




Assemble Scorebook


13

What you will use to Evaluate the ApplicationWhat you will use to Evaluate the ApplicationApplicationCriteria

No independent researchor contact with Applicant

On-line Worksheets

14

What you will use to What you will use to Evaluate the ApplicationEvaluate the Application

On-line Worksheets

15

The Narrative Application Evaluation ProcessThe Narrative Application Evaluation Process

Check for Conflict

of Interest


Application

DraftDraftInitialInitial

Key FactorsKey Factors




Assemble Scorebook


16

Key FactorsKey Factors

• What are they?– Significant attributes of an organization

– that influence the way the organization operates

• Where do they come from?– The Organizational Profile

– Criteria Items

– Eligibility Forms

• Guidelines– Group under the five Areas to Address in the Organizational Profile

– Write in phrases rather than complete sentences

– Numbered list (for use in rest of analysis)

– Revise throughout the evaluation process

– Limit to 1–3 pages in length

17

How to Identify Key FactorsHow to Identify Key FactorsAnswers to “What” questions

from the Organizational Profile

Org ProfileCriteria Items

Key FactorsKey FactorsNo Evaluation of

Key Factors

18

Key Factors: ExampleKey Factors: Example• Key Factors:

– Mission, vision, and values– Employee/staff profile– Customer and market segments

and customer requirements– Competitive position and

critical success factors– Strategic challenges– Governance structure

• Not Key Factors:– Strategic planning

approach

– Complaint management process

– Rewards and Recognition System

– Methods to promote and ensure ethical behavior

Answers “What” - facts or attributes that affect the way the organization operates

Answers “How” - processes that the applicant uses in managing its work

19

Case Study page xvi

Criteria Page 4

21

The Key Factors Worksheet

Key Factors Worksheet

P.1a Organizational Environment

•For-profit, privately held organization/Care Model - For-profit, privately held organization providing assisted living and skilled nursing care in 23 facilities located in Pennsylvania, Kentucky, Tennessee, and Virginia whose facilities are known for a sense of societal responsibility. Care model is centered on advancing seniors’ independence and quality of life utilizing the Aging Actively Consortium's (AAC’s) dimensions of wellness: emotional, intellectual/cognitive, physical, spiritual, social, professional/vocational, and environmental.

•Mission, Vision, Values - Mission (Figure P.1-2): Provide ageless care and timeless living to individuals in a homelike environment that supports their lifestyles and need for care with dignity and respectVision (Figure P.1-2): Be among the top 10% of skilled nursing facilities (SNFs) and assisted living facilities (ALFs) and be a top choice for careValues (Figure P.1-2): Agility, Patience, Empathy, and Excellence (APEX)

•Core Competencies (Figure P.1-2) - •Designing, innovating, and managing facilities to support various lifestyles and deliver excellent clinical outcomes•Developing clinical and service competencies for a caring and exceptional staff•Designing and delivering rehabilitation services to support residents’ activities of daily living•Creating an educational environment to support a sense of mastery for residents

Key Factors: a bulleted list of Applicant responses to questions in the

Organizational Profile.

22

The Narrative Application Evaluation ProcessThe Narrative Application Evaluation Process

Check for Conflict

of Interest


Application

DraftInitial

Key Factors

Draft InitialDraft InitialKey ThemesKey ThemesObservationsObservations



Assemble Scorebook


23

What are Key Themes?What are Key Themes?A A Summary of Key Points found in the EvaluationSummary of Key Points found in the Evaluation

Key Theme: A high-level strength or opportunity for improvement that – Is common (cross-cutting) to more than one Item/Category and/or – Is significant in terms of the applicant’s key factors and/or – Addresses a Core Value of the Criteria – Must be traceable to comments found in the Item Worksheets

As a whole Key Themes:– Summarize how well an applicant has addressed the Criteria

requirements, Key Factors, and Core Values– Serve as an Executive Summary

Importance:– Key Themes are usually the first and sometimes the only part of

the feedback report read by senior leaders– Used by examiners to focus site visit– Used by judges to get a “big picture” of the Applicant

24

Organization of the Key Themes WorksheetOrganization of the Key Themes Worksheet

A. What are the most important strengths or outstanding practices found in Process Items?

B. What are the most significant opportunities, concerns, or vulnerabilities identified in Process Items?

C. what are the most significant strengths found in its response to Results Items?

D. What are the most significant opportunities, vulnerabilities, and/or gaps found in the Results Items?

25

Steps in Developing Independent Key ThemesSteps in Developing Independent Key Themes

1. Read application & capture initial impressions.

2. Evaluate application at the Item-level.

3. Key Themes developed at Consensus– Refined after Site Visit

26

Agenda ReviewAgenda Review

Stages of a Baldrige EvaluationThe Evaluation Process

Before you beginKey FactorsKey ThemesEvaluating each Item

• Process Item Demonstration

• Results Item Demonstration

• Review of the Assignment

27

Baldrige National Quality Program

Developing Independent Developing Independent Review Observations Review Observations Process Item DemoProcess Item Demo

28


Check for Conflict

of Interest


Application

DraftInitial

Key Factors




Assemble Scorebook


29

Evaluate Each Criteria ItemEvaluate Each Criteria ItemRead Criteria Requirements

Determine4-6 Key Factors

Develop ~6 Strengths and OFI’s and

Supporting Evidence

Read Applicant'sResponse

Write 2 Feedback Ready Comments

Determine Score

30

Step 1: Read Criteria RequirementsStep 1: Read Criteria Requirements

31

Item 5.1 RequirementsItem 5.1 Requirements

Page 18 in Criteria Booklet

Support Info:•Notes•Category & Item Description

• Page 42

32

Step 2: Key FactorsStep 2: Key Factors

33

P.1a Organizational Environment

P.1b Organizational Relationships

P.2a Competitive Environment

P.2b Strategic Context

P.2c Performance Improvement System

Item 5.1 Independent Review Worksheet – Key FactorsItem 5.1 Independent Review Worksheet – Key Factors

What 4-6 Key Factors do we most expect to be addressed by this Item?

(or – which Key Factors do you most want to keep in mind as you evaluate this Item)

From Key Factors Worksheet – Handout

4-6 Key Factors are requested at the top of every Item Worksheet

All Key Factors

Item Requirements

Item Key Factors

34

Item 5.1 Independent Review Worksheet – Key FactorsItem 5.1 Independent Review Worksheet – Key Factors

4-6 Key Factors are requested at the top of every Item Worksheet

35

Step 3: Read Application Item ResponseStep 3: Read Application Item Response

Categories 1-6Demonstrate

Processes“HOW?”

36

Read through the Item: Example - Item 5.1a(1)Read through the Item: Example - Item 5.1a(1)

What does the Criteria ask?

How does the Applicant respond?

Case Study: Page 22 & 23

37

Definition – PROCESSDefinition – PROCESS• Linked activities with the purpose of producing a

product or service for a customer (user) within or outside the organization

• Involves combinations of people, machines, tools, techniques, materials, and improvements in a defined series of steps

• Service situations - used more generally– May spell out what must be done including a preferred or

expected sequence

• Knowledge work (e.g., strategic planning, research, development, and analysis) – does not always imply formal sequences of steps

– implies understandings regarding competent performance, such as timing, options to be included, evaluation, and reporting.

Page 62 in Criteria Book

38

Analyzing a Process ItemAnalyzing a Process Item

The purpose of Process Items is to permit

diagnosis of an organization’s most

important processes —the ones that contribute most to organizational performance improvement and contribute to keyoutcomes or performance results.

KEYProcesses

39

Read through the Item: Example - Item 5.1a(1)Read through the Item: Example - Item 5.1a(1)

What does the Criteria ask?

How does the Applicant respond?

Case Study: Page 22 & 23

What Processes can you identify in 5.1a(1)?

40

Step 4: Develop Around 6 Total Strengths Step 4: Develop Around 6 Total Strengths and OFI’s and Supporting Evidenceand OFI’s and Supporting Evidence

41

Developing Strengths and OFIsDeveloping Strengths and OFIs

• Strengths– Practices/processes or results to be

reinforced & build upon

• Opportunity for Improvement– Potential gaps, blind spots & vulnerabilities

• Categories 1-6 reflect Processes

• Category 7 reflects Results

42

Main Idea Supporting Evidence

First Pass: Focus on identifying…

A-D-L-I

Relationship withKey Factors Relationship with

Criteria

Strengths

43

Gaps and OFIs

44

From our Example - Item 5.1a(1)From our Example - Item 5.1a(1)

45

Example OFIs from - Item 5.1Example OFIs from - Item 5.1

46

Evaluating a Process ItemEvaluating a Process Item

• Processes include– Approach (A)

– Deployment (D)

– Learning (L)

– Integration (I)

• A-D-L-I is a tool to help in application evaluation and scoring

• A-D-L-I is the P-D-S-A of High Performing Organizations

47

ApproachApproach

• The methods and processes used by an organization to accomplish its work

• HOW an organization addresses the item requirements

• Approach considers…– Appropriateness for organization– Effectiveness– Alignment with the organization’s needs– Appropriate review and improvement

• Approaches are systematic if they – Are well ordered – Are repeatable – Use data and information so learning is possible– Build in the opportunity for evaluation, improvement, and sharing thereby permitting

a gain in maturity

48

DeploymentDeployment

• Refers to the EXTENT to which an organization’s approach is applied

• The breadth and depth of the approach to relevant work units throughout the organization

• Implementation on steroids…

Is the approach deployed from the

President to the janitor?

49

DeploymentDeployment

Deployment is process specific. AppropriateAppropriate deployment varies from item to item

Examples:• Category 1 – Leadership Processes

– Deployed to workforce segments, stakeholders, other key processes

• Category 3 – Customer Processes– Deployed to customer segments, work force segments

• Category 5 – Workforce Processes– Deployed to workforce segments, locations, departments

50

Learning:Learning:Organizational and PersonalOrganizational and Personal

• New knowledge or skills acquired through evaluation, study, experience, and innovation.

• Criteria looks for both personal and organizational learning

• There are three dimensions of organizational learning:– Continuous improvement of existing processes

– Innovation leading to breakthrough changes

– Knowledge sharing of such improvements and innovations

51

IntegrationIntegration

• The harmonization of plans, processes, information, resource decisions, actions, results, and analysis to support key organization-wide goals. – Begins with alignment

– Culminates in interconnectivity

• Integration asks:– How well do the individual components of a system operate as a fully

interconnected unit?

– How well is the approach aligned with the applicant’s organizational needs?

– How well is the approach integrated with these needs?

52

Indicating A-D-L-I

A-D-L-I

53

Identifying ProcessIdentifying ProcessStrengths & OFIsStrengths & OFIs

1. Criteria– Strength?– Gap?

2. Key Factors – Importance & Appropriateness

3. A-D-L-I – Specifics

54

Step 5: Write 2 Feedback Ready Step 5: Write 2 Feedback Ready CommentsComments

55

Feedback Ready CommentsFeedback Ready Comments

• Turns Strength/OFI & Evidence into feedback the applicant understands– Polite, Professional, Positive– Not: Judgmental or Prescriptive– Concise, one message per comment– Indicates

• Criteria-based strength or potential OFI

• Potential significance of strength/OFI

56

Creating a Feedback Ready Comment: StrengthCreating a Feedback Ready Comment: Strength

2,4,5,6 Systematic integrated approach to employee health & safety

- Exec Director of each facility = risk management/safety officer, serve on system wide Safety Committee- Review all facility/system health safety, security policies & procedures- Meet monthly to review key measures; APEX Goals- Integrated with daily rounds with Collaborative Care Teams- Use of Data to drive improvements

b(1)

b(1) A variety of methods and associated goals (Figure 5.1-2) help ensure workplacehealth, safety, and security. These methods include a system-wide Safety Committee thatmeets monthly to review progress toward goals, unexpected events, and best practices.Daily rounding by Collaborative Care Teams helps identify potential safety hazards.

57

Creating a Feedback Ready Comment: OFICreating a Feedback Ready Comment: OFI

1,2,3, 4, 6

Approaches to management of workforce capability/capacity & to maintain a safe, secure & supportive work climate do not address all segments of workforce

- Little discussion of physicians, volunteers, students--Unclear if security is an issue at some of the locations (e.g. SLFs)-Approach for capacity planning is somewhat unclear – they just post the positions and move people around?

a,b

a, b It is unclear how the applicant manages volunteers, physicians, and students to fully support its work. For example, it is unclear how volunteers are trained and managed and how approaches are deployed to precepted students and credentialed physicians. Approaches for managing these workforce groups may help the applicant deliver high quality care and services and maintain a safe, secure environment.

58

b(1) A variety of methods and associated goals (Figure 5.1-2) help ensure

workplace health, safety, and security. These methods include a system-wide

Safety Committee that meets monthly to review progress toward goals,

unexpected events, and best practices. Daily rounding by Collaborative Care

Teams helps identify potential safety hazards.

Polite?

Professional?

Positive?

Judgmental

Prescriptive

Nugget: Key point based on Criteria

Indicates Relevance to this applicant

Examples for clarity & context

59

Polite?

Professional?

Positive?

Judgmental

Prescriptive

a, b It is unclear how the applicant manages volunteers, physicians, and students

to fully support its work. For example, it is unclear how volunteers are trained

and managed and how approaches are deployed to precepted students and

credentialed physicians. Approaches for managing these workforce groups may

help the applicant deliver high quality care and services and maintain a safe,

secure environment.’




60

Step 6: ScoringStep 6: Scoring

61

Scoring a Process ItemScoring a Process Item

• Two components– A-D-L-I

– Level of Response to Item Requirements

62

Baldrige Scoring: Baldrige Scoring: Reflects Organization’s Maturity in the JourneyReflects Organization’s Maturity in the Journey

• Level of Response to Item Requirements

• Two dimensions of scoring

– For Process Items (Categories 1-6)

• Approach

• Deployment

• Learning

• Integration

– For Results Items (Category 7)

• Levels

• Trend

• Comparisons

• Integration

63

The Performance Excellence JourneyThe Performance Excellence Journey

Beginning On the Road

Mature

Anecdotal evidence, beginning of systematic approaches

Clear evidence of effective, systematic approaches fully responsive to current and changing business needs

Little or no deployment Approaches consistently deployed to all relevant work units

Reaction to problems; early stages of an improvement system

Fact-based, systematic evaluation and improvement process; strong refinement and innovation

Little or no alignment of key processes and work units

Approaches, processes, and work units fully aligned and integrated with organizational needs

Maturity of Processes

64

Basic RequirementsBasic Requirements

65

Overall RequirementsOverall Requirements

66

Multiple RequirementsMultiple Requirements

67

Item FormatItem Format Criteria – Page 29

68

UNDERSTANDING SYSTEMATIC APPROACHUNDERSTANDING SYSTEMATIC APPROACH

- Criteria

69

UNDERSTANDING DEPLOYMENTUNDERSTANDING DEPLOYMENT

70

UNDERSTANDING IMPROVEMENT/LEARNING/INNOVATIONUNDERSTANDING IMPROVEMENT/LEARNING/INNOVATION

71

UNDERSTANDING INTEGRATIONUNDERSTANDING INTEGRATION

72

Scoring BasicsScoring Basics• Scores should align with comments.

• Base the scores on a holistic view of the item and the applicant’s responses

• Avoid requiring more evidence than is reasonable given the applicant’s 50-page limit.

• From low to high, the scoring ranges reflect a growth in organizational maturity.

• Note that the scoring ranges reflect a growth in maturity.

• In the scoring ranges, note how the language for each scoring factor (ADLI) changes to reflect higher levels of organizational maturity

• Tip: Start at the 50 – 65% band

73

Holistic Scoring Rubric based on Maturity• ADLI • Basic – Overall - Multiple

74

Master Scoring Grid

Criteria Point ValuesPage 3 - Criteria

Determined byExaminers

75

Baldrige National Quality Program

Developing Independent Developing Independent

Review ObservationsReview Observations Results Item DemoResults Item Demo

76

The Model for The Model for Performance Excellence Performance Excellence

is a is a Results Driven Model!Results Driven Model!

77

Definition of Results Definition of Results

“Outputs and outcomes achieved by an organization in addressing the requirements of a Baldrige Criteria Item”

78

ResultsResults

Results document how the applicant measures

– its performance over time

– against competitors’ performance

– against relevant comparative data

79

Results Measure A-D-L-I…Results Measure A-D-L-I…

• Make note of those things that the applicant says are important and that the criteria calls for in Categories 1-6

• Check to see if they are reported in Category 7

80

Not all Results are created equal…Not all Results are created equal…

• Current Performance

• Segmentation

• Trends

• Comparisons

81

Results - Current LevelsResults - Current Levels

Employee survey question – I am satisfied with my job.

% S

tron

gly

Agr

eed

82

Current Levels and Segments Current Levels and Segments %

Str

ongl

y A

gree

d


83

Results - Trends Results - Trends

% S

tron

gly

Agr

eed


84

Results - ComparisonsResults - Comparisons %

Str

ongl

y A

gree

d


85

Results: Evaluate Each Criteria ItemResults: Evaluate Each Criteria ItemRead Criteria Requirements

Determine4-6 Key Factors

Develop ~6 Strengths and OFI’s and

Supporting Evidence

Read Applicant'sResponse

Write 2 Feedback Ready Comments

Determine Score

86

Step 1: Read Criteria RequirementsStep 1: Read Criteria Requirements

87

Results Example – Item 7.3Results Example – Item 7.3Page 24 of CriteriaPage 24 of Criteria

Support Info:•Notes•Category & Item Description

• Pages 47 - 48

88

Step 2: Key FactorsStep 2: Key Factors

89

Item 7.3 Key Factors for Tillingate LivingItem 7.3 Key Factors for Tillingate LivingWhat 4-6 Key Factors do we most expect to be addressed by this Item?(or – which Key Factors do you most wantto keep in mind as you evaluate this Item)

From master Key Factor Worksheet

1. Mission, Vision, Values (Figure P.1‐2) ‐ Mission: Provide ageless care and timeless living in a homelike environment, dignity, and respectVision: Be among the top 10% of Skilled Nursing Facilities (SNFs) and Assisted Living Facilities (ALFs) and be a top choice for careValues: Agility, Patience, Empathy, and excellence (APEX)

2. Employee and Workforce Demographics (Figure P.1‐3) ‐ 3,718 employees;

3. Workforce Groups: Employees, physicians, 700 volunteers; Engagement Factors ‐ Good work environment; Good benefits; Positive relationships with coworkers; Pride in the organization; A voice in resident care

4. Workforce Health and Safety Requirements ‐ Protection for exposure to communicable diseases; Protection form injury; Support in managing residents; Support for a healthy lifestyle;

5. Key Competitors: all other CMS 5‐Star‐rated facilities within 50 miles of each facility;

6. Sources of comparative data: CAHPS questions, CMS Nursing Home Compare, USSN Data Dispatcher, Caring Colleagues, NursQM

90

Step 3: Read Application Item ResponseStep 3: Read Application Item Response

91

Results Example – Item 7.3a(1)Results Example – Item 7.3a(1)

What Results are provided FOR 7.3a(1)?

92

Step 4: Develop Around 6 Strengths and Step 4: Develop Around 6 Strengths and OFI’s and Supporting EvidenceOFI’s and Supporting Evidence

93

Defining “Key” Results?Defining “Key” Results?

“Key” refers to• elements or factors most critical to achieving the

intended outcome

In terms of results, look for those• responsive to the Criteria requirements• most important to the organization’s success• results that are essential elements for the

organization to pursue or monitor in order to achieve its desired outcome

94

Key ResultsKey Results

Results evaluated by different factors

95

Le-T-C-I: Le-T-C-I: Let’s see: Evaluating Let’s see: Evaluating outputsoutputs and and outcomesoutcomes

Levels The current level of performance

Trends• Rate or slope of improvement• Breadth or extent of deployment of results

ComparisonsPerformance relative to • appropriate comparisons (industry, similar organizations)• benchmarks or industry leaders

IntegrationLinkage of results to important requirements or processes• as identified in the Organizational Profile and Process ItemsValid indicators of future performanceHarmonized across processes and work units to support organization

goals

96

Tillingate Living Item 7.3a(1) StrengthTillingate Living Item 7.3a(1) Strength

Note that examiners found evidence outside of a(1) related to capability & capacity

97

Tillingate Living Item 7.3a(1) OFITillingate Living Item 7.3a(1) OFI

98

Step 5: Write 2 Feedback Ready Step 5: Write 2 Feedback Ready CommentsComments

99

Creating a Feedback Ready Comment: Results OFICreating a Feedback Ready Comment: Results OFI

a(1) The applicant has limited results for workforce capability and capacity which may challenge its ability to achieve strategic objectives and goals. For example, results on competencies or skill levels are not reported. Also, results for capacity are limited to ratios of nursing care time to health care outcomes. Additional capability and capacity outcomes may support advancing the quality of care and achieving top-decile resident satisfaction while achieving role-model financial stability.

100

Polite?

Professional?

Positive?

Judgmental

Prescriptive




a(1) The applicant has limited results for workforce capability and capacity

which may challenge its ability to achieve strategic objectives and goals. For

example, results on competencies or skill levels are not reported. Also, results

for capacity are limited to ratios of nursing care time to health care outcomes.

Additional capability and capacity outcomes may support advancing the

quality of care and achieving top-decile resident satisfaction while achieving

role-model financial stability.

101

Creating a Feedback Ready Comment: Results StrengthCreating a Feedback Ready Comment: Results Strength

102

Creating a Feedback Ready Comment: Results StrengthCreating a Feedback Ready Comment: Results Strength

a(3) Top-decile performance on employee satisfaction and engagement survey

measures, as well as low vacancy and turnover rates (Figure 7.3-7), help the

organization sustain its strategic advantage of high employee retention. Overall

satisfaction has been at or better than the top-decile level since 2008 (Figure 7.3-

4). Engagement results for "I am proud to work [here]" and "I would recommend

[applicant] to family members“ survey items were at top-decile levels in 2010 and

2011 (Figure 7.3-6).

Polite?

Professional?

Positive?

Judgmental

Prescriptive




103

Step 6: ScoringStep 6: Scoring

104

Results Scoring Guidelines – Page 69Results Scoring Guidelines – Page 69SCORING DIMENSION

DESCRIPTOR

LEVELS No results; poor results reported

Few results reported for Basic requirements

Good results for Basic requirements

Good results for Overall requirements

Good to excellent results for Multiple requirements

Excellent results fully responsive to Multiple requirements

TRENDS Trend data not reported or show adverse trends

Some trend data, with some adverse trends

Some trend data, with majority presented as beneficial

Beneficial trends in areas of importance

Beneficial trends sustained over time in most areas of importance

Beneficial trends sustained over time in all areas of importance

0-5%

90-100%

0-5%

90-100%

105

SCORING DIMENSION

DESCRIPTOR

COMPARISONS Comparative data not reported

Little or no comparative data reported

Early stages of obtaining comparative data

Some current performance levels evaluated against relevant comparisons

Many to most trends evaluated against relevant comparisons or benchmarks;show levels of leadership and very good relative performance

Industry and benchmark leadership in many areas

INTEGRATION Results are not reported for areas of importance

Results are reported for a few areas of importance

Results are reported for many areas of importance

Results reported for most key customer, market, and process requirements

Results reported for most key customer, market, and process requirements and action plans

Results and projections reported for most key customer, market, and process requirements and action plans

0-5%

90-100%

0-5%

90-100%

106

Determine an overall scoring range and score

107


Check for Conflict

of Interest


Application

DraftInitial

Key Factors

Draft InitialKey Themes

(optional)

Finalize Key Themes


Assemble Scorebook


108

Agenda ReviewAgenda Review

Stages of a Baldrige EvaluationThe Evaluation Process

Before you beginKey FactorsKey ThemesEvaluating each Item

Process Item DemonstrationResults Item Demonstration

Review of the Assignment

109

Materials for AssignmentMaterials for Assignment

• Tillingate Living Case Study– GSQC Website

• Criteria for Performance Excellence– http://

www.nist.gov/baldrige/publications/upload/2011_2012_Health_Care_Criteria.pdf

• Worksheet Packet (e-mailed with slides, on websites)• Completed Key Factors• Blank Item 5.2• Partially Completed Item 7.4

• Reference materials– Slides from today’s workshop(s)– Completed Item 5.1

110

The Assignment…The Assignment…• Read Tillingate Living Case Study

• Review Key Factors Worksheet (Assignment Worksheets)– Any questions? Any changes?

• Prepare Independent Review Worksheets for

– Item 5.2: 4-6 Key Factors, ~ 6 Strengths/OFIs worksheets, Score. (2 comments are optional)

– Complete Item 7.3: Review/Change Key Factors, ~ 6 Strengths/OFIs worksheets, Score. (2 comments are optional)

• Plan on 5+ hours of work on Worksheets

• Submit your completed Worksheets by October 22nd ([email protected])

111

Follow the 6-step process to Follow the 6-step process to complete Worksheetscomplete Worksheets

112

Overview of mid-October SessionOverview of mid-October Session

• Review and Refinement of assignment

• Questions and Clarification

• Overview of Scorebook Navigator tool

• Assignment for November 15th

• Establish 2 hour webinar(s) for week of October 22nd…..

113

Final Questions Final Questions &&

Feedback?Feedback?

1 2012 examiner training 2012 examiner training introduction to evaluating a baldrige application...

Documents

application slide

pages slide

criteria item slide

key factors key themes

excellence slide

assignment slide

baldrige evaluation

application online worksheets