dmaic: adapting a six sigma method to program assessment mary early, ph.d. learning support services...

DMAIC: Adapting a Six DMAIC: Adapting a Six Sigma Method to Sigma Method to

Program AssessmentProgram Assessment

Mary Early, Ph.D.Learning Support Services

Arizona State University

Learning Support Services, Arizona State University, 2007

Six SigmaSix Sigma

Pioneered by Bob Smith at Motorola Pioneered by Bob Smith at Motorola in 1986in 1986

Metric for measuring defects & Metric for measuring defects & improving qualityimproving quality

6 6 = a defect rate < 3.4 per million = a defect rate < 3.4 per million Customer-focused, fact-based, data-Customer-focused, fact-based, data-

driven, problem-solving methodologydriven, problem-solving methodology


Applying DMAIC to Student Applying DMAIC to Student AffairsAffairs

DMAIC, a process developed through DMAIC, a process developed through Six Sigma, is a cycle of continuous Six Sigma, is a cycle of continuous improvementimprovement

DMAIC consists of five steps:DMAIC consists of five steps:– DefineDefine– MeasureMeasure– AnalyzeAnalyze– ImproveImprove– ControlControl


DEFINEDEFINE

IMPROVEIMPROVE

CONTROLCONTROL

ANALYZEANALYZE

MEASUREMEASURE

Continuous Improvement: Continuous Improvement: DMAICDMAIC



Define (set objectives of improvement Define (set objectives of improvement project):project):– Develop program- and department-level student Develop program- and department-level student

learning outcomeslearning outcomes– Assess achievement of SLOs Assess achievement of SLOs

Measure (collect data to establish baseline):Measure (collect data to establish baseline):– What SLOs are in place now? What SLOs are in place now? – Where are SLOs missing? Where are SLOs missing? – What assessment strategies are in place? What assessment strategies are in place?



Analyze (find opportunities to Analyze (find opportunities to improve the current state):improve the current state):– Brainstorming session Brainstorming session – Program outcomesProgram outcomes– Data dumpData dump


Sample Template for Program Sample Template for Program SLOsSLOs

Program: Tutoring

Learning Outcome Assessment Method(s)

Time(s) of Assessment

Collection Agent

Application of concept

Tutor observation

During tutoring session

Tutor notes

Acquisition of learning skills

Tutee evaluations

Midterm and end of semester

Evaluation forms

Improved grades

Compare grades of tutored/non-tutored members of classes

After grades are posted

Data WareHouse

Adapted from Gloria Rogers, ABET, Inc.Adapted from Gloria Rogers, ABET, Inc.


Data DumpData DumpContent knowledgeContent knowledgeAcculturation to the Acculturation to the

universityuniversitySelf-knowledgeSelf-knowledgeApplication of Application of

skills/strategiesskills/strategiesInformation and Information and

technology literaciestechnology literaciesEngagement/involvementEngagement/involvementApplication of skills and Application of skills and

strategiesstrategiesCollaborationCollaborationSelf-managementSelf-management

Personal reflectionPersonal reflection

Self-awarenessSelf-awareness

Personal responsibility/ Personal responsibility/ accountabilityaccountability

Critical thinkingCritical thinking

Communication skillsCommunication skills

Culture of classroomCulture of classroom

Campus resourcesCampus resources

Interpersonal relationship Interpersonal relationship buildingbuilding



Improve (develop creative solutions)Improve (develop creative solutions)– Compare and combine data into Compare and combine data into

“envelopes”“envelopes”– LSS developed three envelopes of LSS developed three envelopes of

shared departmental goals:shared departmental goals:־ Academic competenciesAcademic competencies־ Life-skills developmentLife-skills development־ Social competenciesSocial competencies


Learning Support Services’ SLO Learning Support Services’ SLO EnvelopesEnvelopes

Academic CompetenciesAcademic Competencies– Content knowledgeContent knowledge– Application of skills/strategiesApplication of skills/strategies– Critical thinkingCritical thinking– Information and technology literaciesInformation and technology literacies



Life-Skills DevelopmentLife-Skills Development– Application of skills and strategiesApplication of skills and strategies– Critical thinkingCritical thinking– Self-awareness/knowledgeSelf-awareness/knowledge– Personal reflection/awarenessPersonal reflection/awareness– Personal responsibility/accountabilityPersonal responsibility/accountability– Self-managementSelf-management



Social CompetenciesSocial Competencies– Communication skillsCommunication skills– Acculturation to ASUAcculturation to ASU– Culture of classroomCulture of classroom– Campus resourcesCampus resources– Interpersonal relationship buildingInterpersonal relationship building– Engagement/involvementEngagement/involvement– CollaborationCollaboration


Working DocumentsWorking Documents

Three files serve as the SLO Three files serve as the SLO envelopes on our department’s envelopes on our department’s shared driveshared drive

Each program coordinator Each program coordinator contributed learning outcome contributed learning outcome statements and statements and measures/assessments to pertinent measures/assessments to pertinent files (academic, life-skills, and social files (academic, life-skills, and social competencies)competencies)


SLO ExampleSLO Example Academic competencies example for peer Academic competencies example for peer

mentoring: mentoring: – StatementStatement: Students will learn concepts related : Students will learn concepts related

to leadership, peer education and mentoring, to leadership, peer education and mentoring, communication, group dynamics, and diversity, communication, group dynamics, and diversity, as well as familiarizing themselves with as well as familiarizing themselves with campus programs and services that support campus programs and services that support academic and co-curricular student success.academic and co-curricular student success.

– MeasuresMeasures: Student self-reflection through : Student self-reflection through journals and classroom discussion, instructor journals and classroom discussion, instructor observation, course grades, pre- and post-observation, course grades, pre- and post-surveys, mentee and coordinator evaluations. surveys, mentee and coordinator evaluations.



Control (maintain improvements)Control (maintain improvements)– Administer evaluationsAdminister evaluations– Assess successAssess success– Commit to modifying SLOs next yearCommit to modifying SLOs next year


Questions?Questions? Presentation materials will be available Presentation materials will be available

through the NASPA website through the NASPA website (www.naspa.org)(www.naspa.org)

ASU’s Continuous Improvement website:ASU’s Continuous Improvement website:www.west.asu.edu/tqteamwww.west.asu.edu/tqteam

Contact information:Contact information:Mary Early, Ph.D.Mary Early, Ph.D.Learning Support ServicesLearning Support ServicesArizona State UniversityArizona State [email protected]@asu.edu(480) 965-5261(480) 965-5261www.asu.edu/lrcwww.asu.edu/lrc