2003 nawi annual conference: leading change in career & technical education
DESCRIPTION
2003 NAWI Annual Conference: Leading Change in Career & Technical Education. General Session II Panel – What’s New in Assessment Pilot Testing Cross Cluster Skills. Presenter: Brenda Hattaway Assistant Executive Director. ANY COMMON KNOWLEDGE & SKILLS?. ? ? ? ?. - PowerPoint PPT PresentationTRANSCRIPT
© 2003 VTECS
2003 NAWI Annual Conference:
Leading Change in Career & Technical Education
2003 NAWI Annual Conference:
Leading Change in Career & Technical Education
General Session II Panel – What’s New in Assessment
Pilot Testing Cross Cluster Skills
General Session II Panel – What’s New in Assessment
Pilot Testing Cross Cluster Skills
Presenter: Brenda Hattaway Assistant Executive Director
© 2003 VTECS
ANY COMMON KNOWLEDGE & SKILLS?ANY COMMON KNOWLEDGE & SKILLS?ANY COMMON KNOWLEDGE & SKILLS?ANY COMMON KNOWLEDGE & SKILLS?
• Arts, Audio-Video Technology,
& Communications
• Health Science
• Information Technology
• Manufacturing
• Transportation, Distribution
& Logistics
• “The Eleven”
• Arts, Audio-Video Technology,
& Communications
• Health Science
• Information Technology
• Manufacturing
• Transportation, Distribution
& Logistics
• “The Eleven”
We
dec
ided
W
e d
ecid
ed
to f
ind
ou
t!to
fin
d o
ut!
We
dec
ided
W
e d
ecid
ed
to f
ind
ou
t!to
fin
d o
ut!
© 2003 VTECS
WHERE WILL WE FIND COMMON WHERE WILL WE FIND COMMON
KNOWLEDGE & SKILLS?KNOWLEDGE & SKILLS?
WHERE WILL WE FIND COMMON WHERE WILL WE FIND COMMON
KNOWLEDGE & SKILLS?KNOWLEDGE & SKILLS?
• CommunicationsCommunications
• Employability & Career DevelopmentEmployability & Career Development
• Information TechnologyInformation Technology
• CommunicationsCommunications
• Employability & Career DevelopmentEmployability & Career Development
• Information TechnologyInformation Technology
Foundation Knowledge & Skills . . .Foundation Knowledge & Skills . . .
Common Across All Clusters In . . .Common Across All Clusters In . . .
Foundation Knowledge & Skills . . .Foundation Knowledge & Skills . . .
Common Across All Clusters In . . .Common Across All Clusters In . . .
© 2003 VTECS
YES!
We identified 23 skills that are
consistently addressed in all 16
career clusters.
YES!
We identified 23 skills that are
consistently addressed in all 16
career clusters.
DID WE FIND ANY COMMON DID WE FIND ANY COMMON
KNOWDEDGE & SKILLS?KNOWDEDGE & SKILLS?
DID WE FIND ANY COMMON DID WE FIND ANY COMMON
KNOWDEDGE & SKILLS?KNOWDEDGE & SKILLS?
© 2003 VTECS
Develop and interpret tables,
charts, and figures to support written and oral communications.
Develop and interpret tables,
charts, and figures to support written and oral communications.
AN EXAMPLE from COMMUNICATIONSAN EXAMPLE from COMMUNICATIONS
COMMUNICATIONS Skill F02.7COMMUNICATIONS Skill F02.7
© 2003 VTECS
How can weHow can weDocument Document ProficiencyProficiency
Attainment?Attainment?
How can weHow can weDocument Document ProficiencyProficiency
Attainment?Attainment?
© 2003 VTECS
Let’sLet’sDevelopDevelop
AATEST!!TEST!!
Let’sLet’sDevelopDevelop
AATEST!!TEST!!
© 2003 VTECS
And That’s How The And That’s How The Great Adventure Began.Great Adventure Began.And That’s How The And That’s How The Great Adventure Began.Great Adventure Began.
Assessmen
Assessmen
ttAssessmen
Assessmen
ttAssessmenAssessmen
ttAssessmenAssessmen
tt
AssessmenAssessmen
tt
AssessmenAssessmen
tt
© 2003 VTECS
WHAT ITEM FORMAT?WHAT ITEM FORMAT?WHAT ITEM FORMAT?WHAT ITEM FORMAT?
© 2003 VTECS
DevelopDevelopItemsItems
DevelopDevelopItemsItems
• Scenarios (Performance)Scenarios (Performance) with Rubricswith Rubrics
• SituationSituation Multiple ChoiceMultiple Choice
• Scenarios (Performance)Scenarios (Performance) with Rubricswith Rubrics
• SituationSituation Multiple ChoiceMultiple Choice
© 2003 VTECS
Establish Validity . . .Establish Validity . . .
. . . of the Assessment Instruments by:. . . of the Assessment Instruments by:
~ Linking Each Item to a Validated~ Linking Each Item to a Validated
Knowledge/Skills ListKnowledge/Skills List
. . . and of the Assessment Procedures by:. . . and of the Assessment Procedures by:
~ Providing a Test Administration ~ Providing a Test Administration
Manual & MaterialsManual & Materials
© 2003 VTECS
Career Cluster
Assessment
Employability
F09.3
Career Cluster
Assessment
Employability
F09.3
Career Cluster
Assessment
Communications
F02.7
Career Cluster
Assessment
Communications
F02.7
Then….Then….
We Designed the MaterialsWe Designed the Materials
For the Pilot Test Package.For the Pilot Test Package.
Then….Then….
We Designed the MaterialsWe Designed the Materials
For the Pilot Test Package.For the Pilot Test Package.
© 2003 VTECS
What’s in the pilot test package? What’s in the pilot test package?
• Packing Slip
• Test Administration Manual
• Pre-paid “Label” for Material Return
For each Common Skill Assessment:
• Security Checklist
• Test Administrator’s Report Form
• Test Booklets
• Answer Sheets
• Rubric Sheets for Scenarios
• Packing Slip
• Test Administration Manual
• Pre-paid “Label” for Material Return
For each Common Skill Assessment:
• Security Checklist
• Test Administrator’s Report Form
• Test Booklets
• Answer Sheets
• Rubric Sheets for Scenarios
© 2003 VTECS
Test Booklets Test Booklets
& &
Answer SheetsAnswer Sheets
Test Booklets Test Booklets
& &
Answer SheetsAnswer Sheets
Develop and interpret tables, charts, and figures to support written
and oral communications.
Communications F02.7
A Consortium for Innovative Career and Workforce Development Resources
© 002 by2
Career Cluster ProjectAssessment Pilot
STUDENT TEST BOOKLET
Arts, Audio/Video, Technology & Communications
Develop and interpret tables, charts, and figures to support written
and oral communications.
Communications F02.7
A Consortium for Innovative Career and Workforce Development Resources
© 002 by2
Career Cluster ProjectAssessment Pilot
STUDENT TEST BOOKLET
Arts, Audio/Video, Technology & Communications
© 2003 VTECS
A Rubric For Each ScenarioA Rubric For Each Scenario
CriteriaNovice Approaching
ProficiencyProficiency Above
Proficiency
Develop tables, charts and figures to support written and oral communication.
Facts were grouped in a logical order but charted in line or pie chart rather than a bar chart.
Did not document sources of data.
Chart only used lines and was not labeled.
Chart did not include most of the elements to be published—bar chart, color coded with labels, horizontal and vertical axis in readable increments, footnotes and no errors in data interpretation or in spelling.
Compiled facts and arranged in chart but did not use a bar chart.
One source of data was documented.
Bar chart was completed in shades of one color. Axis and bars were not all labeled.
Chart included at part of the elements to be published—bar chart, color coded with labels, horizontal and vertical axis in readable increments, footnotes and no errors in data interpretation or in spelling. Had to be returned for additional work before it met publication standards.
Compiled facts and arranged in an organized manner in a bar chart.
All sources of data were documented.
Bar chart was color coded with labels for each bar. Axis was labeled.
Chart included all the elements to be published—bar chart, color coded with labels, horizontal and vertical axis in readable increments, footnotes and no errors in data interpretation or in spelling.
Compiled facts and arranged in an organized and eye-catching manner in a bar chart.
All sources of data were documented with footnotes or references indicated on chart.
Bar chart was color coded with labels for each bar. Axis were identified in easy to read increments.
Chart included all the elements to be published—bar chart, color coded with labels, horizontal and vertical axis in readable increments, footnotes and no errors in data interpretation or in spelling. Used clever and interesting business style.
COMMUNICATIONS Skill F02.7
COMMUNICATIONS Skill F02.7
© 2003 VTECS
TESTADMINISTRATION
MANUAL
TESTADMINISTRATION
MANUAL
A Consortium for Innovative Career and Workforce Development Resources
© 002 by2
Transportation, Distribution, & Logistics
Career Cluster ProjectAssessment Pilot
Assessment Administration
Manual
A Consortium for Innovative Career and Workforce Development Resources
© 002 by2
Transportation, Distribution, & Logistics
Career Cluster ProjectAssessment Pilot
Assessment Administration
Manual
© 2003 VTECS
PackagePackage
&&
DeliverDeliver
Test MaterialsTest Materials
To Pilot SitesTo Pilot Sites
PackagePackage
&&
DeliverDeliver
Test MaterialsTest Materials
To Pilot SitesTo Pilot Sites
Conduct Conduct Pilot TestPilot TestConduct Conduct Pilot TestPilot Test
© 2003 VTECS
• Score pilot tests Score pilot tests and compile scores.and compile scores.• Score pilot tests Score pilot tests and compile scores.and compile scores.
ScoreScorePilot Pilot TestTest
ScoreScorePilot Pilot TestTest
RevieRevieww
Pilot Pilot TestTest
ResultResultss
RevieRevieww
Pilot Pilot TestTest
ResultResultss
• Perform appropriate Perform appropriate statistical analyses.statistical analyses.
• Perform appropriate Perform appropriate statistical analyses.statistical analyses.
© 2003 VTECS
• Review/revise/delete itemsReview/revise/delete items
• Review/Revise Test Review/Revise Test Administration ProceduresAdministration Procedures
• Review/revise/delete itemsReview/revise/delete items
• Review/Revise Test Review/Revise Test Administration ProceduresAdministration Procedures
Review/ReviseReview/ReviseReview/ReviseReview/Revise
© 2003 VTECS
A Session on the Pilot Test of A Session on the Pilot Test of the Common Skills will be the Common Skills will be presented at: presented at:
For More InformationFor More Information& to Register On-Line& to Register On-LineGo to www.v-tecs.orgGo to www.v-tecs.org
A Session on the Pilot Test of A Session on the Pilot Test of the Common Skills will be the Common Skills will be presented at: presented at:
For More InformationFor More Information& to Register On-Line& to Register On-LineGo to www.v-tecs.orgGo to www.v-tecs.org