welcome workshop goals: understand data sources learn the basic components of an effective...
TRANSCRIPT
![Page 1: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/1.jpg)
Welcome
Workshop Goals:Understand data sources
Learn the basic components of an effective assessment
Analyze and effectively use data
![Page 2: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/2.jpg)
Overview of NOCTI
Over 40 years serving the CTE community Non-profit organization
Owned and operated by the 50 states
Works with both secondary and post-secondary populations
The Whitener Group (a NOCTI subsidiary) provides services to industry
![Page 3: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/3.jpg)
NOCTI Customer Breakdown
![Page 4: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/4.jpg)
Industry Connections
![Page 5: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/5.jpg)
NOCTI Products
Supplemental Products
Over 80 Job Ready tests offered
30 Pathway tests in 10 clusters available
Over 90 private industry employment and/or advancement tests delivered
20 teacher-level assessments
Assessments Study guides for all Job Ready
and Pathway assessments Countdown Kits offered for 40
titles Successful Schools-Successful
Programs Guide
![Page 6: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/6.jpg)
NOCTI Job Ready Assessments
Assess technical skills at the occupation level (e.g., Accounting, Carpentry, Pre-Engineering)
Measure aspects of occupational competence such as factual and theoretical knowledge
Target secondary and post-secondary program completers
![Page 7: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/7.jpg)
Assessment Blueprints
Blueprints include:
Competency List
Content Distribution
Sample Items and Jobs
![Page 8: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/8.jpg)
NOCTI Job Ready Assessment Blueprint
Number of Questions: 222 Areas Covered:
AccountingAccounting
![Page 9: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/9.jpg)
Assessment Blueprints to Prepare
How can I prepare my students for the NOCTI assessment?Assessment Blueprints!!!
Ideas for using blueprints in the classroom:
Use throughout the school year as a syllabus and guide for review
Involve the students: have them calculate the number of questions in each content area
Send a copy home with students to share with parent
![Page 10: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/10.jpg)
Assessment Results
NOCTI’s Standard Score Reporting Package: Individual Score Reports Group Score Reports Comparative Data
Other Reports Available: Task-Link Reports Integrated Academic Reports
![Page 11: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/11.jpg)
NOCTI Standard Score Reports
Individual Score Individual Score Reports Reports
•Provided for each test taker•Scores by duty and total•Pre-/post-test comparisons•Comparative data
Individual Score Individual Score Reports Reports
•Provided for each test taker•Scores by duty and total•Pre-/post-test comparisons•Comparative data
![Page 12: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/12.jpg)
NOCTI Standard Score Reports
Group Score ReportGroup Score Report
•Provided for the testing group•Duty and total scores for each participant
Group Score ReportGroup Score Report
•Provided for the testing group•Duty and total scores for each participant
![Page 13: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/13.jpg)
NOCTI Standard Score Reports
Analysis of Scores Analysis of Scores ReportReport
•Provided for the testing group•Comparative data•Criterion-referenced cut score•Other statistical information
Analysis of Scores Analysis of Scores ReportReport
•Provided for the testing group•Comparative data•Criterion-referenced cut score•Other statistical information
![Page 14: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/14.jpg)
Using Data to Improve Instruction
![Page 15: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/15.jpg)
Using Data to Improve Instruction
Collect Data:Become familiar with
your data
![Page 16: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/16.jpg)
Understanding Your Data
School Program Teacher Class
Student Standards
National State District/System
![Page 17: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/17.jpg)
What the Data Reveal
Achievement gapsDifferences between subgroupsTrends Progress
![Page 18: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/18.jpg)
Assessment
Includes process of collecting, analyzing, and interpreting information
Determines the degree to which students are becoming competent and meeting standards
Occurs at many levels (e.g., classroom and program assessments to state and national assessments)
![Page 19: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/19.jpg)
Good Assessments
Valid Test items measure what they are supposed to
measure: the important content or concepts that were taught and that the student must know
Test is relevant and based on current standards Subject matter experts develop and evaluate
test items
![Page 20: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/20.jpg)
Good Assessments
Reliable Test and its items produce a similar
score over time Scores are consistent, not contradictory Scores are not affected by chance
![Page 21: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/21.jpg)
Student Assessment
On-going with frequent opportunities for feedback Use a variety of different types of assessment (e.g.,
written and performance, formative and summative) during the year
![Page 22: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/22.jpg)
Summative Assessment
Pretests (covering all major content areas) are used to identify entry skills and measure gain
End-of-program or annual test Broader in scope, for comparison of groups
![Page 23: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/23.jpg)
Formative Assessment
Continuous correction from quizzes, homework, projects, participation
Use practice items for instructional revision before summative assessment
![Page 24: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/24.jpg)
Building an Assessment
Test specification matrix or “Blueprint” National Standards Duties and Tasks Written/Performance Weighting Panel of Experts
![Page 25: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/25.jpg)
Step 2
Analyze Data:Draw conclusions from data
![Page 26: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/26.jpg)
Classroom Example
A Carpentry Class
![Page 27: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/27.jpg)
Areas on a NOCTI Carpentry Test
Safety
Tools and Accessories
Blueprint Reading and Estimation
Foundations, Forms and Concrete
Rough Framing
Exterior Finish
Interior Systems Installation
Interior Finish
Carpentry Related Mathematics
![Page 28: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/28.jpg)
Individual Carpentry Pre-Test
Carpentry Pre-test Areas Pre-test % CorrectStudent 1
Post-test % CorrectStudent 1
Safety 54.6 --
Tools and Accessories 30.0 --
Blueprint Reading and Estimation 41.1 --
Foundations, Forms, and Concrete 36.4 --
Rough Framing 53.8 --
Exterior Finish 36.4 --
Interior Systems Installation 40.0 --
Interior Finish 27.3 --
Carpentry Related Mathematics 35.9 --
Total 40.0
![Page 29: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/29.jpg)
Carpentry Class Pre-Test
Carpentry Pre-test Areas
Student 1 % Correct
Student 2 % Correct
Student 3 % Correct
Student 4 % Correct
Student 5 %
Correct
Student 6 %
Correct
Class Average % Correct
Safety 54.6 45.5 36.4 27.3 27.3 54.5 40.9
Tools and Accessories
30.0 20.0 10.0 20.0 10.0 30.0 20.0
Blueprint Reading and Estimation
41.1 17.6 23.5 23.5 29.4 35.3 28.4
Foundations, Forms, and Concrete
36.4 54.5 45.5 45.5 36.4 54.5 45.5
Rough Framing 53.8 38.5 30.8 38.5 30.8 53.8 41.0
Exterior Finish 36.4 36.4 27.3 36.4 36.4 45.5 36.4
Interior Systems Installation
40.0 60.0 40.0 60.0 60.0 80.0 56.7
Interior Finish 27.3 9.1 18.2 27.3 9.1 27.3 19.7
Carpentry Related Mathematics
35.9 45.5 27.3 36.4 36.4 54.5 39.4
Total 40.0 34.0 28.0 33.0 29.0 46.0 35.0
![Page 30: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/30.jpg)
Carpentry Class Post-Test
Carpentry Post-test Areas
Participant Group Site State Nation
Safety 100.0 86.4 79.5 72.5 75.5
Tools and Accessories 91.7 79.2 78.2 70.7 70.6
Blueprint Reading and Estimation
100.0 95.0 83.8 64.3 66.4
Foundations, Forms, and Concrete
92.3 69.2 65.8 51.6 52.9
Rough Framing 97.4 71.8 69.8 63.1 64.1
Exterior Finish 91.7 64.6 67.6 58.1 57.8
Interior Systems Installation 87.5 53.1 61.9 55.6 56.5
Interior Finish 87.5 79.7 75.6 66.6 67.0
Carpentry Related Mathematics
100.0 56.3 68.2 66.1 67.0
Total 94.3 72.0 71.4 62.7 63.4
![Page 31: Welcome Workshop Goals: Understand data sources Learn the basic components of an effective assessment Analyze and effectively use data](https://reader035.vdocuments.us/reader035/viewer/2022062511/5514e05d550346935c8b55e5/html5/thumbnails/31.jpg)
Thank You!
Questions?
www.nocti.org