measuring academic achievement in career-technical education
DESCRIPTION
Measuring Academic Achievement in Career-Technical Education. Presented by: Sandra Pritz, Senior Consultant National Occupational Competency Testing Institute. Rationale for Standardized Assessments— technical and academic. External objective data of student achievement - PowerPoint PPT PresentationTRANSCRIPT
Measuring AcademicAchievement in Career-
Technical Education
Presented by:
Sandra Pritz, Senior Consultant
National Occupational Competency Testing Institute
Rationale for Standardized Assessments—
technical and academic
• External objective data of student achievement
• Comparative data for schools, districts, states, and across states
• Perkins and NCLB core measures
• May be used by employers and postsecondary institutions
Value Added To Change What Is Taught and How to Teach
• “The administrator sits down with teachers and goes over the results to determine changes that may be needed in the curriculum.” (Teacher Sandra Himes, PA)
• “I love the data and detail to be able to improve instructional strategies.The results are charted, show technical gains. If the group shows highs and lows, you can work on the lows. Teachers see where they missed and work on different approaches.” (Administrator Ed Haynor, MI)
Why should career-technical programs assess academic
skills?• State/national academic standards demand
attention.• Career-tech is accountable for academic gains.• High school students must pass academic tests
for graduation.• Academics in context tend to be student-friendly • Workplace demands for technical literacy are
increasing.
Technical Literacy: integrated academics and technical content
at basic, proficient,and advanced levels
• Read, understand, and communicate in the language of a career field
• Understand technical concepts and principles
• Use academic knowledge and skills to solve problems
• Use basic technology
Academic Skills Measured in Context
Academic Skills
+ Technical Skills
+ Employability Skills
Comprehensive Assessment
AcademicSkills
EmployabilitySkills
TechnicalSkills
How Should Career-Technical Programs Assess Academic
Skills?
• In the context of the students’ career-technical program
• In alignment with state and national academic standards
• In alignment with post-secondary program requirements
• Embedded In their career-technical tests, such as end-of-course or program
Testing Academic Skills in Context
• Math, science, and language arts are required to complete most occupational items and performance tasks.
• NOCTI tests occupational competencies, but also tests embedded academics.
• Embedded academics are correlated to national standards (NCTE, NCTM,NSES).
• Student scores can be isolated by discipline.
Academics Identified• Written tests contain embedded
academics that match standards in all three disciplines.
• Performance tests contain a higher percentage of embedded academics than written tests.
• Students tend to score lower on the written than on the performance test, and the items most often missed involve technical literacy.
Ways NOCTI can help
• By providing performance as well as written tests, NOCTI can help assess academics at higher levels.
• NOCTI can respond to the need to reinforce academic instruction through assessment, can customize tests and test reports.
• NOCTI can provide states the service of cross-correlating its tests to their state standards.
Agricultural Mechanics Example from the Agricultural Electrical Power & Processing Duty:
The completed path through which electricity flows is called the
A. fuse B. circuit C.circuit breaker D. ground wire
- NCTE Language Arts: Apply strategies to comprehend, interpret, evaluate, and appreciate texts. Use experience, interactions with readers/writers, meaning of words, word identification , and features, e.g., sound-letter correspondence, sentence structure, context, graphics.
- National Science Education (NSES) Standards:Transfer of energy
Workplace Readiness Example: You earn $8/hour and work 40 hours/week. You receive time and a half for overtime and you worked five overtime hours last week. What was your gross pay last week?
A.$350 B.$360 C.$380 D.$390
- NCTM Math: Select/use various types of reasoning and methods of proofs; Apply/adapt a variety of appropriate strategies to solve problems.
- South Dakota Math, grades 9-12:Use numbers in a variety of equivalent forms to solve problems; Use properties of the number system to solve problems.
- South Dakota Language Arts, grades 9-12 :Use various reading cues/strategies to comprehend text
PA Assessments…
• Math Standards Alignment– 8th, 11th, and National
Standards– Standardized paper
and pencil doesn’t fit everyone—use written and performance.
– Students show improvement using CTE methodologies.
• Criterion Referenced Scores– Advanced, Proficient,
Basic, Below Basic– Aligns with State
Assessment– Top 20 Tests & 200
Pennsylvania Employers
Other IA Assessments in demonstration projects…
• South Dakota – Workplace Readiness
and Carpentry– Language arts, math ,
and science– State and national
standards– 8th and 9-12th grade
standards– Written and
performance
• Oklahoma– 27 different tests on
11 campuses– 14 math and 10
science selected– State standards with
national as available– 9-12th grade
standards– Written at all sites,
performance at sites per their selection
What Do I Receive from NOCTI?
• Score Reports– Coordinator’s Report– Individual Student Reports– Customized Reporting Available—
e.g. embedded academics
• Statistical Information– Group Data– Site Data– State Data– National Data
• Student Certificates
• Personalized Contact Assistance
All results, including integrated
academics, are broken down into
duty, category and total scores.
Note:
Job Ready Welding Written Assessment
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
The
rmal
Cut
ting
Pro
cess
es
Hea
tT
reat
/Met
allu
rgy/
Dis
tort
ion
Con
trol
Wel
ding
Ter
ms/
Blu
eprin
tR
eadi
ng
Wel
ding
Insp
ectio
n
Wel
ding
Con
sum
able
s
Saf
ety
Pow
er S
ourc
e P
rinci
ples
Oxy
fuel
Wel
ding
&B
razi
ng
Shi
elde
d M
etal
Arc
Wel
ding
Gas
Met
al A
rc &
Flu
xC
ored
Arc
Wel
ding
Gas
Tun
gste
n A
rcW
eldi
ng
Mea
surin
g In
stru
men
ts
Wel
ding
Qua
lific
atio
ns &
Pro
cedu
res
Wel
ding
Cos
ts
Tot
al
Pre-Test Scores Post-Test Scores
Pre and Post Test Gain Analysis
Program Improvement
• Focus on industry standards – Plan program and curriculum changes– Assign “just-in-time” lab projects
• Expand emphasis on technical literacy at proficient levels– Reading career -technical material– Interpreting technical information– Calculating in technical problems
Student Achievement Improvement
• Identify gaps—group and individual-- and target instruction to them
• Heighten awareness of responsibility for learning per strengths/weaknesses
• Focus on academic and employability skills
Workplace Readiness Test
• NOCTI Workplace Readiness Test measuring employability skills is now up-scaled.
• A large and stellar Subject Matter Expert Team has culled and rated the critical competencies.
• Career Cluster Foundation Skills are assessed.
Workplace Readiness Assessment
Communications Systems
Information Technology Applications
Problem Solving and Critical Thinking
Safety, Health and Environmental
Leadership, Management and Teamwork
Ethics and/or Legal Responsibilities
Employability and/or Career Development
Workplace Readiness Assessment
82 item multiple-choice assessment; parallel forms with .94 internal consistency/reliability
41-item version to accommodate single-period schedules; no sub-scores; .96 reliability
Career Skills middle school 41-item version
Online delivery format available for all versions
To discuss your interests,visit
www.nocti.org, call 1-800-334-6283, or
write [email protected]
Thank You