facta 6th annual conference "changes in attitude, latitude, and testing" changes in...
TRANSCRIPT
FACTA 6th Annual Conference
"Changes in Attitude, Latitude, and Testing"
Changes in Testing
Opportunities and Risks
Dr. Lou Woodruff
Emerging Directions for Higher Education
Technology Challenging the “Testing Box”
The Future Impact of Off-Line CBT
New Directions in Data Forensics
The Next Big Technology Thing
Changes in Testing – Opportunities and Risks
Expand/Impart Knowledge – Develop Skills
Credential Accomplishments
Mission of Higher Education Institutions
Validate Targeted Learning and Abilities
Expand/Impart Knowledge – Develop Skills
Mission of Higher Education Institutions
There is an increasing recognition that the
knowledge and skills students have developed
outside the college need to be integrated into
their college education
Prior Learning Assessment Programs (PLA)
Credit by Examination Programs
Structures are in place now to validate some knowledge and skills gained outside the college
Digital-Credentials replacing of seat time/credits
Portfolio Evaluation Programs
Placement Testing Programs
Opportunities are expanding to gain knowledge and skills outside the college environment
Massive Open On-Line Courses (MOOCS)
Content focused video modules
On-demand on-line expertise
Social networks as learning tools
Professional Learning Networks (PLN)
Expand/Impart Knowledge – Develop Skills
Credential Accomplishments
Mission of Higher Education Institutions
Validate Targeted Learning and Abilities
Credential Accomplishments
Mission for Selected Higher Ed Institutions
Validate Learning and Abilities
“New”
The College as a Credentialing Institution
No Faculty – Low Overhead – Reduced Service Costs
Charge $ for “credits” awarded
Transfer accomplishments as academic equivalencies
The College provides guidance and resources
Validate broad-based knowledge and skills
Risks for College Testing Programs
Reduced need for traditional testing services
The end of College testing as we know it
The College as a Credentialing Institution
Opportunities for College Testing Professionals
Specialized testing of targeted knowledge/skills
Expertise in portfolio evaluations
Expertise in non-traditional assessments
Another Emerging Higher Education Model
If Higher Education went to Shark Tank
For what Idea might it seek funding?
The College as an Entrepreneurship Institution
In place of degrees, the College would credential
professional portfolios supporting student career paths
Venture capital firms invest in the College in return
for an equity stake in new ventures developed by
students and faculty
The campus offers high-immersion work spaces,
networking, mentorship and business plan
development . . . individualized and self-paced
The College as an Entrepreneurship Institution
Risks for College Testing Programs
No placement testing - No course testing
Massive staffing and budget cuts
Opportunities for College Testing Professionals
Assessment consultant in a business context
Staff with expertise in business evaluation
Integral part of the validation process
The College as an Entrepreneurship Institution
Testing Box
Staying Inside the Testing Box
Commandeering Control
HigherStakes
LowerStakes
Exams
How Much Control . . .
The What
The How
The When
The Where
Testing Box
Staying Inside the Testing Box
Commandeering Control
High stakes standards
were set with P&P
in the 1950’s
The What
The How
The When
The Where
Testing Box
Staying Inside the Testing Box
with the Advances in Electronic Delivery
Stand Alone PCs
Telecom Data (T-1, dial-up)
Server-Based (LAN)
Internet Connected
Internet Delivered
Computer Based Testing
Testing Box
Staying Inside the Testing Box
with the Advances in Electronic Delivery
Encryption
FTP Standards
Data Forensics
Firewalls/Security
Lock-Down Functions
Computer Based Testing
Laptops
Tablets
I-Phones
New Devices
Testing Box
Staying Inside the Testing Box
with Device-Based Testing
On-Line Testing Solution
• Secure the device
• Secure the testing environment
• Secure the data transmission
• Delivery system compatibility
Testing Box
Staying Inside the Testing Box
with Device-Based Testing
“Off-Line” Testing Solution
• Secure the device
• Secure the testing environment
• Secure the data transmission
• Delivery system compatibility
Testing Box
Staying Inside the Testing Box
with Device-Based Testing
“Off-Line” Testing Solution
• Secure the device
• Secure the testing environment
Testing Box
Staying Inside the Testing Box
with Device-Based Testing
Over 2,000,000 high stakes Off–Line tests in 2014
An Encryption Solution
Exams
Test DeliverySystem
Exam Takers
DedicatedExam Web Site
Test AdminSupport
TestCenter
Answer FileUploaded
ExamData Base
Scoring andAdmin Data
Score Reporting
① Advanced encryption of exams integrated with the delivery software
② The secured file, exam and test delivery software, is hosted on dedicated web site
④ At an authorized test center, the exam file
is unlocked and the examinee’s device is fully secured to test without the internet
③ Before the test day, encrypted exam files
and delivery software
are downloaded to test taker’s personal device
⑤ Questions and delivery software are erased after testing. Answers are uploaded when device connects to the internet
⑥ Scoring data sent for score reporting
⑦ Scores reported to exam takers
Off-Line DBT Model
Web
Web
• Campus administrators and faculty will become more aware of and more concerned with cheating
• Course level data forensics will become an expectation on most campuses
• Most course examinations will be delivered on campus using student-owned electronic devices
• The role of college testing professionals will be expanded as a result of these changes
Off-Line Testing Predictions
Data Forensics
D.F. Before the Test !
D.F. During the Test !
After Test Analyses Evidence that test fraud occurred
Data Forensics Before the Test
Imbedded Verification Models
Populating a test with compromised questions,
disguised to lead the perpetrators to reveal
themselves and devalue their brain dump content
Data Forensics During the Test
RT Data Forensic Analyses
Identifying anomalous data patterns during testing, either locally or across a network, with the option in “real time” (RT) to alert the proctor to intervene or to have the test delivery system interrupt that test
Increased Test Security Threats
Increased Test Security Threats
Increased Test Security Threats
Increased Test Security Threats
Increased Test Security Threats
Contact Lens with Camera
Increased Test Security Threats
• become the campus test security expert
• maintain security focused procedures
• promote national security standards
• expand test security expertise
• employ security technology
• implement data forensics
Implications for College Testing Professionals
The Fool-Proof Solution
for All Test Security Concerns
Ban Testing !
Happening Selectively
Already
Big Technology ThingThe next “Big Technology Thing” to change
the way we compute and communicate
and eventually even the way we test
Projection Technology
Projection Technology
2-D Cell Phone Spatial Display
Projection Technology
2-D InteractiveCell Phone Display
Projected Keyboard
Wearable ProjectedCell Phone Display
Projection Technology
Cell Phone 3-D Projection
Projection Technology
Projection Technology
Holographic3-D Projection
Cell Phone Holographic Display
Projection Technology
2D Spatial Computing
Testing Box? ?
Projection Technology
Master the Testing Basics
Learn and Re-Learn the Field
Sustain Continuous Improvement
Pursue Professional Growth
Understand the Implications of Change
Always Exceed Expectations
Collaborate with Colleagues
Today’s Top Seven Take-Aways
