caveon webinar series - the test security framework- why different tests need different test...
TRANSCRIPT
The Test Security Framework: Why different tests need different test security requirements
June 29, 2016
Caveon Webinar Series
www.caveon.com2
Today’s Presenters
Rachel Schoenig, ACTAVP and Head of Test Security
Jennifer Geraets, ACT
Senior Test Security Analyst
Jamie Mulkey, Caveon
VP, Client Services
3
Agenda
• Division and Project Background• Process• Work to Date• Next Steps
www.caveon.com
4
ATP Workforce Skills Credentialing Division
ATP Workforce Skills Credentialing Division
1. Security and Privacy Subcommittee
2. Initiative Outreach Taskforce
www.caveon.com
5
ATP Workforce Skills Credentialing Division
• Workforce: The people engaged in or available for work, either in a country or area or in a particular company or industry.
• Credential: The evidence of mastery, competence, or achievement of a specific topic or skill that is under the control of the individual.
www.caveon.com
6
Security & Privacy Subcommittee
1. Survey of workforce skills credentials and test security
2. Framework for evaluating security and privacy of credential offerings
www.caveon.com
7
Workforce Shifts
Shrinking Workforce
Sources: Pew Research Center; United Nations; Organisation for Economic Cooperation and Development; International Politics and Society
www.caveon.com
8
Workforce Shifts
Reliance on younger generations
Sources: Worldwatch Institute; Organisation for Economic Cooperation and Development
www.caveon.com
9
Workforce Shifts
Migration
Sources: Organisation for Economic Cooperation and Development
www.caveon.com
10
Workforce Skills are Important
www.caveon.com
11
Now what?Access
to educatio
n Better skills and
jobs matches Formal
recognition of skills
www.caveon.com
www.caveon.com12
Incomplete Equation
Skills / Competency Content
TrustworthyPortableCredentials
13
Access to Opportunity
www.caveon.com
www.caveon.com14
15
Access to Opportunity
$$ Thousands $$
www.caveon.com
16
Access to Opportunity
FAKE Credentials are the OPPOSITE
Source: Business Insider
www.caveon.com17
Complete Equation
ContentTrustworthy
PortableCredentials
Test Security
18
ATP Workforce Skills Division
Framework of Test Security
www.caveon.com
19
Process
Testing Compan
yEmploy
eeEmploy
er
www.caveon.com
20
Examinee Perspective• Value
– What does this credential say about my skills?
• Fairness – What does everyone else have to do to
earn this credential? – Is this a fair process?– What happens if someone cheated?
www.caveon.com
21
Credential User Perspective
• Trustworthiness– Did the individual earn the credential
and have the required skills? – How can I authenticate the credential?– How will I know if a credential is
revoked? • Portability and Value
– Who administered the test and what brand is attached to the credential?
– Who else relies on the credential?www.caveon.com
22
Methodology
1. Test Taker Identification2. Test Administration3. Test Design4. Results Validation and Authentication5. Investigation and Remediation6. Sustainability
www.caveon.com
23
Methodology
One Size Fits All
www.caveon.com
24
Methodology
• Surveys – Employees– Employers– Test publishers
www.caveon.com
25
Work to Date• Framework
– 6 categories– 4 tiers– Capabilities / tools at each tier
• Transparency• Self-identification
www.caveon.com
26
Applying the Framework
Hypotheticals
www.caveon.com
27
Credential Security – Pre-Employment testing
• JRJ Manufacturing company needs entry level workers to operate a simple piece of machinery.
• They offer candidates a pre-employment reading test to assess comprehension and the ability to follow directions.– The test is given online, without any proctoring– Score results are used with other factors to
determine if the candidate moves to the next round of employment interviews.
www.caveon.com
28
Using the FrameworkTest Administration – Test Taker
I know my credential has value and that results will be reliable if I follow the rules I have been told the testing rules in
advance and know I am expected to follow all rules concerning appropriate testing conduct
I am able to take the test on my own I may or may not be proctored during
testing
I know my credential is valuable and that testing is fair because there are strong standardized test administration requirements.• I have to test at a designated, secure
testing location• I have been told the rules in advance
and know I am expected to follow all rules concerning appropriate testing conduct
• I am prohibited from bringing unauthorized aids and electronics (digital devices) into the testing room and expect additional security to prevent me or others from doing so
Level 1 Level 4
29
Credential Security – Medical Device Certification• The new JRJ hand-held medical
sonogram is being marketed in the US and Europe but a credential is required for use– Individuals taking the sonogram device
training already have medical credentials (MD, RN, RDMS)
– A computerized test is completed at the end of training course completion. Scores are recorded and tracked.
– The test is monitored by the instructor who taught the course.
www.caveon.com
30
Using the FrameworkTest Taker Identification – Test PublisherLevel 1 Level 4Minimal identification and authentication tools Examinee self-identifies Credential issuer may seek to
match identification through other tools (such as Facebook or Twitter accounts)
Credential issuer will seek to verify identification if questions arise
Strong identification and authentication tools• Identification is established by
requiring government-issued photo identification
• Identification is further established via biometrics tools, challenge questions and presentation of government-issued photo identification
• Test taker image is captured at time of testing
31
Challenges• Communicating the Framework• Getting credential users and examinees
to request and use the Framework levels
• Will users see the Framework’s flexibility?
• Consulting the Framework before test development
www.caveon.com
32
Next Steps
Work in Process• Completion & publication by ATP 2017• Privacy Framework is next
www.caveon.com
33
Next Steps
Reviewers needed!
www.caveon.com
www.caveon.com34
Complete Equation
ContentTrustworthy
PortableCredentials
Test Security
www.caveon.com35
Q&A
www.caveon.com36
Thank You! Follow Caveon on twitter @caveonCheck out our blog www.caveon.com/blogLinkedIn Group “Caveon Test Security”
Rachel [email protected]
Jennifer Geraets [email protected]
Dr. Jamie [email protected]