caveon webinar series - the test security framework- why different tests need different test...

The Test Security Framework: Why different tests need different test security requirements

June 29, 2016

Caveon Webinar Series

www.caveon.com2

Today’s Presenters

Rachel Schoenig, ACTAVP and Head of Test Security

Jennifer Geraets, ACT

Senior Test Security Analyst

Jamie Mulkey, Caveon

VP, Client Services

3

Agenda

• Division and Project Background• Process• Work to Date• Next Steps

www.caveon.com

4

ATP Workforce Skills Credentialing Division

ATP Workforce Skills Credentialing Division

1. Security and Privacy Subcommittee

2. Initiative Outreach Taskforce

www.caveon.com

5

ATP Workforce Skills Credentialing Division

• Workforce: The people engaged in or available for work, either in a country or area or in a particular company or industry.

• Credential: The evidence of mastery, competence, or achievement of a specific topic or skill that is under the control of the individual.  

www.caveon.com

6

Security & Privacy Subcommittee

1. Survey of workforce skills credentials and test security

2. Framework for evaluating security and privacy of credential offerings

www.caveon.com

7

Workforce Shifts

Shrinking Workforce

Sources: Pew Research Center; United Nations; Organisation for Economic Cooperation and Development; International Politics and Society

www.caveon.com

8

Workforce Shifts

Reliance on younger generations

Sources: Worldwatch Institute; Organisation for Economic Cooperation and Development

www.caveon.com

9

Workforce Shifts

Migration

Sources: Organisation for Economic Cooperation and Development

www.caveon.com

10

Workforce Skills are Important

www.caveon.com

11

Now what?Access

to educatio

n Better skills and

jobs matches Formal

recognition of skills

www.caveon.com

www.caveon.com12

Incomplete Equation

Skills / Competency Content

TrustworthyPortableCredentials

13

Access to Opportunity

www.caveon.com

www.caveon.com14

15

Access to Opportunity

$$ Thousands $$

www.caveon.com

16

Access to Opportunity

FAKE Credentials are the OPPOSITE

Source: Business Insider

www.caveon.com17

Complete Equation

ContentTrustworthy

PortableCredentials

Test Security

18

ATP Workforce Skills Division

Framework of Test Security

www.caveon.com

19

Process

Testing Compan

yEmploy

eeEmploy

er

www.caveon.com

20

Examinee Perspective• Value

– What does this credential say about my skills?

• Fairness – What does everyone else have to do to

earn this credential? – Is this a fair process?– What happens if someone cheated?

www.caveon.com

21

Credential User Perspective

• Trustworthiness– Did the individual earn the credential

and have the required skills? – How can I authenticate the credential?– How will I know if a credential is

revoked? • Portability and Value

– Who administered the test and what brand is attached to the credential?

– Who else relies on the credential?www.caveon.com

22

Methodology

1. Test Taker Identification2. Test Administration3. Test Design4. Results Validation and Authentication5. Investigation and Remediation6. Sustainability

www.caveon.com

23

Methodology

One Size Fits All

www.caveon.com

24

Methodology

• Surveys – Employees– Employers– Test publishers

www.caveon.com

25

Work to Date• Framework

– 6 categories– 4 tiers– Capabilities / tools at each tier

• Transparency• Self-identification

www.caveon.com

26

Applying the Framework

Hypotheticals

www.caveon.com

27

Credential Security – Pre-Employment testing

• JRJ Manufacturing company needs entry level workers to operate a simple piece of machinery.

• They offer candidates a pre-employment reading test to assess comprehension and the ability to follow directions.– The test is given online, without any proctoring– Score results are used with other factors to

determine if the candidate moves to the next round of employment interviews.

www.caveon.com

28

Using the FrameworkTest Administration – Test Taker

I know my credential has value and that results will be reliable if I follow the rules I have been told the testing rules in

advance and know I am expected to follow all rules concerning appropriate testing conduct

I am able to take the test on my own I may or may not be proctored during

testing

I know my credential is valuable and that testing is fair because there are strong standardized test administration requirements.• I have to test at a designated, secure

testing location• I have been told the rules in advance

and know I am expected to follow all rules concerning appropriate testing conduct

• I am prohibited from bringing unauthorized aids and electronics (digital devices) into the testing room and expect additional security to prevent me or others from doing so

Level 1 Level 4

29

Credential Security – Medical Device Certification• The new JRJ hand-held medical

sonogram is being marketed in the US and Europe but a credential is required for use– Individuals taking the sonogram device

training already have medical credentials (MD, RN, RDMS)

– A computerized test is completed at the end of training course completion. Scores are recorded and tracked.

– The test is monitored by the instructor who taught the course.

www.caveon.com

30

Using the FrameworkTest Taker Identification – Test PublisherLevel 1 Level 4Minimal identification and authentication tools Examinee self-identifies Credential issuer may seek to

match identification through other tools (such as Facebook or Twitter accounts)

Credential issuer will seek to verify identification if questions arise

Strong identification and authentication tools• Identification is established by

requiring government-issued photo identification

• Identification is further established via biometrics tools, challenge questions and presentation of government-issued photo identification

• Test taker image is captured at time of testing

31

Challenges• Communicating the Framework• Getting credential users and examinees

to request and use the Framework levels

• Will users see the Framework’s flexibility?

• Consulting the Framework before test development

www.caveon.com

32

Next Steps

Work in Process• Completion & publication by ATP 2017• Privacy Framework is next

www.caveon.com

33

Next Steps

Reviewers needed!

www.caveon.com

www.caveon.com34

Complete Equation

ContentTrustworthy

PortableCredentials

Test Security

www.caveon.com35

Q&A

www.caveon.com36

Thank You! Follow Caveon on twitter @caveonCheck out our blog www.caveon.com/blogLinkedIn Group “Caveon Test Security”

Rachel SchoenigRachel.Schoenig@act.org

Jennifer Geraets Jennifer.Geraets@act.org

Dr. Jamie MulkeyJamie.Mulkey@caveon.com