Cheater Cheater

Pumpkin Eater

How to Beat the Cheat

PROTECT

ENFORCE

INVESTIGATE

Victoria Quinn-Stephens

Career Certifications

Program Manager

Cisco

Peggy Crowley

Anti-Piracy Program Manager

Microsoft

Liz Burns

EMC Proven Professional

Program Manager

EMC

John Fremer

President

Caveon Test Security

Session Moderator

PROTECTVictoria Quinn-Stephens

Career Certifications

Program Manager

Cisco

Cisco Certifications

• Three levels of certification:

– Associate (ex: Cisco Certified Network Associate - CCNA)

– Professional (ex: Cisco Certified Security Professional – CCSP)

– Expert: (ex: Cisco Certified Design Expert –CCDE)

• Target markets:

– Customers, Partners, Cisco Employees

– Educators and Students

• Certified one millionth candidate in February ’08

Cisco Certifications

Exam security policies driven by:

– Diverse global constraints and objectives

– Aspiration to maintain integrity of program

• Credibility with customers

• Value to employers

• Platform for career development

Ensuring that those who achieve a Cisco

certification possess the requisite skills and

knowledge

Cisco: Methods of Security Planning

Detection

Reporting

Communications

Prevention

Enforcement

Security

Threats

Brain

dumps

Proxy

TestingCheating

Cisco: Protect

• CHIP Security Pilot [China, Hong Kong, India & Pakistan]

– Test new channel security features in high risk countries

– Communicate stricter security standards in CHIP

– Measure results in a more controlled environment

– Survey candidate response to increased security

• Single vendor migration to VUE 8.1.07

– Partner for continuous security improvements

– Innovate new security solutions

– Scale security protections to market expansion

Cisco: Protect

A comprehensive

strategy that

integrates proactive

measures and best

practices.

• Harden testing channel

• Improve agreements

• Build better exams

• Institute continuous

improvement plan

• Support industry security

initiatives

Cisco: Protect

• Harden Testing Channel– Continual TCA training and re-certification

requirements

– Candidate authentication process [“CAP”]

– Real time data forensic enabled results hold feature

– Photo on score report

• Delayed [after forensic & psychometric analysis]

• Photo ID included on score report

– Certification credentials & photo verifications on web

– Increase volume of integrity shops

– Security country zones for test centers

– Systemic retake policy

– Test center security signage & tip line promotion

Cisco: Protect

• Improve Agreements– Candidate NDA

– Candidate testing rules

– Candidate conduct policy

– Candidate code of ethics

– Exam policies

– Retake policies

– Penalties and appeals process

– Test center policy & procedure guidelines

– Testing vendor contracts

Cisco: Protect

• Build better exams– Build more performance-based testing

– Add more complex simulations

– Randomize answer option [multiple choice]

– Clone items

• Increases amount of content that can be

developed in a shorter period of time

• Watermark cloned items – unique identification

– Increase rate of content development

• More content increases difficulty of memorization

– Localization of exam content

Cisco: Protect

• Institute continuous improvement plan– Create comprehensive security plan

– Implement yearly security audits

– Meet with cross-functional teams for input &

collaboration

– Critically assess customer sat, test center and

partner feedback or survey findings

– Schedule business reviews w/testing vendor

– Schedule security summits w/testing vendor

Cisco: Protect

• Support Industry Security Initiatives– Back industry’s leadership role to:

• Act as a clearinghouse

• Conduct outreach

• Police braindump activity

• Take enforcement actions

INVESTIGATE

Liz Burns

EMC Proven Professional

Program Manager

EMC

EMC Proven Professional

• Business realities that drive exam security

policies

– Global exam delivery

– VUE and Prometric for exam distribution

– Exam Volume

• 1000+ exams per month

– No authorized partner channel

– 50% EMC employees

EMC Investigative Approach

• To identify websites– Monthly

• Search EMC Exams on Google and eBay. Send take down request.

• Web Crawl report on new websites. Block from EMC network

• Forensic analysis on high volume exams. Various actions

– Daily• 24x7 web alerts. Various actions

• To identify individuals and test centers– Monthly

• Forensic analysis on high volume exams. Various actions

– Daily• 24x7 web alerts. Various actions.

Objective – Regular, Repeatable Processes and Measurable Results

Case Studies

• EMC sees a reduction in monthly exam

fraud incidents when we do regular

investigation and enforcement

• Case Study One – Using web alerts to

identify when, where and who steals exams

• Case Study Two – Using exams to identify

test takers who are using downloaded exams

1) EMC publishes to test vendors:

E20-500 on 9/10/07

E20-381 on 10/29/07

2) Both exams appear on web:

11/8/07

3) EMC orders

exams on 11/8/07

4) Monitor exam

registration activity

5) Exams stolen 11/10/08

from India testing center

Testing center

closed

Pros and Cons to Process• Cons

– Does not prevent current exam theft

• Pros– Does prevent future exam theft

• No new EMC exams on web since November 07

• When attempted to purchase other new exams the website operator replied:

“Dear:

Not technical problem, the reason from the test center. Whenever we download any EMC exam, after this EMC exam will be removed by prometric . So now we cannot got the emc exam data, so we also cannot do the exam for you. Thanks

Developed process to identify individuals

memorizing stolen exam content to pass exam

but who do not understand content

Monthly

forensics

Individuals Identified

and action taken

Pros and Cons to Process• Cons

– Again, does not prevent current exam theft

or cheating

• Pros

– Does identify individuals who should retest

or be banned from Program

– Does send a message to testing

community

ENFORCE

Peggy Crowley

Anti-Piracy Program Manager

Microsoft

Microsoft Learning APEducation

• To warn candidates of potential dangers

• To educate candidates of proper procedures

• To deter “bad guys”

Engineering• Striving to use innovations

in exam security

• Exploring other Microsoft technologies for delivery

Enforcement

• Legal Actions

• Security Issues

Intelligence

• Data Forensics

• Metrics to drive the other pillars

• Metrics to justify resources

Combating Piracy With…

Legal Enforcement Process

•Lead Tracking

•Lead Qualifying Process

•Test Purchase Program

•Actions•C&Ds

•Takedown Notices

•Internet Monitoring Program

•Auction Monitoring Program

•Lawsuits

Case Study: TestKing• Case filed August 2006

– Started as a John Doe suit

• Settled May 2007

– immediately cease marketing, selling, distributing, publishing, reproducing, disseminating, offering or otherwise knowingly transferring in any way any actual Microsoft Certification Exam content

– a permanent injunction prohibiting distribution of infringing material

– post the following notice on all affiliate websites: “Testking Materials do not contain actual questions and answers from Microsoft’s Certification Exams”

Questions?

Don’t forget to fill out your evaluations!