Exams in Prometric

What Should be in Prometric?• Production Exams

– Certification exams for the 6 major courses (BCCPA, BCCPP, BCWAA, BCWAP, BCPSA, BCPSP) – for various releases of the courses such as v311, v321, v331, v341, etc.

– Waiver exams (for Secure Gateway, WAN Acceleration, Visibility) and Troubleshooting exams for Jon’s courses. These may have human-graded questions.

– Quizzes and similar customer-oriented non-certification exams. E.g. a WAN Acceleration Quiz (due today). Could also add quizzes for ProxyAV, Reporter, Director, etc.

• Other Stuff– Legacy exams, if they have historical data of production exam results– Beta exams as a place to measure new test question quality and as staging area to

create new production exams– Sandbox area where to copy existing production exams and do various experiments– Exams that do not fit anywere else – non-English exams, exams created for some other

special purpose.

Visibility and LocationExam Category Visibility Folder

Certification Exams Production (on-line), by voucher

PRODUCTION

Waiver Exams and Troubleshooting Exams

Production (on-line), by voucher

CONTROL RELEASE, sometimes Outside of any folder

Quizzes Production (on-line), by private keycode

PRODUCTION

Legacy Exams Pre-release (offline) END OF LIFE

Beta Exams Production (on-line), by private keycode

CONTROL RELEASE

Sandbox Area Pre-release (offline) or Incomplete (offline)

(Outside of any folder)

Exams that do not fit anywhere else

Production (on-line); or some other status – decided by author

(Outside of any folder)

Not visible for those not concerned

Lifecycle of Production ExamsStage Exam is … By … The deliverable …

1 Written and related files are placed in version control (SVN) – can produce PDF with exam items

Exam author, reviewers and SMEs

PDF or similar content sent to Latha for a sanity check

2 Transformed to Prometric XML and imported (or manually input)

Any Prometric user with admin rights

Prometric Beta Exam in “CONTROL RELEASE” (online – access by keycode)

3 Tested (i.e. ALL items are answered) Some 3-8 nice guys / volunteers

Still Prometric Beta – but with answer statistics

4 Suspicious questions corrected and updated online and in SVN

Exam author Corrected in Prometric, also randomizations within sections are added

5 Configured for production Christine and Thao Moved to PRODUCTION folder, vouchers are generated

6 Used in production Customers More statistics is accumulated in Prometric

7 Reviewed to analyze customer complaints and other problems

Exam author Next exam version or release is updated

8 Taken away from production (when it is fully replaceable by newer versions)

Christine and Thao Moved to END OF LIFE (Pre-release offline). Exam statistics exported and saved for backup/audit purposes

Exam Naming Convention• Since there are several releases of the same exam, these are named

consistently. “Serial #” field is important, because title may coincide• Certification Exam examples:

– bccpa_v331_r1, bccpa_v321_r2• Waiver/Troubleshooting Exams

– bcsg_waiver_v341_r0911, bcwa_waiver_331_0911, bcps_waiver_v321_r0911,

• When a production exam is copied (e.g. to inspect what material is there) add suffix “_COPY”

• While exam is in beta-testing phase, add suffix “_BETA”.

Suggestions for change

• Do not create multiple releases for the same certification exam (if there are urgent corrections, correct them online, or better – just hide the offending exam questions).

• Create separate folders for foreign-language exams, waiver/troubleshooting exams, quizzes.

• For each certification exam/version define “lifespan” – including its expiration date.