eatp- using a blended delivery model to drive strategic success for sap certification

Using a blended test delivery

model to drive strategic

success for SAP certification

o Susan Martin

• SAP

• susan.martin@sap.com

o Ralf Kirchgaessner

• SAP

• ralf.kirchgaessner@sap.com

o John Kleeman

• Questionmark

• john@questionmark.com

Speakers

1. Learn about SAP’s global,

multilingual certification program

2. Appreciate challenges of

administering a truly global IT

certification program

3. Grasp practical issues and

benefits in implementing

blended delivery in certification

Objectives

o Introduction

o SAP Certification Business Value

o Details about current SAP certification Program and its 2 Delivery Channels

o What do we want to achieve with an integrated channel?

o Blended delivery possibilities

o Questions

Agenda

SAP at a glance: business

overview

SAP at a glance: business overview SAP is the world leader in enterprise applications in terms of software and software-related service revenue. Based on market capitalization, we are the world’s third largest independent software manufacturer. ---------------------------------------- More than 183,000 customers in 130+ countries. More than 55,765 employees in 130+ countries. --------------------------------------- A 40-year history of innovation and growth. -------------------------------------- Annual revenue (IFRS) of € 14,23 billion. ------------------------------------- Listed under the symbol "SAP" on stock exchanges, including the Frankfurt Exchange and NYSE.

SAP Education

350,000 people certified

55,000+ people trained

via eLearnings & eAcademies

400+ virtual, live,

classroom titles In

9 languages

600+ courses

available

Training provided

in 40 countries Leader in Worldwide IT Training & Education*

300,000 individuals trained annually

2,200+ partner

companies engaged in training & certification of their staff

~420 eLearning/eAcademy titles available

29,000+ trainings

delivered to partner consultants

Up to 440 events per

week delivered via cloud infrastructure

* Source: IDC MarketScape: Worldwide IT Education & Training 2012 Vendor Analysis

SAP Education Continuous Evolution of Our Offerings

2010 & Before

2011 2012 2013 2014 2015

Traditional Classroom Training

Flexible Online Learning

SAP Learning Hub

SAP Workforce Performance Builder

SAP Education – Mobile Learning

SAP Innovation Curriculum (HANA, Mobility)

SAP Education – Social Learning

Traditional Flexible & Innovative Mobile & Social

SAP Education’s

Certification Program

Role-based

Certifications specific not only to a particular

solution but to the type of individual certified and

their job tasks

Multi-level

Designed to stratify the skills assessment and

provide certification to various levels of

practitioner

Focus on plan – build – run

Practical, functional assessments

Flexible delivery

Ensuring exams are available to all regardless of

geographic constraints

Ce

rtif

icat

ion

Pro

gram

SAP‘s Global Certification

Program

Creati

on

Administration

Delivery

Reporting

Quality

assurance

Certification

Results-

management

200 + WTS + onsite + 4000 VUE

# Locations

Proctored (SAP & VUE)

2 Channels

~ 50,000 # Candidates / year

> 100 in up to 20 lang.

# diff. Tests

1995

Consultant

Certification

Established

Enable effectively the requirements for the new Certification Program of SAP Education Services, including:

o Certification Infrastructure evaluation and implementation, incl. interfaces into existing SAP infrastructure

o Evaluation and definition of changes in Production Processes for certifications

o Implementation of new Delivery Processes and Operations Model

o Migration / transformation scripts for existing tests, and test items

o Phased implementation, including infrastructure rollout, and training of proctors in SAP subsidiaries, Customer Sites and / or Educational Partners

Selection & Implementation

of Questionmark

High-Level Overview

Multilingual Program

Global Program with up to 20 languages

Esp. important for Midmarket

SAP Language Services translates TMS projects

Challenge: Transfering test content in multi languages

Languages

en-USde-DEes-ESpt-BRja-JPru-RU

Two Delivery Channels

o Reasons • Coverage, e.g. North America, ANZ

• Flexible Education Delivery Models (E-Academies)

• Midmarket Audience

o Tasks • Content Handover

• Client Review

• Results Upload

• Fulfillment with External Vendor

o Challenges • Publishing Costs

• Time to market

• Translation

VUE Delivery Channel

Intranet & WTS

SAP Training Locations

Database

Application

Production System

Assessment, Participant, Scheduling and Results

Database

Application

Content Authoring System

Assessment, Participant, Scheduling and Results

QM Authoring Managers

Certificate Issuance

Scheduling Results

SAP LSO

Event Booking and Scheduling Certification Issuing

Scheduling (Internal)

Assessments

Pearson VUE

F I R E W A L L

Results in Excel file format

Results

Email to Participants confirming successful registration

Internet

Logon details sent to Participants & Proctors

Certificate Issuance

F I R E W A L L

Data Migration/Transformation to Pearson VUE format

Certification Delivery Matrix

Event-Based Certification Individual Certification

SAP Training Center Customer/ Partner Location

Pearson Vue Test Center

Target Audience: Cust Part Ind SAPC Cust Part Cust Part Ind SAPC

Proctoring SAP Proctor SAP Proctor onsite External Proctor

Opportunity Costs - Travel - Time required

Med* Med* Med* Low* Low Low Med Med Med Med

Flexible Delivery Low Low Low Low Med Med High High High High

Availability Scheduled event On request 10/5

*Opportunity costs are lower if exam taker attends training at SAP training center upfront

o Integration and Harmonization of the 2 Delivery

Channels

o Idea is to connect out of Pearson VUE testing

centers to SAP Questionmark Perception

systems, instead of using the Pearson VUE

Athena delivery engine.

• @

Future: QM@VUE

Benefits

Category Benefit

Publishing Save costs

Time to market Exams available at VUE in parallel to

SAP, no delay anymore

Reporting / Resultmanagement

Easier / faster result import Combinded item statistics possible

User Experience Same UI for participants

Translation Exams available in all languages at

once

Market Penetration Increased volume in regions

due to more languages

o A Questionmark view

Blended delivery and

certification

Impersonation Content Theft Cheating

Key Threats to be

Addressed in High-stakes

Exam Delivery

Pre-employment

Public

Certifications

& Licensing

Regulatory Compliance

Sales and Technical Channel

Verification

Threat Level in Higher Stakes

Short Term with Low Trust Relationships

Long Term with High Trust Relationships

Larg

e P

rogr

ams

Smal

l Pro

gram

Higher threats require more

“Oversight” and so cost more

Lower threats require less “Oversight”

and so cost less to administer

Formative Quizzes Course Evaluations

Diagnostic Tests

Low Stakes Tests

Medium Stakes Exams

High Stakes Exams

Levels

of M

onitoring

Higher stakes, more monitoring

Stakes of Assessment

Monitoring Tests

Securely

1. I need to deliver on multiple platforms

• Third party test centers

• Video monitoring

• In own training centers and offices

• Mobile test centers

• Web-based testing

2. I don’t want to author, review & translate

separately for each platform

3. I need results amalgamated

Test publisher challenge

Deliver to any device in context

Single results set

Questionmark Blended Delivery

Author once

Available Securely

SME

Windows based

Authoring Manager

Same assessment, different delivery

methods

Secure Browser

Web Browser Questionmark to Go

LMS

Integration

Windows / CD

Test Centers

Printing and Scanning

Mobile Devices

Social Networking Platforms Embedded in Wikis,

Blogs, and Web pages

Questionmark integrates with regional and global test center operators

Assessment scheduled via • Either/or

o Test center provider

o Questionmark

Assessment delivered by • Browser at test center

• connected to Questionmark data center

Results combined with results from other delivery

Bricks and mortar test center

“Online monitoring of exams is

coming of age as higher internet

speeds enable this technology”

• Eric Shepherd, Questionmark CEO

Questionmark offers video

monitoring with ProctorU

Video Monitoring

Deliver certification exams at a

conference or trade show

• Mobile devices

• Portable PCs

Assessments “on location”

Mobile test centers

Author once, deliver anywhere

A new delivery channel just

needs a new “driver”

SAP likely to take advantage of

these capabilities to improve

programme

Key concepts

Questions

Thank You!

Please Complete Your Evaluation and Hand to the Session Coordinator.