A Self Service Solutions Benchmark(Public Sector)An Urchin-based framework for a benchmark of multiple, non-HTML entities (in the Public Sector) hosted in a distributed environment in order to prioritize Public Spending against use and Value Created

Kristoffer EwaldDirector Analytics & Professional ServicesThe Milk / GuavaOctober 16th 2009

Background

About Kristoffer...

Here's a shot from my first presentation at Google in 2006

About

Summary

Fmr. Managing Director & Co-Founder of the first Scandinavian Google Analytics Authorized Consultant: The Milk (2005-2007), Acquired by Guava Online Marketing in 2007.

Now VP Analytics & Professional Services, in Guava

Guava Professional Services Team located in Denmark, Sweden and UK – Provides all Analytics Professional Services

Fmr. Country Coordinator Web Analytics Association Denmark (2005-2008)

Member Web Analytics Association Research Committee

Specialist, with background as an Economist, on Analytics and Online Marketing Benchmarks – coined the term Online Space Management with The Milk.

The Case

The Case

The public sector in the Kingdom of Denmark runs 250+ online-self-service solutions, build in a variety of technologies, making it possible to do a lot of things online - i.e. Register that you’re moving to a new address, sign up kids for kindergarten, Request unemployment compensation, Pension etc. (Denmark is probably one of the top 3 countries in regards to digital public sector).

These solutions have been build over time in a decentralized process, where the different departments have had a large number of different suppliers deliver the solutions. Up until now no centralized tracking or benchmark of the use of these digital self-service solutions have existed.

We’re developing a solution based on Urchin that will deliver a centralized data-warehouse for Analytics data for all these digital-self-service solutions.

The trick we have to pull off is that a lot of things happen server-side, and not in the browser, and that we need to track “inside” the self-service-solutions themselves, since they’re also used as widgets and made available in many different locations under different domains. We also need to know when a success happens outside the self service solution and even offline.

So we’re generating gif-hits server side and client side without the JavaScript.

After the data have been collected, the Urchin data will be pulled out with the API, stored in a data warehouse where it’ll be matured with data from other sources (FORM), and presented via a custom build front end.

Lot’s of API use will happen. The Urchin reports themselves will only be used by the core team for analysis.

The challenge is to tie sessions in the self-service solutions to sessions in the browser with the same ID contained, in order to be able to create “Mega Sessions”.

Current data quality that forms the basis of priorities is very aggregated – something like 16% of citizens in OECD used a public sector self service solutions during the last 3 months.

Summary of key concepts

Server Side tracking

Non JS client side tracking

Creation of “Master Sessions”

Use of IDs to create key for integration via API

Unique ID-tagged links, generated on the fly, to bridge gap between server side and client side tracking

Unique ID’s stamped in PDF’s + Tracked with Transaction to measure offline success

Source: 8pt, medium gray

Why?

Why build a benchmarking system?

The public sector needs benchmarking in order to prioritize what to develop and sustain

They need to know the maturity of the solutions

They need to know how much they’re used and on what domains

They need to know the distribution across the public portfolio of services

Measure the impact on the budget via integration with ”FORM” – the public sector business reference model

Source: 8pt, medium gray

Quiz – estimated potential Saving?

How (Framework)

“FORM” and linking with our benchmark

The “FORM” (Public Sector Business Reference Model) - task map is a meta data based tool that shows how each service is placed within the collected portfolio of public services, it’s linked for each service to the relevant section of the the Public Sector budget, the relevant legislation, the relevant Department etc.

So, the end result will be a benchmark that will give the combined Public Sector in Denmark insights into what Services are effective, how they contribute to the Public Sector budget, and where opportunities for improvement exist.

One example: when you move, the estimated cost to handle the paperwork is USD 40, online it’s USD 1. (and we have several hundred thousand people that move every year), and that’s just one service… Source: 8pt, medium gray

How we’ll build the Benchmark

There are three parts needed to do this:1) Server side tracking of solution-use server side

2a) Client side calls to utm.gif from inside widget (i.e. Java)

2b) Client side calls to utm.gif from browser (via .js)

The real challenge is to tie these discrete utm.gif hits into “Master Sessions”, and have a naming system for them including a key, to be able to integrate with data in FORM, and measure conversions.

In other words – we need to create “Master-Sessions”, containing several urchin-sessions (something also very relevant in a number of other applications i.e. e-mailing systems or CRM integration).This is one of the reasons Urchin is necessary, since we can’t do this with only 50.000 records, the current limit in GA.

Source: 8pt, medium gray

The Core Challenge in two parts

1. Find a way to connect discrete sessions into “Master sessions” and Pass information from client side tracking to server side, and Integrate with regular site tracking in order to record conversions (outside the self service applet).

2. Extract the Master Sessions, mature it with information from FORM, and create roll-up reports that’ll benchmark across all the variables in FORM.

Source: 8pt, medium gray

Challenge 1 – Create “Master Sessions”

Solution: Create virtual pageviews of the following structure:

To be supplemented with Transactions to get non-aggregated actions with IDs, in order to integrate with offline success – i.e. when a pdf is created (we’ll stamp the ID’s into the pdf)

/FORM_ID/PropertyID/InstanceID/Folder/Action_label#?variable1=x&variable2=y etc.

Label ValueFORM_ID KeyPropertyID Host where incident take placeInstanceID Random number+unix time codeFolder Folder nameAction_label# PV + VPV

API useBridging btw. Server and Client SideOffline Integration

API use

API will be used to pull data out, mature with “FORM_ID” as key with the existing data matrix, and create roll-up benchmarks against the Dimensions from Urchin vs FORM.

Source: 8pt, medium gray

Bridging

Create unique IDs in campaign data, to create a link btw. Server Side Pageviews and Client Side Pageviews.

Source: 8pt, medium gray

Offline Integration

Create a unique ID, stamp it into PDF’s generated in the user session, and set a Transaction, in order to tie forms printed and send in to conversions

Source: 8pt, medium gray

Denmark Population = 5m(OECD = 1.2 Bn)

Potential saving in Denmark:$ (40 offline -1 online) = 39*500,000 incidents *250 systems =4,875,000,000 $ = (approx.)$1,000 p.a. pr. capita �

Answer to the Quiz - potential saving?

Thank youKristoffer.ewald@guava.com