EMA

IT Arch Meeting, 5/2010

EMA’s History

• Pitched 10/2008(!) as lightweight split testing• Received positive interest from all verticals

except Redball• Designed several components, couple of

which were built• Shelved Q109, resources diverted to Graffiti

EMA Today

• Support running tests for customers that can do the site side instrumentation

• Use DW/Redball tracking fields• Supported 5 tests over the last year• Have interest from sites that can’t handle site

side (Findarticles, Gamespot)

EMA Components

TES(server)

Client Lib(js)

TMI(Tools)

DW

DW Tracking Convention

EVA(reporting)

40% development doneComponent Percent Complete Notes

DW Tracking Spec 100%How to instrument your site for EMA evaluation

EVA Beta 80% http://c13-pts-dev40.cnet.com:3293/ema/

TES Beta 60%, Untested

User Tools 20% - Dev Stopped Here

Started using Django

Client Lib 0% Still some open issues

EVA

• Runs significance test on tracked DW data• Requires manual setup• Needs tie-in to user tools (TMI)

TES

• Prototyped using POME (Rubics’ Python stack)• Takes affiliate_id, test_id• Decides – Whether the user is in the test– Which variant user should see

• Returns payload based on test variant

TMI and Client

• No significant development for either component

• Bulk of development is here• Also, a couple outstanding design issues:– Automatic set up/tear down for tests (SQL

generation, etc.)– DW Cookie synchronization

Deployment

• Finish TMI and Client• Solve remaining design/integ issues• Documentation• Pilot partner (FA?)