[@indeedeng talk] diving deeper into data-driven product design

Graham Davis Donald WysockiSr. Product Manager Product Director

Diving Deeper into Data-Driven

Product Design

I help people get jobs.

Our guiding principle

Do what's best for job seekers

Simple

Fast

Comprehensive

Relevant

Data-Driven Product Design

1. Vision

Data-Driven Product Design

Data-Driven Product Design

1. Vision2. V1

Data-Driven Product Design

1. Vision2. V13. Test and measure

Data-Driven Product Design

1. Vision2. V13. Test and measure4. and test and measure and...

Company Reviews

Graham Davis

Indeed Apply

In the past 12 months...

500 million jobs on Indeed

39 billion job searches

6 billion jobs viewed

Estimated720 million applications

~20 minutes to apply

10,000,000 days

How it works

Applications

● Email

Interested parties, please email your resume with the subject line "PHP Web Developer" tojobs@smallworldlabs.com

Applications

● Email● Simple resume upload

Applications

● Email● Simple resume upload● Account + complex forms

Applications

● Email● Simple resume upload● Account + complex forms● Mobile

Applications

● Email● Simple resume upload● Account + complex forms● Mobile

Applications

● Email● Simple resume upload● Account + complex forms● Mobile

Easily apply to all jobs on all devices.

Vision

So, where do we start?

Nearly every job application was completed

on another site

2,000 jobs posted to Indeed by small businesses

V1: How can we make it easier?

What if job seekers who previously applied could re-

use their resume?

-10%

0%

Why?

Maybe job seekers didn't understand they have a saved

resume

Can an Indeed Resume make it easier to apply?

Applications Per Week+0%

For job seekers who are signed in & have an Indeed

Resume, the application rate increased by 2x

Promising, but...

2x a small number is still a small number

We need more jobs

Opportunities

Applications

● Email● Simple resume upload● Mobile

Applications

● Email● Simple resume upload● Mobile

5,000

Postedon Indeed

9,500Jobs

Applywith Email

A new test:1. US only2. Signed-in3. Indeed Resume

Applications Per Week+33%

Can we make this available to more job seekers?

Expanded test:1. US only2. Signed-in3. Indeed Resume

Total Job Applications

+70%

Can we get more jobs?

INTL14,500

US

207,000Expand Internationally

Applications per Week

+58%

Can we get even more jobs?

Applications

● Email● Simple resume upload● Mobile

Applications

● Email● Simple resume upload● Mobile

Apply on the original website

Trusted,Everywhere

First (and fast) partner test:ZipRecruiter

Applications from Indeed+109%

More applications from the same number of job seekers

How can we make it easier for job seekers to find these

jobs?

+2% applications

Does branding actually help job seekers?

+6% applications

Progress:● Easier apply process● More jobs● Easier to find

Applications

● Email● Simple resume upload● Mobile

Applications

● Email● Simple resume upload● Mobile

The big win...

WebMobile

> 1/3 of searches

<5% of the Fortune 500 accept applications on

their mobile sites

3rd party applymobile apply

more resumesmore posted jobs...

Applications per Week

+584%

Data driven design isn't just about numbers and charts...

I help people get jobs.

Company Reviews

Graham Davis

I help people get jobs.

I help people get jobs.

the best<

What is it like to work at...

typical day

typical daymanagement

typical daymanagementco-workers

647

important decision

opportunity to help job seekers

Help job seekers find the best employers.

Vision

3,000,000

0

Jan 2011: V1

Not perfect

We don't know anything yet

Good enough to start testing

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

Remove anything that's not absolutely necessary

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

● "write a review" button● review form● search for companies● see reviews on company page● see reviews on job pages● see reviews on search results● best places to work● filter by location● filter by job function● review commenting● review voting● featured reviews

Built it

August 2011: 8,000

August 2011: 8,000only 0.2% of jobs

August 2011: 8,000only 0.2% of jobs

1 of every 50 search pages

August 2011: 8,000only 0.2% of jobs

1 of every 50 search pagesNOT good enough

Goal: every search page

500,000

300

300

10,000

300

10,000

+3000%

So we started testing

Why aren't people submitting reviews?

How can we make it easier?

What can we learn from successful review sites?

+27%387

What can the data tell us?

How can we improve company search?

-45%

Step back and assess

What works for one site does not necessarily work

for another site

Indeed is not a reviews site

Indeed is a job site

How can we engage our job seekers where they are?

The flow: Oct 2011

What can the data tell us?

2 ways to improve a funnel

2 ways to improve a funnelImprove conversion at step

2 ways to improve a funnelImprove conversion at step

Remove step entirely

Can we remove steps?

Yes,if we target job seekers with

Indeed resumes.

New flow: Nov 2011

+343%

1,507

What can the data tell us?

Avg reviews per reviewer: 1

Avg reviews per reviewer: 1Avg jobs per resume: 3.6

How can we increase reviews per reviewer?

Before

After

+46%3,719

Avg reviews per reviewer: 1Avg jobs per resume: 3.6

1.5

Avg reviews per reviewer: 1Avg jobs per resume: 3.6

why not 3.6?1.5

What can the data tell us?

Indeed

Zoominfo

?42%

Indeed

Zoominfo

My Perfect Gig

+57%5,834

What would happen if we had more targeted

promos for job seekers with resumes?

+36%7,919

Can we remove more steps?

Before

After

+104%

13,010

Can we remove anything from the review form?

RatingsReview TitleReviewProsConsJob TitleJob LocationStart/End Date

RatingsReview TitleReviewProsConsJob TitleJob LocationStart/End Date

Resume

+24%17,168

Today...

3,000,000

1.7M companies

60,000 / week

60% of jobs

6 out of 10

we moved the needle more by removing than

adding

Simple

Perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away.

Antoine de Saint Exupéry

Donald Wysockidonald@indeed.com

Graham Davisgdavis@indeed.com

Q&A

Next @IndeedEng TalkEngineering Velocity: Building GreatSoftware Through Fast Iteration

Dan Heller, Engineering Manager

7pm Wednesday, August 28th