The Van Wagner Ad Sets Optimization Modeltm

Search Engine Marketing New England (SEMNE)Natick, MA May 18, 2010© Copyright 2010. Find Me Faster Nashua, NH. All Rights Reserved.

Matt Van Wagner

Best of Breed

Could too much A/B Testing be detrimental to the overall health

of our PPC campaigns?

How A/B Testing Works

The idea is to find outwhich message resonates best

with your target audience.

How A/B Testing Works

Run ads long enough to develop a performance profile.Declare a champ. Kill the loser.

Bring in a new challenger. Repeat until bored.

You declare Ad A the champ, and delete loser Ad B.What do you expect your Adgroup CTR to be going forward?

(a) 3.75% (b) 6.25% (c) 7.5% (d) 9.25%

Setting Expectations

Answer: It most cases, your new Ad Group CTR will range from 3.75% - 6.25%.

Surprised?

CTR: Adgroup vs. Ad

Worst Case: Ad Group CTR

Have you ever experienced this sort of surprise after a

round of A/B Testing?

Pitfalls of A/B Testing

Why performance may decline after an A/B Test:

- Keywords do not equal queries- match type influence

- Ad Impressions does not equal searchers

- Ad Rotation is not truly even

- A/B Testing will result in a best ad. It does not know anything about your ad group.

Keywords ≠ Queries

Broad & Phrase Match bring in many variations

Keyword = gluten free cakesSearch Queries =

gluten free desserts gluten free cookies gluten free cupcakesgluten free gourmet desserts gluten free recopiesgluten free desserts easy where to buy gluten free brownies gluten free cheesecake ma gluten free cat treats gluten free store bought cookies gluten free pie crust

gluten free store bought cookies gluten free bakeries gluten free frozen cookie dough stoneham mass gluten free bakeries ritt's gluten free cake gluten free cheesecake glutenfree angel food cakegluten free m & m cookiesgluten free bakeries in stoneham gluten free wedding cakes gluten free desertsgluten free cheese cakegluten free bakeries mi del gluten free cookies

Impressions ≠ Searchers

Searches are not randomly distributed - Searches for specific keywords happen in ’hotspots’ ”...as many as 50% of all searches are repeat searches...”*

* Yusuf Mehdi, Microsoft Senior Vice President, March 2010

Even Ad Rotation is Not Truly Even

Even when Ad Rotation is set to Even, it is not truly even:

- Search engines attempt to achieve approximately even search volume

- History of [ad + keyword] combination impacts Quality Score

- Relevance of [ad+keyword] impacts Quality Score

Diversity of Sales Pitches

Selection: All Shades of Blue WidgetsSavings: Save 20% on Blue WidgetsQuality: Eco-Friendly Blue WidgetsAvailability: Blue Widgets Shipped Overnight! Brand: Official Site ACME® Blue Widgets

Savings: Save 20% on Blue Widgets

Will you sell more widgets if you are allowed to use 5 great sales pitches or if you are limited to just 1 of of those pitches?

How a few well-performing ads can outperform the single best ad

in an ad group.

Ad Set Optimization

The Value of Ad Message Diversity

Selection: All Shades of Blue WidgetsSavings: Save 20% on Blue WidgetsQuality: Eco-Friendly Blue WidgetsAvailability: Blue Widgets Shipped Overnight! Brand: Official Site ACME® Blue Widgets

Savings: Save 20% on Blue WidgetsSavings: Save 20% on Blue WidgetsSavings: Save 20% on Blue WidgetsSavings: Save 20% on Blue WidgetsSavings: Save 20% on Blue Widgets

Wouldn’t it be a bit annoying for a salesperson to harp on the same thing 5 times in a row?

Beginning of a Testing Cycle

The Results Start Coming In

Cutting Poor Performing Ads

Without the other ads and repeat searches, you can expect that impressions, clicks and CTR performance changes in unexpected ways.

Down to the Finals

Yes? No? Don’t Know?

Should You Declare a Winner?

You get the same # of visitors, clicks, but CTR declines.Why? Perhaps people who like to save money assume ECO means more expensive, and don’t bother clicking.

Assuming the Worst Case

What would happen if you kept both ads online?

Multiple Searches Changes Things

Since we know that people will search multiple times, anywhere from 2-20 times over the course of a search session, it becomes almost certain they will see both ads.

With two ads left online, it becomes highly likely that a person will encounter the ad they prefer.

1 search: 50% (either ad A or ad B)2 searches: 75% (AA, AB, BA, BB)

3 searches: 87.5% (etc...)4 searches: 93.75%

Implementing the Van Wagner Ad Sets Optimization Model

Use A/B Testing to narrow out to well-performing ads. Don’t try to reduce to one winning ad.

Set Ad Rotation to even – at all times.

Ad Sets can mask the performance of poorly performing ad groups. You still have to optimize your ad groups and watch your search queries.

Instead of Reducing to a single ad, test more combinations.

New (Old) Google Keyword Match Type

Broad Match Modifier - More control than broad match - More flexible than phrase match

- shuts down expanded broad match

Uses + sign in front of a broad match keyword - Will match queries only when that it contains BMM includes misspellings, singular/plural forms, abbreviations, acronyms, and stemming variations. - Will not pick up close synonyms or related searches.

Available soon in US. Now in active beta in Canada & UK

Broad Match Modifier