Friendly, Appealing or Both? Characterising User Experience in Sponsored Search Landing Pages Marc Bron, Miriam Redi, Fabrizio Silvestri, Hue Evans, Mahlon Chute and Mounia Lalmas

Motivation and goals

•  Bad post-click experience with ads results in loss of daily active users and as a consequence revenue

•  Two dimensions: mobile friendliness and aesthetic appeal •  Help advertisers improving quality of their ads

●  create ground truth ●  develop features and learn classifliers ●  understand the post-click experience

Search ads

What is “mobile friendliness”?

A web page is mobile friendly if it has a good user experience on a

mobile device.

Good experience is a combination of great performance and mobile

specific experience.

What is “ad mobile friendliness”?

A web page that is mobile friendly as previously defined and

it makes it easy for the user to understand what the ad is about (e.g. the product advertised) and allows the user to

convert (e.g. purchase the product advertised).

What we did

Features Learning a classifier

editorial guidelines

Ground truth Aesthetics appeal

HTLM Visual Random forest

700,000 ad search landing pages: Bad: 6% Fair: 50% Good: 35% Excellent: 9%

What does this tell us?

Ground truth

Mobile Friendliness – 4 levels

EXCELLENT GOOD FAIR BAD

Mobile-optimized Conversion easiness

Good product/

business experience

Mobile optimized

Simplified navigation

Good readability

High usability

Missing good mobile friendliness criteria

Fair: missing ONE criteria

Bad: missing 2 OR MORE criteria (e.g. a desktop page!)

Aesthetically pleasing checkbox ●  Good color and shape harmony ●  Photographically beautiful images (if any) ●  Colors and fonts tend to harmonize with the theme of the site, without being centers of attention ●  The layout of each page tends to direct eye movement to the key part of the page

Annotation results

4,025 landing pages 63% agreements on mobile friendliness Of the 37% disagreements: 3% by one rating level 5% by two or more rating levels 2% Not Judged rating (page not longer present) 5% aesthetically pleasing Spearman's rho = 0.2 between mobile friendliness and aesthetically pleasing (p < 0.001)

Features

Family of HTML features (63) •  Mobile optimized: whether a page is specifically designed for mobile or

whether it is a desktop page. •  Window size: aspects of the size of the rendered html image and used

to detect whether the size of a page is suitable for mobile devices. •  Readability: identify the formality of the language used in the landing

page text. Intuition is that dense and formal texts may be less pleasant to read on a mobile device.

•  Input: number and type of input elements in a page. Intuition is pages that require users to provide information through many forms may be considered less mobile friendly.

•  Navigation: proportion of internal links, external links, and text contained in a page. E.g., mobile friendly pages may provide access to different sections of a page through internal links.

Family of visual features (66)

Color Distribution Features:

Hue, Saturation, Brightness Rule of Thirds: Image

Composition and Layout

Emotional Response Features:

Pleasure, Arousal, Dominance

Depth of Field: Sharpness contrast between

foreground and background

Objective Quality Features:

Sharpness, Noise, JPEG quality, Contrast Balance,

Exposure Balance

Learning classifiers

Classification tasks

ad landing pages

1,2,3,4 0/1

Random forest

split the data based on advertisers 10-fold cross 90% and 10%

Results

Feature family and level of mobile friendliness Feature family AUCw AUCbad AUCfair AUCgood AUCexcellent

readability navigation input window size mobile optimised layout texture & contrast color image quality

.589

.642

.674

.731

.752

.707

.706

.727

.727

.518

.695

.682

.850

.797

.825

.741

.795

.855

.662

.677

.697

.741

.769

.722

.734

.766

.719

.553

.566

.618

.707

.739

.668

.662

.676

.696

.786

.626

.661

.629

.687

.620

.632

.623

.627

all .788 .890 .800 .762 .693

Easier to predict bad then fair, less easy to predict excellent.

Aesthetics

HTML feature category

AUCw Visual feature category

AUCw

input navigation mobile optimised readability

.541

.551

.566

.570

layout quality texture color

.633

.680

.688

.695

Using all features: AUCw = 0.661, which is compared to state-of-the-art on image beauty classification in computational aesthetics research

Feature analysis

Mobile friendliness •  Most discriminative features

related to window size + mobile optimized

•  Cleaner and less pixelated pages are more mobile friendly

•  Low quality pages more often have very bright colour combinations

•  Presence of a small number of bright colors with an otherwise moderate use of brightness associate well with mobile friendliness

Aesthetics

•  Presence of objects in certain focus areas better for aesthetic appeal

•  Aesthetic appeal mostly explained with visual features, in particular image brightness

•  Too bright pages are not good, while pages with mild brightness more aesthetically pleasing

Accounting for mobile friendliness & aesthetic can provide easier navigation & inspiring and convincing ad experience

Mobile friendliness vs. aesthetic appeal

HTML features •  mobile optimized and structure important

for mobile friendliness •  width image negatively correlated to mobile

friendliness, but positively correlated to aesthetics

Visual features •  smooth and rougher pages are

aesthetically pleasing •  symmetry very important for mobile

friendliness •  importance of brightness for aesthetics less

prominent compared to mobile friendliness •  uniqueness more important for mobile

friendliness than aesthetics

Study dimensions of post-click experience for search ads, mobile friendliness and aesthetic appeal à to “consume” the ad

Good and excellent ad landing pages positively associated with long clicks but not necessarily with conversions.

Few visually attractive landing pages but these were positively associated with conversions.

A/B testing with bad landing pages filtered out decrease of -6.04% in short clicks increase of +1.15% in ad click-through rate

Final results

700,000 ad search landing pages: Bad: 6% Fair: 50% Good: 35% Excellent: 9%

Examples of excellent mobile friendly and aesthetically appealing landing pages

keep it simple, with one nice background, and a simple call to action