designing with big data

Designing with Big Data

Ha Phan @hpdailyrant

Designing with Big Data © Ha Phan

Long Story ShortLearned about Data at Porch

About 2 years ago, I joined a startup called Porch. And then I had a conscious uncoupling

Designing with Big Data © Ha Phan

PorchAggregated data to create a home improvement network

Porch was started by Scott Austin in late 2011.

Building trust between homeowners & home pros by making home improvement & home pros data transparent.

Designing with Big Data © Ha Phan

Selling a vision to investors is often about creating a utopia of perfect data & where all

segment of users give a damn.

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

What is Big Data?

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

Volume !

Velocity !

Variety

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

We’re moving towards predictive technology where data is the UI

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

“Big data is bullshit. It’s like atoms. You could either make a nuclear bomb or a bouquet of flowers with them.”

Richard Wurman, TED Founder

Designing with Big Data © Ha Phan

Web of Services,Mashups

Web of People

Social Networks

!Web 3.0

Web of SensorsInternet of Things. Web of

Databases Web of

Pages

Web 3.0We are all building sensors

Designing with Big Data © Ha Phan

TOP DOWN

BOTTOM UP

Platform

Data Ingestion ModelsTop Down or Bottom Up

User Generated Content

Structured & Unstructured Data

Designing with Big Data © Ha Phan

Data Information

Presentation Knowledge

Designing with Big Data © Ha Phan

Data

From Data to Knowledge

10,400

36,876

47,94825,04618,984 57,948

28,9488,964 32,948

18,94827,98365,94817,465 14,576

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

Data

Information

Cost of kitchen remodels in your neighborhood

From Data to Knowledge

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

Data

Don’t spend more than $32,874 based on ROI

Information

Knowledge / Value

From Data to Knowledge

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

The Reality of working

with Big Data

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

Pose Question

Acquire Clean

Contextualize / Model

Give Access

Data Pipeline in a NutshellAcquisition to Integration

Ingest

Designing with Big Data © Ha Phan

“Not everything that can be counted counts. And not everything that counts can be counted.”

Albert Einstein

Designing with Big Data © Ha Phan 25

Finding & framing the stuff that counts to the user

The Role of UX Humanizing Data

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan 25

The Role of UX Humanizing Data

First. Pose questions to frame discovery in data analysis

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

Pose Question

Data Management CycleUX & Product’s Role

What if homeowners could know which pros worked on their neighbors homes?

Designing with Big Data © Ha Phan

Every problem is about scaling relevance

Address Home Pro

See actual projects in homes

Actual projects the pro worked on

Pro who worked on the home

Designing with Big Data © Ha Phan

Ingesting Permit Data as a scaleable bandaid

Designing with Big Data © Ha Phan 25

Second. Evaluate & visualize your ingredients

The Role of UX Humanizing Data

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

Data Management CycleUX & Product’s Role

Acquire Ingest Clean

How can I optimize the data to enrich the experience?

What does the data really look like?

Designing with Big Data © Ha Phan

Permit Data is Barely Human

Designing with Big Data © Ha Phan 25

Sampling Data: Walking Stick for Blindness

Run queries to evaluate critical mass of dimensions and properties such as:

• Natural language

• Scale

• Measures

• Frequency, Temporal Dimension

Designing with Big Data © Ha Phan

Visualizing DataPicturing Dimensions

Time dimension

Designing with Big Data © Ha Phan 25

Third. Information Architecting

The Role of UX Humanizing Data

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

Information ArchitectingHow can the data best be classified & used in various contexts?

Designing with Big Data © Ha Phan

Reconcile IA properties

Address

PhotoDescription

Date Filed

Value

PhotoDescription

Date Completed

Cost

Home Pro

PhotoDescription

Date Filed

Value

Gap

Designing with Big Data © Ha Phan

Reconcile IA properties

Gap

Gaps in ontology leads to gaps in experience

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan 25

Fourth. Plan for Gaps !

- meaning - coverage - relevance

The Role of UX Humanizing Data

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan 25

Quantity Doesn’t Equal CompellingFinding compelling content literally is like finding needle in haystack

San Francisco, CA Little Rock, AR Phoenix, AZ

Designing with Big Data © Ha Phan San Diego, CA August, 2013

Designing with Big Data © Ha Phan

PorchHome Report

A home CarFax, another way of using the same data.

Pick a point of engagement that maximizes relevance & optimize your data

Designing with Big Data © Ha Phan

Define “Fit for Use” Rules

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan 29

The Empty Room ProblemSpotty coverage

Designing with Big Data is to design for tiers of imperfect data.

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan 25

Designing with Big Data © Ha Phan

Tiered Coverage ReportBy Location and Type

Designing with Big Data © Ha Phan 25

Designing for tiered experiences Evolving based on data coverage

Designing with Big Data © Ha Phan

How does the data inform relevance?

More is not always better

Designing with Big Data © Ha Phan

Designing with Big Data © Ha Phan

“There is no true information. Only spin.”

Abby Covert, Information Architect

Designing with Big Data © Ha Phan 25

It’s a journey !

of connecting the dots !

from !

digital footprints

Designing with Big Data © Ha Phan

How quickly can you differentiate with a credible compelling solution

or are you just spinning data?

Designing with Big Data © Ha Phan

Relevance The user interacts with the “best” or most relevant data. The rest s/he doesn’t care about.

What is your tip of the iceberg?Have best data inform relevance algorithm for each use case

Bad Quality or Low Relevance Data

Designing with Big Data © Ha Phan

Road mapping is about enhancing relevance & value Providing deeper engagement — aligning data initiatives with Product Strategy

Data Initiatives

Designing with Big Data © Ha Phan

Impact product strategy from the ground up Almost Done

• Provide questions that lead to discovery • Evaluate how data informs context • Visualize data early • Plan for tiered experiences to address gaps • Prioritize tips of the icebergs • Balance Top Down and Bottoms Up

Designing with Big Data © Ha Phan

“The hidden message of Big Data is that tiny choices, such as those made by software designers, when amplified through the network end up shaping humanity in subtle ways.

It is up to us to put forth visions of how things could be.”

Jonathan Harris, Co-Creator of WeFeelFine

Designing with Big Data © Ha Phan

Thank you