you've got (big) data! now what?

Jess Freaner • @jessfreaner

You’ve Got (Big) Data! Now What?

for Big Data Week - Chicago • Sept 21, 2016

why have we not seen the transformation

promised by “Big Data” ?

tools?

tools?

talent?

talent?

talent?

talent?

process.

gunshowcomic.com

gunshowcomic.com

agenda

- a common approach to data problems

- trying to plan in an ambiguous world

- a better (iterative) approach

- a quick informal example

- case study

a common approach

identify the problem

determine the solution

communicate results

linear

linear

clearly defined

an ambiguous world

identify the best locations to plant new trees

unclear problemsthe data world

how many? what kinds of trees?

move old trees? replace old trees?

identify the best locations to plant new trees

unclear problemsthe data world

aesthetically pleasing? maximize growth? increase foliage? offset CO2 emissions?

how many? what kinds of trees?

move old trees? replace old trees?

identify the best locations to plant new trees

unclear problemsthe data world

aesthetically pleasing? maximize growth? increase foliage? offset CO2 emissions?

how many? what kinds of trees?

move old trees? replace old trees?

identify the best locations to plant new trees

unclear problemsthe data world

ecosystem consequences new bug species toxins in ground soil drought starts

aesthetically pleasing? maximize growth? increase foliage? offset CO2 emissions?

how many? what kinds of trees?

move old trees? replace old trees?

identify the best locations to plant new trees

new data emerge

unclear problemsthe data world

aesthetically pleasing? maximize growth? increase foliage? offset CO2 emissions?

how many? what kinds of trees?

move old trees? replace old trees?

identify the best locations to plant new trees

new data emergeecosystem consequences new bug species toxins in ground soil drought starts

and new solutions!build a bike path

add benches plant bushes

introduce ladybugs

unclear problemsthe data world

identify the best locations to plant new trees

why trees?

unclear problemsthe data world

an iterative approach

better

anticipate refining your solutioniterative problem solving

generate ideas

build prototypeevaluation

1-4 week iterations

human-centered designpersonas, use cases

build device prototypessurveys, interviews, ethnography

generate ideas

build prototypeevaluation

1-4 week iterations

anticipate refining your solutioniterative problem solving

business requirements

minimum viable productsplit testing, A/B testing

personas, use cases

build device prototypes

generate ideas

build prototypeevaluation

1-4 week iterations

lean startuphuman-centered design

anticipate refining your solutioniterative problem solving

surveys, interviews, ethnography

business requirementsstory/user cards

write codeQA; requirements churnminimum viable productsplit testing, A/B testing

personas, use cases

build device prototypes

generate ideas

build prototypeevaluation

1-4 week iterations

agile programminglean startuphuman-centered design

anticipate refining your solutioniterative problem solving

surveys, interviews, ethnography

scientific method hypothesis

experimentmeasurement

agile programming story/user cards

write codeQA; requirements churn

lean startup business requirements

minimum viable productsplit testing, A/B testing

human-centered designpersonas, use cases

build device prototypessurveys, interviews, ethnography

generate ideas

build prototypeevaluation

1-4 week iterations

anticipate refining your solutioniterative problem solving

an informal example

quick

- explore tools / libraries

- learn more about Chicago

- play with social media data

- write a blog post

my goalrecent passion project

wouldn't it be great…

- go for quantity

- encourage wild ideas

- defer judgement

where to startrecent passion project

wouldn't it be great…

- go for quantity

- encourage wild ideas

- defer judgement

where to startrecent passion project

wouldn't it be great…

- go for quantity

- encourage wild ideas

- defer judgement

- most will be bad

where to startrecent passion project

wouldn't it be great…

- go for quantity

- encourage wild ideas

- defer judgement

- most will be bad

- a few will be good

where to startrecent passion project

brainstormrecent passion project

brainstormrecent passion project

brainstormrecent passion project

- text

- pictures

- geolocation

- networks

- timestamps

brainstormrecent passion project

- text

- pictures

- geolocation

- networks

- timestamps

- where?

- when?

- who?

brainstormrecent passion project

- text

- pictures

- geolocation

- networks

- timestamps

- where?

- when?

- to go?

- to avoid?

- who?

iterationsrecent passion project

+ =

- map Instagram activity to Chicago neighborhoods

eat at which Chicago

restaurants?

iterationsrecent passion project

iterationsrecent passion project

iterationsrecent passion project

put one day on the map

missing something?

missing datarecent passion project

redefined solutionrecent passion project

- what patterns might you find?

redefined solutionrecent passion project

- what patterns might you find?

redefined solutionrecent passion project

- meaningful stories / insights

redefined solutionrecent passion project

- meaningful stories / insights

- map Instagram activity to Chicago neighborhoods

recent passion project

+ =eat at which

Chicago restaurants?

redefined problem

recent passion project

+ = ?- who’s (not) represented in Chicago’s Instagram activity ?

redefined problem

iterationsrecent passion project

iterationsrecent passion project

final outputrecent passion project

final outputrecent passion project

final outputrecent passion project

eat at which Chicago

restaurants?

linear

linear

iterative

iterative

who is missing from Chicago’s

instagram?

iterative

a case study

procter & gambledata driven expertise exploration

procter & gamble fortune 50multinational

110,000+ employees

backstory

procter & gamble fortune 50

10 product categories

multinational

110,000+ employees

backstory

procter & gamble fortune 50

10 product categories

multinational

110,000+ employees

backstory

procter & gamble fortune 50

10 product categories

multinational

110,000+ employees

backstory

Kathie

informal conversations to clarify goalsprocter & gamble

Kathie

informal conversations to clarify goalsprocter & gamble

Kathie

Bob

Grace

George

Lyle

Tim

Mollie

pizza

falafel

burrito

tacos

salad

veggies

2 kids girlfriend

wife

husbanddog

roommates

diapers

metals

chemistry

englishhealth care

geology

informal conversations to clarify goalsprocter & gamble

informal conversations to clarify goalsprocter & gamble

Kathie

Kathie

informal conversations to clarify goalsprocter & gamble

informal conversations to clarify goalsprocter & gamble

informal conversations to clarify goalsprocter & gamble

foster collaboration ?

?

?

?

informal conversations to clarify goalsprocter & gamble

?

?

?

how?!?

informal conversations to clarify goalsprocter & gamble

linear

linear

visualize expertise in a

company

linear

visualize expertise in a

company

resources analysesvisualizations

data-driven expertise explorationprocter & gamble

data-driven expertise explorationprocter & gamble

Lorem Ipsum: a narrative about blankets.

Author: Charlie Brown

Date: 31 Jan 2012

Lorem Ipsum is a dummy text used when typesetting or marking up documents. It has a long history starting from the 1500s and is still used in digital millennium for typesetting electronic documents, page designs, etc.

In itself, the original text of Lorem Ipsum might have been taken from an ancient Latin book that was written about 50 BC. Nevertheless, Lorem Ipsum’s words have been changed so they don’t read as a proper text.

Naturally, page designs that are made for text documents must contain some text rather than placeholder dots or something else. However, should they contain proper English words and sentences almost every reader will deliberately try to interpret it eventually, missing the design itself.

However, a placeholder text must have a natural distribution of letters and punctuation or otherwise the markup will look strange and unnatural. That’s what Lorem Ipsum helps to achieve.

I would like to thank Peppermint Patty for her support on studying Lorem

Ipsum as well as the infinite wisdom of Linus van Pelt and his willingness to use his blanket in my experiments.

data-driven expertise explorationprocter & gamble

Lorem Ipsum: a narrative about blankets.

Author: Charlie Brown

Date: 31 Jan 2012

Lorem Ipsum is a dummy text used when typesetting or marking up documents. It has a long history starting from the 1500s and is still used in digital millennium for typesetting electronic documents, page designs, etc.

In itself, the original text of Lorem Ipsum might have been taken from an ancient Latin book that was written about 50 BC. Nevertheless, Lorem Ipsum’s words have been changed so they don’t read as a proper text.

Naturally, page designs that are made for text documents must contain some text rather than placeholder dots or something else. However, should they contain proper English words and sentences almost every reader will deliberately try to interpret it eventually, missing the design itself.

However, a placeholder text must have a natural distribution of letters and punctuation or otherwise the markup will look strange and unnatural. That’s what Lorem Ipsum helps to achieve.

I would like to thank Peppermint Patty for her support on studying Lorem

Ipsum as well as the infinite wisdom of Linus van Pelt and his willingness to use his blanket in my experiments.

data-driven expertise explorationprocter & gamble

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

VS

concept sketch comparisonsprocter & gamble

what matters & why?

what matters & why?

moving forwardprocter & gamble

what matters & why?

search engine with relevance

metrics

moving forwardprocter & gamble

what matters & why?

demographics

moving forwardprocter & gamble

what matters & why?

human readable expertise summary

moving forwardprocter & gamble

what matters & why?

moving forwardprocter & gamble

Lorem Ipsum: a narrative about blankets.

Author: Charlie Brown

Date: 31 Jan 2012

Lorem Ipsum is a dummy text used when typesetting or marking up documents. It has a long history starting from the 1500s and is still used in digital millennium for typesetting electronic documents, page designs, etc.

In itself, the original text of Lorem Ipsum might have been taken from an ancient Latin book that was written about 50 BC. Nevertheless, Lorem Ipsum’s words have been changed so they don’t read as a proper text.

Naturally, page designs that are made for text documents must contain some text rather than placeholder dots or something else. However, should they contain proper English words and sentences almost every reader will deliberately try to interpret it eventually, missing the design itself.

However, a placeholder text must have a natural distribution of letters and punctuation or otherwise the markup will look strange and unnatural. That’s what Lorem Ipsum helps to achieve.

I would like to thank Peppermint Patty for her support on studying Lorem

Ipsum as well as the infinite wisdom of Linus van Pelt and his willingness to use his blanket in my experiments.

data-driven expertise explorationprocter & gamble

Lorem Ipsum: a narrative about blankets.

Author: Charlie Brown

Date: 31 Jan 2012

Lorem Ipsum is a dummy text used when typesetting or marking up documents. It has a long history starting from the 1500s and is still used in digital millennium for typesetting electronic documents, page designs, etc.

In itself, the original text of Lorem Ipsum might have been taken from an ancient Latin book that was written about 50 BC. Nevertheless, Lorem Ipsum’s words have been changed so they don’t read as a proper text.

Naturally, page designs that are made for text documents must contain some text rather than placeholder dots or something else. However, should they contain proper English words and sentences almost every reader will deliberately try to interpret it eventually, missing the design itself.

However, a placeholder text must have a natural distribution of letters and punctuation or otherwise the markup will look strange and unnatural. That’s what Lorem Ipsum helps to achieve.

I would like to thank Peppermint Patty for her support on studying Lorem

Ipsum as well as the infinite wisdom of Linus van Pelt and his willingness to use his blanket in my experiments.

data-driven expertise explorationprocter & gamble

demographics

Lorem Ipsum: a narrative about blankets.

Author: Charlie Brown

Date: 31 Jan 2012

Lorem Ipsum is a dummy text used when typesetting or marking up documents. It has a long history starting from the 1500s and is still used in digital millennium for typesetting electronic documents, page designs, etc.

In itself, the original text of Lorem Ipsum might have been taken from an ancient Latin book that was written about 50 BC. Nevertheless, Lorem Ipsum’s words have been changed so they don’t read as a proper text.

Naturally, page designs that are made for text documents must contain some text rather than placeholder dots or something else. However, should they contain proper English words and sentences almost every reader will deliberately try to interpret it eventually, missing the design itself.

However, a placeholder text must have a natural distribution of letters and punctuation or otherwise the markup will look strange and unnatural. That’s what Lorem Ipsum helps to achieve.

I would like to thank Peppermint Patty for her support on studying Lorem

Ipsum as well as the infinite wisdom of Linus van Pelt and his willingness to use his blanket in my experiments.

data-driven expertise explorationprocter & gamble

demographics connections

Lorem Ipsum: a narrative about blankets.

Author: Charlie Brown

Date: 31 Jan 2012

Lorem Ipsum is a dummy text used when typesetting or marking up documents. It has a long history starting from the 1500s and is still used in digital millennium for typesetting electronic documents, page designs, etc.

In itself, the original text of Lorem Ipsum might have been taken from an ancient Latin book that was written about 50 BC. Nevertheless, Lorem Ipsum’s words have been changed so they don’t read as a proper text.

Naturally, page designs that are made for text documents must contain some text rather than placeholder dots or something else. However, should they contain proper English words and sentences almost every reader will deliberately try to interpret it eventually, missing the design itself.

However, a placeholder text must have a natural distribution of letters and punctuation or otherwise the markup will look strange and unnatural. That’s what Lorem Ipsum helps to achieve.

I would like to thank Peppermint Patty for her support on studying Lorem

Ipsum as well as the infinite wisdom of Linus van Pelt and his willingness to use his blanket in my experiments.

data-driven expertise explorationprocter & gamble

demographics

expertise

connections

low fidelityprototype iterationsprocter & gamble

higher fidelityprototype iterationsprocter & gamble

prototype iterationsprocter & gamble high fidelity

procter & gambledata-driven expertise exploration

– Kathie Felber, Technical CoP Leader

They delivered an innovative, one-of- a-kind tool that I use

every day to increase collaboration and better

understand our company

visualize expertise in a

company

linear

linear

iterative

iterative

search for expertise in a

company

iterative

doing is better than planning

while doing

while doingrefine your approach

while doingrefine your approach

get feedback

while doing

keep asking questions!

refine your approachget feedback

in conclusion

in conclusion

in conclusion

in conclusion

in conclusion

in conclusion

Thanks!You’ve Got (Big) Data! Now What?

Jess Freaner • @jessfreaner @DsAtweet