Transcript

1  BAMMF  June  20,  2014  

•  Marshal  McLuhan  in  1964.    – A  medium  itself,  not  the  content  it  carries,  should  be  the  focus  of  study.    

– A  medium  affects  the  society  in  which  it  plays  a  role  not  only  by  the  content  delivered  over  the  medium,  but  also  by  the  characteris@cs  of  the  medium  itself.  

BAMMF  June  20,  2014   2  

Media  

Mul+  

BAMMF  June  20,  2014   3  

Media  

+   +  

Digital  technology  allowed  combining  and  using  media  leading  to  a  major  disrupAon.  

Photos  are  the  killer  content  type  on  mobile.  Quick  to  consume  like  text,  but  easier  to  produce  on  a  

phone.    

BAMMF  June  20,  2014   4  

From  Mary  Meeker’s  presenta@on  

Big  Data  

BAMMF  June  20,  2014   5  

To  know  our  world,    (MulAmedia)  Data  is  the  Source.  

McLuhan  was  right.    UnAl  20th  Century  -­‐-­‐  

Medium  was  the  message.  

BAMMF  June  20,  2014   7  

BAMMF  June  20,  2014   8  

What  is  an  Elephant?  

BAMMF  June  20,  2014   9  

Six  Blind  Men  Exploring  an  Elephant.  

BAMMF  June  20,  2014   10  

An  Elephant  according  to  Six  Limited  PerspecAves.  

The  Blind  Men’s  Elephant  

BAMMF  June  20,  2014   11  

But,  an  Elephant  is  an  Elephant.  A  Real  Animal.  

MulAmedia  and  Big  Data  are  both  the  Latest  Elephant.  

 

BAMMF  June  20,  2014   12  

Lots  of  people  excited  about  these  but  with  narrow  perspecAves.  

BAMMF  June  20,  2014   13  

Two  Major  Human  AcAviAes    

•  CommunicaAons  

•  ComputaAons  

BAMMF  June  20,  2014   14  

Both  require  MulAmedia  AND  Big  Data.  

CommunicaAon  

BAMMF  June  20,  2014   15  

Nature  Gave  Us  Five  Basic  Senses  

BAMMF  June  20,  2014   16  

to  Experience  the  World  

We  promptly  created  Silos  

BAMMF  June  20,  2014   17  

Storytelling:  The  Oldest  and  most  powerful  Art.  

•  A  story  is  a  chain  of  events  that  begins  at  one  place  and  ends  at  another.  

•  A  story  is  a  selecAve  batch  of  informaAon.  It  selects  details,  arranges  them,  perhaps  embellishes  them.  

BAMMF  June  20,  2014   18  

h6p://janefriedman.com/2011/09/27/what-­‐is-­‐a-­‐story/  

Where  do  Events  happen?  

•  In  Real  World.  

•  Images,  video,  audio,  all  other  sensors  are  just  a  means  to  detect  and  capture  some  aspect  of  the  event.  

BAMMF  June  20,  2014   19  

 Events  are  ‘Connectors’  Events  create  ‘Context’  

Event  contains  Information  and  Experiences  

Spatial  Causal    

Experiential   Informational  

Temporal    Structural  

So  What  is  Digital  Storytelling?  

•  Prepare  a  database  of  all  events.  –  Collect  all  experienAal  and  informaAonal  data  about  each  event.  

•  Considering  audience  and  the  message,  select  important  events.  

•  Render  Events  –  For  each  event  decide  what  informaAon  and  experienAal  data  should  be  used.  

–  Render  all  events  and  related  data  in  the  best  format  for  highest  quality  of  experience.  

BAMMF  June  20,  2014   21  

From  Events  to  Stories  

CollecAng  Events  

SelecAng  Events  

Rendering  Events  

Oral  Storytelling  

BAMMF  June  20,  2014   23  

Novels  

BAMMF  June  20,  2014   24  

Video  

BAMMF  June  20,  2014   25  

Do  you  edit  your  videos?  

•  Why  is  video  ediAng  so  difficult?  – Even  a_er  so  many  products?  

•  How  can  we  make  it  usable  by  consumers?  •  Is  Video  ediAng  a  real  problem?  •  Is  it  possible  to  reinvent  video  ediAng  given  all  the  ‘big  data’?  

BAMMF  June  20,  2014   26  

Virage Video Indexing Lyn Russell

00:08:26:12 00:08:34:29 00:08:40:00

00:08:26:12 00:08:34:29 00:08:40:00

“Clinton spoke with reporters” “When She arrived” “Older citizens

who have”

Clinton running for senate

Speaking at rally

Improving Medicare

Lyn Russell

Hillary Clinton Face ID

Speaker ID

User Annotation

Speech to Text

Time Code

Keyframes

Video

Time

Hillary Clinton

Different  Data  Tracks  

Ac+vity  

Loca+on  

Manual  

Visual  

Audio  

OTHER  

Walk  

Walk  

Walk  

Walk  

Walk  

Walk  

Exercise  Drive  Home   Drive   Mee+ng   Work   Exercise   Home   Drive   Mee+ng   Work  Home   Mee+ng   Work   Exercise  

Event  Track  

Personalized  Storytelling:    Events  in  a  Football  Game.  

BAMMF  June  20,  2014   29  

All  communicaAon  is  story  telling.  

Even  scienAfic  papers.  

BAMMF  June  20,  2014   30  

Fundamental  Opportunity  in  MulAmedia  CommunicaAon  and  CompuAng:  UAlize  complementary  and  correlated  informaAon  from  mulAple  sources  to  form  holisAc  picture  of  the  world.  

BAMMF  June  20,  2014   31  

ComputaAon:  at  least  50,000  years  

•  CounAng  was  primarily  used  to  keep  track  of  social  and  economic  data  such  as  number  of  group  members,  prey  animals,  property,  or  debts.  

•  CompuAng  was  invented  for  Resource  management.  

BAMMF  June  20,  2014   32  

What  is  Important  in  ‘Big  Data’?  

MulAmedia  

RealAme   Uncertainty  

BAMMF  June  20,  2014   33  

Big  data  is  the  Rorschach  blot  of  our  Ames  –  an  incomprehensible  shape  on  to  which  we  project  our  dreams  and  

nightmares,  hopes  and  fears.  

BAMMF  June  20,  2014   34  

Rorschach  blot    

BAMMF  June  20,  2014   35  

You  can  only  control  what  you  measure.  

The most Fundamental Problem in Society:

Connecting People to Resources Effectively, Efficiently, and Promptly

in given Situations.

36  BAMMF  June  20,  2014  

Current    Social  Networks  

Important  UnsaAsfied    Needs  

37  

Time  to  Focus  BAMMF  June  20,  2014  

What  differenAates:  

•  Advanced  SocieAes  from  Primi6ve  SocieAes  •  Advanced  countries  from  a  Developing  countries  

38  BAMMF  June  20,  2014  

How  People’s  needs  are  connected  to  Resources.    

 

Hunter  Gatherer  

Food  delivery  where  you  are.  

39  BAMMF  June  20,  2014  

UnAl  2000,  you  went  to  a  phone.  

Now  the  call  comes  to  you.  

40  BAMMF  June  20,  2014  

UnAl  recently,  you  were  a  folder.  

Very  Soon   You  are  Your  Data.  

41  BAMMF  June  20,  2014  

Needs  

Need = f (persona, situation)

BAMMF  June  20,  2014   42  

Where do most Data Scientists apply their Magic?

We  live  in  a  targeted  world.    AdverAsers  need  efficient  ways  to  si_  through  the  volume  of  data.    

BAMMF  June  20,  2014   43  

Using BIG Data to solve Big PROBLEMS

Time  is  right.  

BAMMF  June  20,  2014   44  

We  Live  in  Dynamic  World.  

Big  Data  will  keep  gegng  bigger  in  Cyber  Space.    

BAMMF  June  20,  2014   45  

 Animals    Machines    SocieAes  

Published  first  in  1942  

• Desired  state  (Goal)  •  System  model  and  Control  Signal  (AcAons)  

•  Current  State  (using  mulAmedia  data)  

EventShop : E. Situation Detection

Predic+ve  Situa+on  

Recogni+on  

Evolving Global Situation

Predic+ve  Personal  Situa+on  

Recogni+on  

Personal EventShop

Evolving Personal Situation

Need- Resource Matcher

Recommendation Engine

Persona  Database  

Resources

Needs

Data  Inges+on  

Wearable Sensors

Calendar

Location ….

Dat

a So

urce

s ….

Data  Inges+on  

and  aggrega+on  

Database Systems

Satellite

Environmental Sensor Devices

Social Network

Internet of Things

Actionable Information

BAMMF  June  20,  2014  48  

CriAcal  Requirements:  

•  Environmental  SituaAon  RecogniAon.  –  Detects  geo-­‐spaAal  situaAons.  –  Space  is  a  key  dimension.  –  Helps  in  idenAfying  resources  and  connecAng  to  them.  

•  Personal  SituaAon  RecogniAon.  –  Centered  around  an  individual.  –  Uses  environmental  situaAon.  –  Helps  in  modeling  individuals  and  idenAfying  their  needs.  

BAMMF  June  20,  2014   49  

Data  to  Situa+ons  

 Situa+ons  

Event  Streams  

Mul+modal  Observa+ons  

Low-­‐level  Analysis  

Aggrega+on  and  Classifica+on  

BAMMF  June  20,  2014   50  

A  Grand  Challenge  for  data  has  always  been:  

Sense  making  from  mulAmodal  massive  geo-­‐social  data-­‐streams.    

51  BAMMF  June  20,  2014  

A  REAL  Challenge  

•  SituaAon  Modeling  •  SituaAon  RecogniAon  

(Object  recogniAon,  event  recogniAon,  speech  recogniAon,  …  dominated  last  century.)  

BAMMF  June  20,  2014   52  

Challenge:    Unifying  MulAmodal  Big  Data  

•  Spa+o-­‐temporal-­‐thema+c  (STT)    real-­‐@me  streams  •  E-­‐mage  as  a  unifying  GRID  representa+on  

53  

(a) Pollen levels (Source: Visual) (b) Census data (Source: text file) (c) Reports on ‘Hurricanes’ (source: Twitter stream)  

d) Cloud cover (Source: Satellite imagery) (e) Predicted hurricane path (source: KML) (f) Open shelters coverage(Source: KML)  

BAMMF  June  20,  2014  

54  

Billions  of  data  sources.  Environment  for  

 SelecAng,  and      Combining    

appropriate  sources  to  detect  situaAons.  PredicAon  for  Pro-­‐acAve  acAons  InteracAons  with  different  types  of  Users    

BAMMF  June  20,  2014  

EventShop  

BAMMF  June  20,  2014   55  

6/20/14   56  

Flood level - Shelter

Flood Level Shelter

Twitter

Classify (Flood level - Shelter)

6/20/14   57  

The Creative Destruction of Medicine

CompuAng  Power,  Data  Universe  

Social  Networking  

Internet  

Mobile  connecAvity,  Bandwidth  

InformaAon  Systems  

Imaging  

Wireless  Sensors  

Genomics  

New  Medicine  

CreaAv

e  De

strucAon

 

BAMMF  June  20,  2014   58  Old  Medicine  

Health = Genetics + Lifestyle

BAMMF  June  20,  2014   59  

Asthma Triggers

Indoor  Allergens  

Outdoor  Allergens  

Smoke  

Air  PolluAon  

Chemical  Irritants  

Asthma/Allergies  

BAMMF  June  20,  2014   60  

Everyone’s Respiratory Health is Different

Disaster  Situa+on  

Assimila+on      and  Control    

Environmental  

Resources  and  Historic  Data  

Governmental  Agencies  

Internet  of  Things  

Social  Sources  

Experts  

Users  

All  Users  are  not  EQUAL.  

BAMMF  June  20,  2014   62  

AcAviAes  

CommunicaAons  

Media  

Persona:  Turning  Disassociated  Data  into  Meaningful  InformaAon  

AGGREGATED  PERSONAL    ANALYTICS  

Sensors  

63  

Persona  

BAMMF  June  20,  2014  

Wearable 2013: Data Data Everywhere, nor any Insight to Use.

 h6p://www.medhelp.org/  BAMMF  June  20,  2014   64  

HEALTH  PERSONA  

Logical  Sensor   Physiological  Sensors  Fitness  Tracking  Sensors  

Life  Event  

Kine+c  Event  

Physiological  Event  

Food  Event  

Personicle  

(Health) Persona Framework

Calendar  Ac+vity

-­‐level  

Heart  rate  

Home   Drive   Mee+ng   Work   Exercise  Walk  

Walk  

BAMMF  June  20,  2014   65  

Time  

Foursquare  

Personal  Calendar  

Moves  

Drive  

Walk  

Mee+ng  in  DBH  room  3233  

Walk  

Drive  

Mee+ng  with  trainer  in  gym  

Ac+vity  Level  

Personicle  

Personicle  

Home   Drive   Walk  

Mee+ng   Work   Walk  

Drive  Exercise   Work  

Laleh  at  University  Club  

Ea+ng  dinner  

BAMMF  June  20,  2014   66  

Insight: Exercise Triggers My Asthma

•  Red: “Severe asthma attack”. •  Orange: High activity level.

•  Intense Exercise triggers asthma! Personicle  

Ac+vity  Stream  

t1   t2   t3   t4   t5  

BAMMF  June  20,  2014   67  

UCI  Allergy/Asthma  App  

Triggers   Recommendation  

Stop  Jogging!  

Symptoms  SHORTNESS  OF  BREATH  

CHEST  TIGHTNESS  OR  PAIN  

COUGHING  

WHEEZING  

How Do You Feel?  

BAMMF  June  20,  2014   68  

PredicAve  Techniques  for  PrevenAve  Advise  

69  BAMMF  June  20,  2014  

From Persona to Societal Health

Health  Persona  of  different  people    

Environmental  factors  

Disease  outbreak  

Disease  Modeling  

Geographical  loca+on  specifica+ons  

BAMMF  June  20,  2014   70  

Use  ObjecAve  Massive  Volume  of  Personal  Data  for  Building  Beker  Disease  Models.  

Persona and Societal Health

Use  Disease  Model  and  Personal  Data  for  Beker  Quality  of  Life.  

71  BAMMF  June  20,  2014  

Live  EventShop  and  CollaboraAon    •  Live  EventShop  Demo  

– h6p://auge.ics.uci.edu/eventshop/  •  Current  Collaborators    

– Cyber-­‐Physical  Cloud  Compu+ng  Project  •  NICT,  NIST  

– SLN4MOP  Project  •  Sri  Lanka  Farmers;  Prof.  Ginige  in  Sydney  leading  

– Open  Source  EventShop.  •  HCL  

– Other  partners     BAMMF  June  20,  2014   72  

MulAmedia  and  Big  Data  

•  MulAmedia  was  always  Big  Data.  •  Many  Major  Challenges  in  Big  Data  are  the  same  as  in  MulAmedia.  

•  Time  for  MulAmedia  community  to  Accept  and  Act.  

BAMMF  June  20,  2014   73  

Thanks  for  your  Ame  and  akenAon.  

For  ques+ons:  [email protected]  


Top Related