designing intelligent social systems 121205
DESCRIPTION
With emerging technologies and big data, it is now possible to design intelligent social systems. In this presentation, ideas related to designing such systems are presentedTRANSCRIPT
12/5/12 1
• Social systems rely on primi0ve technology. • Big Data has opened Big Opportuni0es. • Situa0on recogni0on is a key technology. • EventShop may be useful in designing Intelligent Social Systems.
• Comments. • Sugges1ons. • Collabora1on opportuni1es. • [email protected] • Gmail, FB, TwiBer: jain49
Send:
Intelligent: displaying or characterized by quickness of understanding, sound thought, or good judgment. Social Systems: Social systems are the paBerns of behavior of a group of people possessing similar characteris1cs due to their existence in same society.
• Introduc1on • Social Systems • Intelligent Social Systems • Designing Intelligent Social Systems • Situa1on Recogni1on • Concept recogni1ons • Personalized Situa1ons • EventShop
An Interes0ng Situa0on
When we were data poor – we searched for words in documents.
Now that we are data rich – should we s0ll search for words?
Time has come for us to stop thinking data poor; really start thinking and behaving data rich.
7
Volume
Varie
ty
Big Data offers Big Opportuni4es. But, …. ?????
Middle 4 Billion
Top 1.5 Billion
BoOom 2 Billion
Middle of the Pyramid (MOP):
Ready.
Most aOen0on by Technologists – so far.
Not Ready
9
Data is Essen0al. But, we are really interested in its products:
Informa0on, Knowledge, and Wisdom.
12/5/12 10
Knowledge Observe
Recognize
Act Big Data
Planning Control
Objects Situa0ons
12/5/12 11
Past is EXPERIENCE Present is EXPERIMENT Future is EXPECTATION
Use your Experiences In your Experiments
To achieve your Expectations
12/5/12 12
Astrology
To
Astronomical Volumes of Data
• People • Things • Events
We are immersed in Networks of
It is now possible to be Pansophical. 12/5/12 13
12/5/12 Proprietary and Confiden1al, Not For Distribu1on 14
Our mobile wireless infrastructure can be “reality mined” to understand the paOerns of human behavior, monitor our environments, and plan social development. -‐-‐-‐-‐ Pentland in “Society’s Nervous System: Building Effec0ve Government, Energy, and Public Health Systems”
• Objects -‐-‐ popular in the West. • Rela0onships and Events – popular in the East. • Objects and Events – seems to be the new trend.
• The Web has re-‐emphasized the importance of every object and event being connected to others -‐-‐ East Meets West.
Geography of Thought by Richard NisbeB
• Data • Objects • Rela0onships and Events
• Take place in the real world. • Captured using different sensory mechanism.
– Each sensor captures only a limited aspect of the event.
• Can be used to bridge the seman1c gap.
Events: Types and Granulari1es • Conferences
– Days • Sessions
– Talks » Purpose of the talk
• Wedding • An Earthquake • The Big Bang • World Wide Web • Yahoo: Winter School 2012 • Me
– My Birth, – Being here, and – Dying in 100 years.
People Things Places Time Experiences Events
E by Westerman and Jain
E* by Gupta and Jain
Sense making from mul1modal massive geo-‐social data-‐streams.
20
• Introduc1on
• Social Systems • Intelligent Social Systems • Designing Intelligent Systems • Situa1on Recogni1on • Concept recogni1ons • Personalized Situa1ons • EventShop
Poli0cs Religion
Economics Health
Educa0on
Connec4ng People to Resources effec4vely, efficiently, and promptly
in given situa4ons.
• Minimize hunger in the world. • Maximize female educa1on in India. • Minimize ‘deaths’ in the coming hurricane in Florida.
• Minimize work-‐hours lost in traffic during week days in Bangalore.
• System: – A set of diverse parts forming a whole. – Parts are put together with a common objec1ve/purpose.
• Each part could be considered a system. • Each part plays a role towards the system objec1ve.
• Designing the informa1on flow among parts is essen1al to make a system work apprpriately.
• A social system is composed of persons or groups who share a common objec1ve.
• An individual objec1ve is usually a part of the group’s objec1ve.
• Persons • Families • Organiza1ons • Communi1es: City, State, Country • Socie1es • Cultures
• Top Down: – The social system determines its parts. – People’s behavior determined by society.
• BoBom Up: – The Society is the sum of its indivduals – Individual ac1ons determine the character of the society.
• Each social en1ty is a holon. • Holon: Each en1ty is simultaneously a part and a whole.
• A social component is made up of parts and at the same 1me maybe part of some larger whole.
• Any system is by defini1on both part and whole.
• The primary ‘currency’ of a social system is informa1on.
• System behavior can be understood as the movement of informa1on: – Within a system – Between the system and its environment
• Informa1on is used to sense as well as to control or act.
• Introduc1on • Social Systems
• Intelligent Social Systems • Designing Intelligent Systems • Situa1on Recogni1on • Concept recogni1ons • Personalized Situa1ons • EventShop
• Systems that perceive, reason, learn, and act intelligently.
• Adaptability to varying environmental situa1ons is a key element of intelligent systems
• Social systems that perceive, reason, learn, and act intelligently.
• What does ‘perceive’, ‘reason’, ‘learn’, and ‘act’ mean in the context of social systems?
• Introduc1on • Social Systems • Intelligent Social Systems
• Designing Intelligent Social Systems
• Situa1on Recogni1on • Concept recogni1ons • Personalized Situa1ons • EventShop
• Desired state (Goal) • System model and Control Signal (Ac0ons)
• Current State (for Feedback)
Observed State
Real World Events
Observa0o
ns
Feedback
Control Signals
Social Networks
Connecting
People
Needs and Resources
Not even FaceBook!
Current Social Networks
Important Unsa1sfied Needs
12/5/12 46
• Resources – Physical: food, water, goods, … – Informa:onal: Wikipedia, Doctors, … – Transporta:on – Employment – Spiritual
• Timeliness • Efficiency
Connecting People
And Resources
Aggregation and
Composition
Situation Detection
Alerts
Queries
Information
12/5/12 48
• Introduc1on • Social Systems • Intelligent Social Systems • Designing Intelligent Systems
• Situa0on Recogni0on • Concept recogni1ons • Personalized Situa1ons • EventShop
Connec4ng People to Resources effec4vely, efficiently, and promptly
in given situa4ons.
• rela1ve posi1on or combina1on of circumstances at a certain moment.
• The combina1on of circumstances at a given moment; a state of affairs.
• Situa1on awareness, or SA, is the percep1on of environmental elements within a volume of 1me and space, the comprehension of their meaning, and the projec1on of their status in the near future.
• What is happening around you to understand how informa1on, events, and your own ac1ons will impact your goals and objec1ves, both now and in the near future.
• Example 1: – A person shou1ng. – 1000 people shou1ng.
• In a contained building • In main parts of a city
• Example 2: – One person complaining about flu. – Many people from different areas of a country complaining about flu.
Facebook and TwiBer (now GOOGLE +)
Massive collec1on of events. Have been repor0ng events as micro-‐blogs
Time
Does the flap of a buEerfly’s wings in Brazil set off a tornado in Texas?
12/5/12 57
Have been repor0ng events as micro-‐blogs
Sensors and Internet of Things are crea1ng and repor1ng even more events than humans are.
FROM TWEETS TO REVOLUTIONS
Time
Atomic and Composite Events
• Given a plethora of event data. How can we: – Disambiguate relevant and irrelevant events? – Combine events into meaningful representa1ons ? – Allow inference and cascading effects? – Support different interpreta1ons based on applica1on domain?
– Support Control & decision making?
1. Inherent support for event-‐based (temporal) reasoning
2. The ability of the controller to reason based on symbols (rather than just signals)
3. Explicit inclusion of domain seman1cs (to support mul1ple applica1ons)
An ac4onable abstrac4on of observed spa4o-‐temporal characteris0cs.
62
• Introduc1on • Social Systems • Intelligent Social Systems • Designing Intelligent Systems • Situa1on Recogni1on
• Concept recogni0ons • Personalized Situa1ons • EventShop
Time Line
Data Type
1950 2000
Time Line
Data Type
Character 1959
Objects 1963
Events 1986
Speech 1962
Situa0on 2010
1950 2000
66
Environments
Real world Objects
Situa1ons
Ac1vi1es
Single Media
SPACE TIME
Scenes Loca1on aware
Visual Objects
Trajectories
Visual Events
Loca1on unaware
Sta1c Dynamic
Loca1on aware
Loca1on unaware
Sta1c Dynamic
Data = Text or Images or Video
• 1963: Object Recogni1on [Lawrence + Roberts] • 1967: Scene Analysis [Guzman] • 1984: Trajectory detec1on [Ed Chang+ Kurz] • 1986: Event Recogni1on [Haynes + Jain] • 1988: Situa1on Recogni1on [Dickmanns]
1960 1970 1980 1990 2000 2010
Object Scene Trajectory
Event
Situa1on
68
Environments
Real world Objects
Situa1ons
Ac1vi1es
SPACE TIME
Loca1on aware
Loca1on unaware
Sta1c Dynamic
Heterogeneous Media
Loca1on aware
Loca1on unaware
Sta1c Dynamic
Data is just Data. Meta-‐data is also data. Caste system does not exist here. Medium and sources do not maOer.
• Introduc1on • Social Systems • Real Time Social Systems • Designing Real Time Systems • Situa1on Recogni1on • Concept recogni1ons
• Personalized Situa0ons • EventShop
A) Situa0on Modeling B) Situa0on Recogni0on C) Visualiza0on, Personaliza0on, and Alerts
…
STT Stream
Emage
Situa1on
C1 ⊕
v2 v3 ⊕
v5 v6
@
∏
Δ @
i) Visualiza1on
ii) Personaliza1on
+
+ Available resources
iii) Alerts
Personal context
Personalized
situa1on
70
12/5/12 Proprietary and Confiden1al, Not For Distribu1on 71
73
STT data
Tweet: ‘Urrgh… sinus’
Loc: NYC, Date: 3rd Jun, 2011 Theme: Allergy
Situa1on Detec1on
User-‐Feedback
‘Please visit Dr. Cureit at 4th St immediately’
Date: 3rd Jun, 2011
Aggrega1on,
1) Classifica1on 2) Control ac1on
Opera1ons
Alert level = High
• Introduc1on • Social Systems • Intelligent Social Systems • Designing Intelligent Systems • Situa1on Recogni1on • Concept recogni1ons • Personalized Situa1ons
• EventShop
• E-‐mage
– Visualiza1on – Intui1ve query and mental model – Common spa1o temporal data representa1on – Data analysis using media processing operators (e.g. segmenta1on, background subtrac1on, convolu1on)
76
• Spa1o-‐temporal element – STTPoint = {s-‐t-‐coord, theme, value, pointer}
• E-‐mage – g = (x, {(tm, v(x))}|xϵ X = R2 , tm ϵ θ, and v(x) ϵ V = N)
• Temporal E-‐mage Stream – TES=((ti, gi), ..., (tk, gk))
• Temporal Pixel Stream – TPS = ((ti, pi), ..., (tk, pk))
77
12/5/12 78 Proprietary and Confiden1al, Not For Distribu1on
12/5/12 79 Proprietary and Confiden1al, Not For Distribu1on
12/5/12 80 Proprietary and Confiden1al, Not For Distribu1on
Retail Store Loca0ons
Net Catchment area
• Humans as sensors • Space + Time as fundamental axes • Real 0me situa0on evalua0on (E-‐mage Streams)
(a) Pollen levels (Source: Visual) (b) Census data (Source: text file) (c) Reports on ‘Hurricanes’ (source: Twitter stream)
d) Cloud cover (Source: Satellite imagery) (e) Predicted hurricane path (source: KML) (f) Open shelters coverage(Source: KML)
81
• Help domain experts externalize their internal models of situa1ons of interest e.g. epidemic.
• Building blocks: – Operators – Operands
• Wizard: – A prescrip1ve approach for modeling situa1ons using the operators and operands
82 Singh, Gao, Jain: Situa:on recogni:on: An evolving problem for heterogeneous dynamic big
mul:media data, ACM Mul0media ‘12.
Growth rate (Flu reports) Feature
Thresholds (0, 50)
Data source
Meta-‐data
-‐Emage (#Reports) Representa1on level
TwiBer-‐Flu
83
Knowledge or data driven building blocks
Get_components (v){ 1) Identify output state space 2) Identify S-T bounds 3) Define component
features: v=f(v1, …, vk)
• If (type = imprecise) – identify learning data source, method
4) ForEach (feature vi) { If (atomic)
• Identify Data source.
• Type, URL, ST bounds • Identify highest Rep. level reqd. • Identify operations
Else Get_components(vi)
} }
84
v f1
v4
⊕
v2 v3 @
D1
Emage
Δ
D2
∏
Emage
Δ
D3
Δ
@
Emage D2
∏
Emage
Δ
f2 ⊕
v5 v6
<USA, 5 mins, 0.01x 0.01>
ϵ { Low, Mid, High}
Epidemic Outbreaks
Unusual Ac1vity? Growth Rate
⊕
Current ac1vity level
Historical ac1vity level
⊕
Emage (#reports ILI)
Δ
TwiBer-‐Flu
⊕
TwiBer.com <USA, 5 mins, 0.01x 0.01>
Emage (Historical avg)
Δ
TwiBer-‐Avg
DB, <USA, 5 mins, 0.01x 0.01>
Δ
TwiBer-‐Flu
Emage (#reports ILI)
TwiBer.com <USA, 5 mins, 0.01x 0.01>
ϵ {Low, mid, high}, <USA, 5 mins, 0.01x
0.01>
Growing Unusual ac1vity
γ 1) Model
Emage (#reports ILI)
Δ
TwiBer-‐Flu
Emage (popula1on)
Δ
CSV-‐ Popula1on
⊕
π
TwiBer.com <USA, 5 mins, 0.01x 0.01>
Census.gov, <USA, 5 mins, 0.01x 0.01>
2) Revise
Subtract
Subtract
Mul1ply
Classifica1on: Thresh (30,70)
Normalize [0,100]
3) Instan1ate
85
Level 1: Unified representa1on (STT Data)
Level 3: Symbolic rep. (Situa1ons)
Proper1es
Proper1es
Proper1es
Level 0: Raw data streams e.g. tweets, cameras, traffic, weather, …
Level 2: Aggrega1on (Emage)
…
STT Stream
Emage
Situa1on
86
Opera1ons
87
⊕
PaBern Matching
Aggregate
ψ
@ Characteriza1on
∏ Filter
γ Classifica1on
72%
+
+
Growth Rate = 125%
Data Suppor1ng parameter(s) Output Operator Type
+
Classifica1on method
Property required
PaBern
Mask
Δ Transform … Spa1o-‐temporal
window
88
⊕ Aggregate +
γ Classifica1on Classifica1on method
@ Characteriza1on Growth Rate = 125%
Property required
PaBern Matching ψ 72%
+PaBern
∏ Filter +Mask
Φ Learn Learning method
{Features}
{Situa1on}
f f
1) Data into right representa1on
2) Analyze data to derive features
3) Use features to evaluate situa1ons
Suppor1ng parameter(s)
Data Output Operator Type
Singh, Gao, Jain: Social Pixels: Genesis and Evaluation, ACM Multimedia ‘10. 89
S. No Operator Input Output
1 Filter ∏ Temporal E-‐mage Stream Temporal E-‐mage Stream
2 Aggrega0on ⊕ K*Temporal E-‐mage Stream Temporal E-‐mage Stream
3 Classifica0on γ Temporal E-‐mage Stream Temporal E-‐mage Stream
4 Characteriza0on : @ • Spa0al • Temporal
• Temporal E-‐mage Stream • Temporal Pixel Stream
• Temporal Pixel Stream • Temporal Pixel Stream
5 PaOern Matching ψ • Spa0al • Temporal
• Temporal E-‐mage Stream • Temporal Pixel Stream
• Temporal Pixel Stream • Temporal Pixel Stream
• Select E-‐mages of US for theme ‘Obama’. – ∏spa1al(region=[24,-‐125],[24,-‐65]) (TEStheme=Obama)
• Iden1fy 3 clusters for each E-‐mage above. – γkmeans(3) (∏spa1al(region=[24,-‐125],[24,-‐65])(TEStheme=Obama))
• Show me the speed for each cluster of ‘Katrina’ e-‐mages
– @speed(@epicenter(γkmeans(n=3) (∏spa1al(region=[24,-‐125],[24,-‐65]) (TEStheme=Katrina)))) • How similar is paBern above to ‘exponen1al increase’?
– ψexp-‐increase(@speed(@epicenter(γkmeans(n=3) (∏spa1al(region=[24,-‐125],[24,-‐65])
(TEStheme=Katrina))))
90
1) Macro situa0on
Macro data-‐sources Personal
Context
Profile + Preferences
2) Personalized situa0on
User data
91
IF person ui <is-‐in> (PSj) THEN <connect-‐to> rk
Personalized situa0on: An ac4onable integra4on of a user's personal context with surrounding spa4otemporal situa4on.
3) Personalized alerts
Available resources
Resource data
Personalized Situa1on Recogni1on: Operators
⊕
PaBern Matching
Aggregate
ψ
@ Characteriza1on
∏ Filter
γ Classifica1on
+
+
Growth Rate = 125%
Data Suppor1ng parameter(s) Output Operator Type
+
Classifica1on method
Property required
PaBern
User loca1on
…
… … …
… …
…
… Match= 42%
92
• IF 𝑢𝑖 𝑖𝑠𝑖𝑛 𝑧𝑗 𝑇𝐻𝐸𝑁 𝑐𝑜𝑛𝑛𝑒𝑐𝑡 (𝑢𝑖, 𝑛𝑒𝑎𝑟𝑒𝑠𝑡(𝑢𝑖, 𝑟𝑘))
1) 𝑖𝑠𝑖𝑛(𝑢𝑖, 𝑧𝑗) →𝑚𝑎𝑡𝑐ℎ(𝑢𝑖, 𝑟𝑘)) 𝑓:(𝑈×𝑍)→(𝑈×𝑅)
• U = Users • Z = Personalized Situa1ons • R = Resources
2) 𝑛𝑒𝑎𝑟𝑒𝑠𝑡(𝑢𝑖, 𝑟𝑘)=𝑎𝑟𝑔𝑚𝑖𝑛 (𝑢𝑖.𝑙𝑜𝑐, 𝑟𝑘.𝑙𝑜𝑐𝑠)
93
12/5/12 94
Billions of data sources. Selec0ng and combining appropriate sources to detect situa0ons. Interac0ons with different types of Users
Decision Makers Individuals
12/5/12 95
Front End GUI
NewDataSource
NewQuery
E-‐mageStream
E-‐mage Stream
E-‐mage Stream
Data Cloud
Back End Controller
Stream Query Processor
Data IngestorRegisteredData
Sources
RegisteredQueries
Raw Spatial Data Stream
API Calls
Raw DataStorage
Personalized Alert Unit
AlertRequest
User Info
12/5/12 96
11/28/2012 97
+setBagOfWords()
TwitterWrapper
+setBagOfWords()+setColors()
FlickrWrapper
+setURL()+setTheme()+setParas()
Wrapper
+hasNext() : bool+next() : STTPoint
STTPointIterator
DBSTTPointIterator
+hasNext() : bool+next() : <unspecified>
Iterator
-‐timeWindow : long-‐syncTime : long-‐latUnit : double-‐longUnit : double-‐swLat : double-‐swLong : double-‐neLat : double-‐neLong : double
FrameParameters
-‐Parameterize
1 1
-‐theme : String-‐start : Date-‐end : Date-‐latUnit : double-‐longUnit : double-‐swLat : double-‐swLong : double-‐neLat : double-‐neLong : double-‐image
Emage
11..*
1..*
+hasNext() : bool+next() : <unspecified>
Iterator
+hasNext() : bool+next() : Emage
EmageIterator
11
1..*
-‐theme : String-‐value : double-‐start : Date-‐end : Date-‐latUnit : double-‐longUnit : double-‐latitude : double-‐longitude : double
STTPoint
-‐initResolution-‐finalResolution
ResolutionMapper
1 1
+hasNext() : bool+next() : <unspecified>
Iterator
+hasNext() : bool+next() : Emage
STMerger
+setURL()+setTheme()+setParas()
VisualImageIterator
CSVWrapper KMLWrapper
11/28/2012 98
Situa0onal controller
• Goal • Macro Situa1on • Rules
Micro event e.g. “Arrgggh, I
have a sore throat” (Loc=New York, Date=12/09/10)
Macro situa0on
Control Ac0on “Please visit nearest CDC
center at 4th St immediately”
Date=12/09/10
Alert Level=High
Level 1 personal threat + Level 3 Macro threat -‐> Immediate ac0on 12/5/12 99
• What personal informa1on can be shared? • How should it be shared to benefit the user? • Developing an architecture for personal informa1on management.
102
Asthma Threat level
Allergy reports Pollen Count
⊕
∏
Emage (Pollen Level)
Δ
Visual-‐ Pollen level
Air Quality
∏
Emage (AQI.)
Δ
Visual-‐ Air quality
∏
Emage (Number of reports)
Δ
TwiOer-‐Allergy
c ϵ {Low, mid, high}, [USA, 6 hrs, 0.1x 0.1]
Weather.com, [USA, 6 hrs, 0.1x 0.1]
TwiOer API, [USA, 6 hrs, 0.1x 0.1]
Pollen.com, [USA, 6 hrs, 0.1x 0.1]
Macro situa1on model
103 /
Personal threat level c ϵ {Low, mid,
high} γ
Physical exer0on Asthma threat level
⊕
TPS (Funf)
Δ
Funf-‐ac0vity
Phone sensors, (relaxMinder app),
[USA, 6 hrs, 0.1x 0.1]
EventShop
∏ Normalize (0, 100)
And
Classifica0on: Thresh(30,70)
∏ Normalize (0, 100)
[USA, 6 hrs, 0.1x 0.1]
TPS (Asthma)
∏ UserLoc
Personal threat level c ϵ {Low, mid,
high} γ
Physical exer0on
Asthma threat level
⊕
TPS (Funf)
Δ
Funf-‐ac0vity
Phone sensors,
(relaxMinder app), [USA, 6 hrs, 0.1x 0.1]
EventShop
∏ Normalize (0, 100)
And
Classifica0on: Thresh(30,70)
∏ Normalize (0, 100)
[USA, 6 hrs, 0.1x 0.1]
TPS (Asthma)
∏ UserLoc
104
12/5/12 107
Flood level - Shelter
Flood Level Shelter
Classify (Flood level - Shelter)
12/5/12 108
12/5/12 Proprietary and Confiden1al, Not For Distribu1on 109
Outline • Introduc1on • Social Systems • Real Time Social Systems • Designing Real Time Systems • Situa1on Recogni1on • Concept recogni1ons • Personalized Situa1ons • EventShop
• Going Forward
• Social observa1ons are now possible with liBle latency.
• Now possible to design social systems with feedback.
• Situa1on Recogni1on and Need-‐Availability iden1fica1on of resources becomes a major challenge.
• EventShop is a step in the direc1on of implemen1ng Social Life Networks.
Useful Links • Demo:
– hBp://auge.ics.uci.edu/eventshop • Data Defini1on Language Schema
– hBp://auge.ics.uci.edu/eventshop/documents/EventShop_DDL_Schema
• Query Language Schema – hBp://auge.ics.uci.edu/eventshop/documents/EventShop_QL_Schema
• Example Query in JSON – hBp://auge.ics.uci.edu/eventshop/documents/EventShop_Example_Query
11/28/2012 112
Thanks for your 1me and aBen1on.
For ques1ons: [email protected]