epistenet: facilitating programmatic access & processing of semantically related data
TRANSCRIPT
![Page 1: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/1.jpg)
Epistenet: Facilitating Programatic Access & Processing of Semantically Related Data
Sauvik DasCarnegie Mellon University
Jason WieseUniversity of Utah
Jason HongCarnegie Mellon University
![Page 2: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/2.jpg)
Overview
Motivation
Key features
Motivating Example
Conclusion
![Page 3: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/3.jpg)
Goal
Facilitate the programmatic access to and processing of semantically related mobile personal data.
![Page 4: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/4.jpg)
Goal
Facilitate the programmatic access to and processing of semantically related mobile personal data.
![Page 5: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/5.jpg)
personal data other datacapturable data about the user
capturable data not about the user
call logslocationbrowser historyactivity
battery levelconnectivityprocessing load
![Page 6: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/6.jpg)
Goal
Facilitate the programmatic access to and processing of semantically related mobile personal data.
![Page 7: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/7.jpg)
Phone GPS People
Personal data is currently isolated in source-of-origin silos…
![Page 8: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/8.jpg)
Epistenet, instead, semantically structures mobile personal data in knowledge graph…
![Page 9: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/9.jpg)
Goal
Facilitate the programmatic access to and processing of semantically related personal data.
![Page 10: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/10.jpg)
and provides a unified query interface to access this data.
![Page 11: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/11.jpg)
Timestamp > 7/7/15
Attribute Constraint Filter
Ontology => SensorReading
Ontology Constraint Filter
Reference Node
Relative Filters
Related Node
Developers can query for personal data through expressive filter mechanisms.
![Page 12: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/12.jpg)
Usage
EpistenetObject eo = uqi.getMostRecent( new OntologyClassFilter(OntologyClass.Communication));
Get a random correspondence
EpistenetObject[] rel = uqi.get( new RelativeFilter(eo).createdWithinRange(5*60*1000L);
Get all other data within 5 minutes of correspondence
![Page 13: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/13.jpg)
Overview
Motivation
Key features
Motivating Example
Conclusion
![Page 14: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/14.jpg)
Motivation
Mobile personal data underlies many of the most interesting application concepts of the present and future.
![Page 15: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/15.jpg)
Examples
Smart app launchersIntelligent assistants
Life loggers eHealth
![Page 16: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/16.jpg)
Personal data access
Location, calendar, interests, contacts
Application usage, time of day, activity,
communications
Location, time, application usage
Heart rate, activity logs, food logs, self-reports
![Page 17: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/17.jpg)
Problem
Today, mobile personal data application development difficult is difficult because…
![Page 18: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/18.jpg)
Queries based on source of origin, not
meaning
Semantic relationships between data lost
Data Silos
![Page 19: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/19.jpg)
Persistence dramatically different between log
and sensor data
Programming complexity: event-based programming
vs. database queries
Sensor / log disparity
![Page 20: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/20.jpg)
Data silos Sensor / log disparity
Queries based on origin, not
meaning
Semantic relationships
lost
Persistence vastly different,
lose context
Programming complexity
Challenges
![Page 21: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/21.jpg)
Data silos Sensor / log disparity
Queries based on origin, not
meaning
Semantic relationships
lost
Persistence vastly different,
lose context
Programming complexity
Challenges
Epistenet
Knowledge graph
Relative, semantic and descriptive
query filters
Historical sensor data around log
data events
Unified query interface
![Page 22: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/22.jpg)
Data silos Sensor / log disparity
Queries based on origin, not
meaning
Semantic relationships
lost
Persistence vastly different,
lose context
Programming complexity
Challenges
Epistenet
Knowledge graph
Relative, semantic and descriptive
query filters
Historical sensor data around log
data events
Unified query interface
![Page 23: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/23.jpg)
Overview
Motivation
Key features
Motivating Example
Conclusion
![Page 24: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/24.jpg)
Key features
Knowledge graph
Unified Query Interface
Relative, semantic and descriptive filters
Histories of sensor and log information
![Page 25: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/25.jpg)
Knowledge graph
Epistenet organizes personal data into a single knowledge graph.
![Page 26: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/26.jpg)
Ontology
The Epistenet ontology organizes personal data semantically. Data providers can contribute personal data under any ontology class.
MessageCommunic
ation
Person
Sensor reading
Phonecall Location
Accelerometer
![Page 27: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/27.jpg)
Knowledge graph objects
Epistenet objects are individual personal data points (e.g., a phone call, a location ping, a set of accelerometer readings).
![Page 28: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/28.jpg)
Knowledge graph objects
Descriptive attributes
Meta attributes
Ontology classes
(e.g., caller, duration)
(persistence, timestamp, permissions)
![Page 29: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/29.jpg)
Unified query interface
![Page 30: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/30.jpg)
Epistenet provides a single interface to access all personal data in the knowledge graph.
![Page 31: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/31.jpg)
How can it be used?
uqi.get(filter) .getRandom(filter) .getN(filter, N) .getMostRecent(filter)
. . .
![Page 32: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/32.jpg)
How can it be used?
uqi.get(filter) .getRandom(filter) .getN(filter, N) .getMostRecent(filter)
. . .
![Page 33: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/33.jpg)
Filters
Filters are sieves on the personal data graph that developers use to isolate personal data points they want.
![Page 34: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/34.jpg)
Attribute-constraint filters
Sender is not unknownnew AttributeConstraintFilter().addNonEqualsConstraint(“sender”, “unknown”)
Constrain any meta or descriptive attributes of an object.
![Page 35: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/35.jpg)
Ontology-constraint filters
OntologyClass is “Location”
Constrain ontology class.
new OntologyConstraintFilter(OntologyClass.Location)
![Page 36: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/36.jpg)
Ontology-constraint filters
OntologyClass is “SensorReading” or any subclass
Constrain ontology class. Can also traverse ontology.
new OntologyConstraintFilter(OntologyClass.SensorReading, true)
![Page 37: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/37.jpg)
Relative filters
Constructs queries relative to other personal data points. Thus, any personal data point is an index into the knowledge graph.
Reference Node Related Node
![Page 38: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/38.jpg)
+ - xunion (or) intersect (and) except (xor)
Filter compositions
![Page 39: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/39.jpg)
Historical Sensor Data
![Page 40: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/40.jpg)
Ontology-constraint filters
Epistenet also captures sensor histories around logged data points to bridge the persistence disparity. Also allows developers to get sensor data from before install-time.
![Page 41: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/41.jpg)
Overview
Motivation
Key features
Motivating Example
Conclusion
![Page 42: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/42.jpg)
Motivating Example: Autobiographical Authentication
![Page 43: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/43.jpg)
![Page 44: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/44.jpg)
DescriptionThree steps:
Find a “fact” to ask a question about
Create an answer list
Provide contextual hints
![Page 45: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/45.jpg)
Find a fact
EpistenetObject fact =uqi.getRandom(sms.intersectCompositWith(atTime));
Filter sms =new OntologyConstraintFilter(OntologyClass.SMSMessage);
Filter atTime =new MetaConstraintFilter().timestampWithinRange(lt, ht);
![Page 46: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/46.jpg)
Create answer listFilter similar =new RelativeFilter(fact).sameOntologyClass().createdBefore();
EpistenetObject[] answers = uqi.get(similar);
![Page 47: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/47.jpg)
Provide contextual hintsFilter hintFilter =new RelativeFilter(fact).differentOntologyClass().createdWithinRange(10*60*1000L);
EpistenetObject[] hints = uqi.get(hintFilter);
![Page 48: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/48.jpg)
Overview
Motivation
Key features
Motivating Example
Conclusion
![Page 49: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/49.jpg)
Epistenet organizes mobile personal data into a knowledge graph.
Epistenet affords easy access to subgraphs of the knowledge graph via the UQI.
Epistenet bridges the data persistence and query gaps between sensor and log data.
Conclusion
![Page 50: Epistenet: Facilitating Programmatic Access & Processing of Semantically Related Data](https://reader031.vdocuments.us/reader031/viewer/2022030208/58a68a721a28ab31478b49bb/html5/thumbnails/50.jpg)
Epistenet: Facilitating Programatic Access & Processing of Semantically Related Data
Sauvik DasCarnegie Mellon University
https://gitlab.com/scyrusk/epistenet
@scyrusk || [email protected]
http://sauvik.mew/ Jason Wiese and Jason Hong