life logging melekam tsegaye shaun bangay alfredo terzoli research area: wearable, pervasive and...
TRANSCRIPT
Life <--> LoggingLife <--> Logging
Melekam TsegayeMelekam TsegayeShaun BangayShaun BangayAlfredo TerzoliAlfredo Terzoli
Research area: Wearable, Pervasive and Ubiquitous ComputingResearch area: Wearable, Pervasive and Ubiquitous Computinghttp://www.cs.ru.ac.za/research/students/g98t4414/researchhttp://www.cs.ru.ac.za/research/students/g98t4414/research
25/11/200425/11/2004
Life loggingLife logging
Recording/capturing everything you see Recording/capturing everything you see and do. and do.
Past, present and future eventsPast, present and future events Why ? Why ?
for the sake of improving your memoryfor the sake of improving your memory Your memory defines who you are!Your memory defines who you are!
Amnesia?Amnesia?
Forms of life logsForms of life logs
Manual paper diariesManual paper diaries Electronic diaries on PDAsElectronic diaries on PDAs Audio/Video recordingsAudio/Video recordings Emergence of the Weblog - blogEmergence of the Weblog - blog
Current TrendsCurrent Trends
Weblogs are mostly text and image based Weblogs are mostly text and image based e.g. Blogger.com (blog via email)e.g. Blogger.com (blog via email)
Next natural transition is audio bloggingNext natural transition is audio blogging Individuals make an audio recording, publish Individuals make an audio recording, publish
to a web server e.g. audioblog.comto a web server e.g. audioblog.com Hardware needed to start life loggingHardware needed to start life logging
Camera, microphone, local storage Camera, microphone, local storage (Digicam/Cell phone)(Digicam/Cell phone)
Web serverWeb server
audioblog.comaudioblog.com
Effortless audio loggingEffortless audio logging recording audio to a remote server from recording audio to a remote server from
your web browseryour web browser cell phonecell phone emailemail
audioblog.com recently introduced a video audioblog.com recently introduced a video blogging extensionblogging extension
Life-log applicationsLife-log applications
Nokia lifeblogNokia lifeblog Software for organizing video clips recorded Software for organizing video clips recorded
using a cell phone camerausing a cell phone camera
Life-log applications(2)Life-log applications(2) Microsoft sense camMicrosoft sense cam
Passive Capture, improved participation in eventsPassive Capture, improved participation in events• ACM workshop on Continuous archival and retrieval of personal experiences New ACM workshop on Continuous archival and retrieval of personal experiences New
York, USA October 2004 York, USA October 2004 Correlations and relationships from sensors used to manage captured imagesCorrelations and relationships from sensors used to manage captured images
• light intensity sensorlight intensity sensor• x/y tilt (tilt sensor )x/y tilt (tilt sensor )• motion sensor to avoid taking blurry pictures (accelerometer)motion sensor to avoid taking blurry pictures (accelerometer)• temperaturetemperature• infra red sensorinfra red sensor• clockclock• locationlocation• MMC card storage MMC card storage • 2xAA cell batteries, 12 hours2xAA cell batteries, 12 hours• Size of a pagerSize of a pager
Useful: quick replay of a day’s eventsUseful: quick replay of a day’s events
Our workOur work
wPim - Wearable audio enhanced wPim - Wearable audio enhanced personal information management systempersonal information management system
Record audio wherever you areRecord audio wherever you are Organize people and events (past, future and Organize people and events (past, future and
present)present) Retrieve using time of capture and textual Retrieve using time of capture and textual
descriptorsdescriptors
Current WorkCurrent Work
Once you capture your memories how do you retrieve Once you capture your memories how do you retrieve segments of interest for later use ? segments of interest for later use ?
using time, location ?using time, location ? We want to analyze data produced by multiple sensors We want to analyze data produced by multiple sensors
at time of av capture and extract useful information at time of av capture and extract useful information to be used to auto tag complex streams such as video and audio to be used to auto tag complex streams such as video and audio
Assembling a large number of sensors and conducting Assembling a large number of sensors and conducting experiments can be difficultexperiments can be difficult
Some sensors are not available in a convenient formSome sensors are not available in a convenient form We waste too much time dealing with hardware issues and We waste too much time dealing with hardware issues and
produce poor software solutionsproduce poor software solutions
Current Work(2)Current Work(2) The approach we take in our attempts to solve the problems The approach we take in our attempts to solve the problems
mentioned earlier ismentioned earlier is To define multiple virtual sensors that sense environmental and user To define multiple virtual sensors that sense environmental and user
statesstates make the virtual sensors generate data (essentially, simulate the make the virtual sensors generate data (essentially, simulate the
process sensing data) process sensing data) extract useful contextual information from the generated dataextract useful contextual information from the generated data make use of the extracted information to build a contextual tag databasemake use of the extracted information to build a contextual tag database use the information in the tag database to automatically tag complex use the information in the tag database to automatically tag complex
data streams such as video and audio streamsdata streams such as video and audio streams Not simulation only, we want to be able use a combination of virtual Not simulation only, we want to be able use a combination of virtual
and real sensors to run our experimentsand real sensors to run our experiments E.g. A wearable life logging facility that uses a combination of virtual and E.g. A wearable life logging facility that uses a combination of virtual and
real sensorsreal sensors
PrototypePrototype
sensors for: Video, Audio, Locationother sensors and I/O devices through a multi-channel (a2d) data capture device.
ProblemsProblems
Social issuesSocial issues PrivacyPrivacy LawsLaws Hostility and fear from societyHostility and fear from society First person experience capture vs big brother First person experience capture vs big brother
scenarioscenario Hardware limitationsHardware limitations
Wireless network connectivity is not yet ubiquitous, Wireless network connectivity is not yet ubiquitous, bandwidth limitationsbandwidth limitations
long lasting power sourceslong lasting power sources comfortable wearable hardwarecomfortable wearable hardware