cs415 human computer interactionmercury.pr.erau.edu/~siewerts/cs415/.../fall-2016/... · – verify...
TRANSCRIPT
![Page 1: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/1.jpg)
October 30, 2016 Sam Siewert
CS415Human Computer Interaction
Lecture 10 – Advanced HCIUniversal Design & Intro to Cognitive
Models
![Page 2: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/2.jpg)
Summary of Thoughts on ITSCollective Wisdom of Our Classes (2015, 2016)– Self driving Cars will become ubiquitous by:
– Revolutionary vs. Evolutionary from Driver Assistance2015 Class
– Evolutionary [3] – “cars will be fully self-driving once people get more used to assisted driving”, “I think it will continue to advance from driver assistance …”, “2015-25 – more driver assistance, 2025-35 – fully capable bad driving assistance, 2040 or so – fully (self)-driving cars”, “slowly evolve”
– Revolutionary [0] – Not much in engineering is revolutionary, it just appears as such from time to time to general public (they don’t know of our private toil)
2016 Class (TBD)– Evolutionary [TBD]– Revolutionary [TBD]
Sam Siewert 2
Year 2020 2025 2030 2040 Out of2015 class 1 1 1 1 6
2016 class 0 1 3 1 5
![Page 3: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/3.jpg)
AssignmentsAssignment #4 – Posted, Last Practice Lab (Cameras and OpenCV)
Assignment #5 – Propose Group Project (Groups of 3)
Assignment #6 – Project
Final Oral Exam – Presentation of Project
Sam Siewert 3
![Page 4: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/4.jpg)
This Week … Chapter 10 – Introduction to Universal Design– Making Computers Available to Everyone– Computer Solutions for Prosthetics– Active Learning Discussion – The Human Cyborg?
Chapter 12 – Cognitive Models– GOMS – Goal, Operators, Methods and Selection– Goal - describing what the user wants to achieve– Operators - basic actions that the user must perform in order to
use the system– Methods - several ways in which a goal can be split into sub-
goals– Selection – choice of method and prediction of which will be
used
Sam Siewert 4
![Page 5: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/5.jpg)
Assignment #4Use of Digital Cameras with Embedded Linux (V4L2/UVC/USB) - LinuxTV
Applications in Intelligent Transportation, Augmented Reality, Communications (e.g. Skype, Apple Facetime, WeChat)
Jetsons Have OpenCVinstalled (2.4 or 3.1)
Sam Siewert 5
Laptop Demo of Example Transforms
![Page 6: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/6.jpg)
Assignment #4 - ObjectivesPlay with the Cameras and Cheese First– Verify Hardware– Get a Feel for Performance– Capabilities
Learn about Camera Devices and Device Interfaces
Develop a Camera Application
Consider Processing for Display and Interaction
Human Aspects – Color, Frame Rate, Segmentation
Sam Siewert 6
![Page 7: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/7.jpg)
Assignment #4 DemoReal-time Camera Scene Analysis– Uncompressed, Un-encoded Camera Frames (Not MPEG)– Transforms RGB or Graymap XY Pixel Array Into Segments and
Major Components in Frames– Eliminate Backround (Difference of Frames – Current, Historical)– Segments Analyzed (Recognition) – e.g. Gesture (Thumbs Up,
Thumbs Down)
Sam Siewert 7
Segmenting Linear Elements Segmenting Circular Elements
![Page 8: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/8.jpg)
Universal Design and Embedded HCIChapter 10 – Introduction to Universal Design– Making Computers Available to Everyone– Computer Solutions for Prosthetics– Active Learning Discussion – The Human Cyborg?
Popular NewsTed Talk on Color BlindnessTed Talk – We are Already Cyborgs
Multi-modal Systems– Speech– Audio– Touch– Handwriting– Gestures
Sam Siewert 8
![Page 9: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/9.jpg)
Universal DesignDesign for Diversity– People with Sensory, Physical or Cognitive Impairment– Age Groups– Varying Cultures and Backgrounds– Localization – Languages
7 Principles of Universal Design1. Equitable Use – No User is Excluded2. Flexibility – Methods of Use, Pace, Precision, Ability3. Intuitive – Wide Range of Knowledge and Experience4. Perceptible Information – Graphic, Verbal, Text, Touch5. Error Tolerance – Minimize Impact of Mistakes (e.g. Undo)6. Low Physical Effort – Comfortable, Minimal Fatigue7. Size and Space – Sitting or Standing, Body Size, Posture,
Mobility Sam Siewert 9
![Page 10: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/10.jpg)
SpeechGeneration – Text to Speech - SynthesisRecognition – Speech to Text, to Syntax and Semantics
Sam Siewert 10
![Page 11: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/11.jpg)
Speech RecognitionBetter Today, But Still in Limited Use
Google – Demonstration Page
Sam Siewert 11
![Page 12: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/12.jpg)
Speech SynthesisScreen Readers, Web Readers
http://www.bobby-approved.com/
Sam Siewert 12
![Page 13: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/13.jpg)
Uses for Speech Recognition and Synthesis
Where and When to Use
Sam Siewert 13
![Page 14: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/14.jpg)
Haptic InterfacesHaptics - Touch– Cutaneous (skin tactile sensations)– Kinesthetics – movement perception and position
Current Use – Surgical Training, e.g. LapSim
Sam Siewert 14
![Page 15: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/15.jpg)
Handwriting SystemsRecord – e.g. Signatures (FedEx, UPS, DocuSign, …)AOS Swipe in Settings (Hybrid Handwriting/Typing)Recognize Input– No Systems Provide General Cursive Script Recognition– Some Success with Character at a Time Input– Apple Newton (Limited Success)
Touch Screen Typing
Sam Siewert 15
Apple Newton – Stylus PDA
![Page 16: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/16.jpg)
Gesture RecognitionCameras (Active or Passive) with Depth Mapping, Segmentation and TrackingData Gloves – Rotation (Yaw, Pitch, Roll), Translation (X,Y,Z), Rates of Translation/Rotation, and Finger Flexures
Camera Alone, Glove Alone, or Sensor Fusion
Glove is Intrusive, but Improves Accuracy
Challenge – Full Speed American Sign Language Recognition
Sam Siewert 16
![Page 17: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/17.jpg)
3D Active Computational Photometry Concept (Rev-A + TI Kit)
Sam Siewert 17
AnalogCamera #1
RGB(Visible)
Altera FPGACVPU
(Computer Vision Processing Unit)
MobileSensor Network
Processor(TI OMAP, Atom)
Networked Video
Analytics
HD DigitalCamera Port(Snapshot)
USB 2.0, PCIeHost Channels
Flash SD Card
AnalogCamera #2
(Near Infrared)
TI DLP Light-crafter Kithttp://www.ti.com/tool/dlplightcrafter
Depth Map
IR Pattern Projection
https://www.cs.purdue.edu/homes/aliaga/cs635-10/lec-structured-light.pdf
Photo credits and reference:Dr. Daniel Aliaga, Purdue Universityhttps://www.cs.purdue.edu/homes/aliaga/
![Page 18: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/18.jpg)
3D Computer Vision TransformsLong Range ( > 5 meters) Using Passive Binocular Methods– Impractical to Project from a UAV or Long Range Observer– Requires Image Registration– Accurate Camera Intrinsic (Camera Characteristics) & Extrinsic (e.g. Baseline)
Short Range ( < 5 meters), Structured IR Light Projection for RGB-D– Compare to ASUS Xtion and PrimeSense – Off-the-Shelf– Robust Depth Maps with Less Noise– Showing Significant Promise to Improve CV Scene Segmentation and Object
Recognition Compared to 2D– “Change Their Perception”, By Xiaofeng Ren, Dieter Fox, and Kurt Konolige,
IEEE RAS, December 2013.
Sam Siewert 18
Noise in Passive Depth Maps
Robust Active Depth Map“Change Their Perception”, By Xiaofeng Ren, Dieter Fox, and Kurt Konolige, IEEE RAS, December 2013.
![Page 19: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/19.jpg)
Design for Users with DisabilitiesMinority of Market, but Support Required
Visual ImpairmentHearing ImpairmentPhysical ImpairmentSpeech ImpairmentCognitive Impairments – Dyslexia, Autisim Sam Siewert 19
![Page 20: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/20.jpg)
Cultural and Age ConsiderationsElderly, Age 50+ TabletChildren, Kids Tablet – e.g. IPAD Mini 2, Amazon Fire Kids Edition, etc.Language Localization – Fundamental Translation, Keyboard, Graphics for Native Language Support
Cultural Differences – E.g. Nodding or Shaking Head in Agreement
Global Marketplace – High Value Today with Globalization
Sam Siewert 20
![Page 21: CS415 Human Computer Interactionmercury.pr.erau.edu/~siewerts/cs415/.../Fall-2016/... · – Verify Hardware – Get a Feel for Performance – Capabilities Learn about Camera Devices](https://reader033.vdocuments.us/reader033/viewer/2022060604/60587c292d828f202a20726c/html5/thumbnails/21.jpg)
Cognitive Models - IntroductionHierarchical Models – Task and Goal Structure
Linguistic Models – System Grammar
Physical and Device Models – Human Motor Skills
Cognitive Models – Combine All 3 of the Above
Sam Siewert 21