human interface where the bits meet the flesh bits meets eyes and ears
DESCRIPTION
Human Interface where the bits meet the flesh bits meets eyes and ears. Think beyond the box. Tom Zimmerman IBM Almaden Research Center San Jose, California [email protected] October 2007. Research Theme. Transducers; Converting one thing into another - PowerPoint PPT PresentationTRANSCRIPT
Human InterfaceHuman Interfacewhere the bits meet the fleshwhere the bits meet the flesh
bits meets eyes and earsbits meets eyes and ears
Tom ZimmermanTom ZimmermanIBM Almaden Research CenterIBM Almaden Research Center
San Jose, CaliforniaSan Jose, [email protected]@almaden.ibm.com
October 2007October 2007
Think beyond the box
Research Theme Research Theme
Transducers; Transducers; Converting one thing into anotherConverting one thing into another User Interface, User Experience, First PersonUser Interface, User Experience, First PersonHooking people to gadgetsHooking people to gadgets– Bits Meet the FleshBits Meet the Flesh– Bits Meet Eyes & EarsBits Meet Eyes & Ears
Transducer ExamplesTransducer Examples– Dance into Music (dance synthesizer, suction cup)Dance into Music (dance synthesizer, suction cup)– Hand into Finger Spelling (DataGlove)Hand into Finger Spelling (DataGlove)– Sound into Light (Singing Coach)Sound into Light (Singing Coach)
Violin Home Health Monitoring Contact Microscope
VR Data Glove Personal Area Network Honda Smart Seat
Fantasy & Necessity Are the Mothers of Invention
Discrete EventsDiscrete Events
Converting continuous input into Converting continuous input into discrete events.discrete events.
Continuous input examplesbrightness, sound level, pressure
Discrete output; 0/1, off/on, mouse click
Comparator- Ideal SignalComparator- Ideal Signal
SensorContinuous Input
Quantized DigitizedOutput
Threshold
Sensor +_
Threshold
Digital Output
Comparator- Real World SignalComparator- Real World SignalNoise in input causes output jitterNoise in input causes output jitter
threshold
cross high for on cross low for off
Triggering Discrete Events Triggering Discrete Events with Hysteresiswith Hysteresis
Continuous Input (e.g. brightness,
sound level, pressure)
Quantized/DigitizedOutput
High threshold
Low threshold
Hysteresis, “play”
cross high for on cross low for off
TV Synth InterfaceTV Synth Interface
Hacking Remotes, Mice, Kbd
Circuit Board
4066Quad CMOS Switch
Photo Transistor
Comparatorw/ hysteresis
60 HzFilter
MouseKeyboardIR RemoteSynthesizer
TV
Sensor Signal Processing Switch
Mouse SensorsMouse Sensors• Microphone; Speaker Radiation Pattern• Light; Water Ripple Tank• Ultrasonics; Wavelength, Horn Radiation Pattern• Temperature; Weather Station• Wind speed; See Jet Stream, Car Aerodynamics• Pressure• Tilt• Rpm• Vibration• Pendulum; Simple Harmonic Motion• Acceleration; Seismograph, Earthquakes, Cars/Trucks• Voltage• Current• Resistance• Capacitance• Magnetic field • Electric field
• Left button • Right button• X direction• X increment• Y direction• Y increment
1 to 100 Hz
Wireless Doorbell Wireless Doorbell
Wireless Motion DetectorWireless Motion Detector
Working with Continuous OutputWorking with Continuous Output
PowerGlove Protoype (VPL 1983)PowerGlove Protoype (VPL 1983)
DataGlove DataGlove (Fiber Optic)(Fiber Optic)
Fiber Optic Polhemus Magnetic Fiber Optic Polhemus Magnetic Flex SensorsFlex Sensors Position & OrientationPosition & Orientation
sensor
source
Young Harvill
Head Mounted DisplayHead Mounted Display(NASA 1986)(NASA 1986)
PowerGlove (Mattel 1987)PowerGlove (Mattel 1987)
Computer Data Entry And 1.3 Million SoldManipulation Apparatus And Method $89 Retail
The Spirit Chair (MIT 1994)The Spirit Chair (MIT 1994)
Mediums use spirit cabinets to guide fieldsto contact spirits who make sounds ...
Data In Data Out
Near-Field Communication (MIT/IBM 1995)
Contagious Information Contagious Information (MIT 1995)(MIT 1995)
7/11Fred
Jan Bill
Car Seat Sensor (NEC 1998)Car Seat Sensor (NEC 1998)
Musical InstrumentsMusical Instruments
Human Actions
Musical
InstrumentPitchAmplitudeTimbreDuration
Vocalizer (Breakaway 1981)Vocalizer (Breakaway 1981)
Singing Coach (Exploratorium 1990)Singing Coach (Exploratorium 1990)
RePerformer (Zim/Wantman 1992)RePerformer (Zim/Wantman 1992)
Pen InputPen Input
Pen That Remembers Everything You Write Pen That Remembers Everything You Write (IBM 2000-2002)(IBM 2000-2002)
Premiere UI unobtrusive, low cognitive load, small, portable, quick, cheap
Universal any language
popular input methodclassic form factor
Modelesstext, graphics, doodle comment on document
direct manipulation
BUT hard to index, search,
archive, distribute
Other Pen SystemsOther Pen Systems
PDA; writing on plastic LCD; writing on glass
Tablet; invisible writing CrossPad; writing on 2 lbs.
Toy PensToy Pens
Battery (AAA)
InkCPU and
Flash Memory
CMOS ImagerLens
Normal or dot code paper
Pen Research (IBM 2002)
Ultrasonic Optical
IRDA
Electric Field Pen (IBM 1997)Electric Field Pen (IBM 1997)
Dynamic Signature VerificationDynamic Signature Verification“Sign and Go” (2003)“Sign and Go” (2003)
• Biometric verification using handwriting cadence (timing)• A means to electronically verify customer signatures • Customer signs on a digital signature capture station• Software checks signature against reference signatures and checks for major changes in signature
BiometricsBiometrics
Classification of BiometricsClassification of Biometrics
Physical Feature Physical Feature –Iris, face, finger, hand, DNAIris, face, finger, hand, DNA–Can alter presentationCan alter presentation
Behavior Trait Behavior Trait –Signature, voice, gait, keystrokeSignature, voice, gait, keystroke–Can change behaviorCan change behavior
Identification & VerificationIdentification & VerificationIdentificationIdentification– ““Who am I?”Who am I?”– 1:N matching (difficult, big search)1:N matching (difficult, big search)– Example:Example: 100k customers @ 0.1% error = 100 errors 100k customers @ 0.1% error = 100 errors– UseUse: Discover identity, Watch lists: Discover identity, Watch lists
VerificationVerification– ““Am I who I claim to be?”Am I who I claim to be?”– 1:1 matching (easier)1:1 matching (easier)– Requires identifier (not secure but unique)Requires identifier (not secure but unique)
token (loyalty card) token (loyalty card) memory (phone number)memory (phone number)
– Use:Use: Verify identity Verify identity
Performance MetricsPerformance MetricsFalse Accepts (FA) False Accepts (FA) – likelihood that someone may be falsely accepted likelihood that someone may be falsely accepted bad guy gets inbad guy gets in
False Rejects (FR)False Rejects (FR)– likelihood that a genuine user may be rejected likelihood that a genuine user may be rejected good guy gets rejectedgood guy gets rejected
Lower the FA and FR, the better the system performanceLower the FA and FR, the better the system performanceAcceptance threshold trade-off FA to FRAcceptance threshold trade-off FA to FREqual Error Rate when FA=FREqual Error Rate when FA=FRFailure to enroll, Failure to acquireFailure to enroll, Failure to acquire– Light eyes (iris), thin skin (finger), simple/changing signature, noisy Light eyes (iris), thin skin (finger), simple/changing signature, noisy
environment (voice)environment (voice)– Need backup/alternativeNeed backup/alternative
Example FA=2% FR=1%Example FA=2% FR=1%– 2 out of 100 good customers get rejected2 out of 100 good customers get rejected– 1 out of 100 bad guy attempts get in1 out of 100 bad guy attempts get in– If 1 in 1000 customers are bad guys, 1 forgery per 100,000 transactionsIf 1 in 1000 customers are bad guys, 1 forgery per 100,000 transactions
Biometric ComparisonBiometric Comparison
BiometriBiometricc
EER*EER* AdvantageAdvantage DisadvantageDisadvantage
IrisIris < 0.1%< 0.1% Most accurate, no contactMost accurate, no contact Critical alignment, Critical alignment, expensive reader expensive reader
FingerFinger 0.1-10%0.1-10% Wide use, standard Wide use, standard databases, competitive databases, competitive market, legal precedencemarket, legal precedence
Special reader, criminal Special reader, criminal association, gummy association, gummy attack, modest cost readerattack, modest cost reader
SignatureSignature 1.7-3%1.7-3% Uses existing POS signature Uses existing POS signature capture hardware, social capture hardware, social acceptance, signature dual acceptance, signature dual purpose (agreement & purpose (agreement & biometric)biometric)
Behavior (requires Behavior (requires cooperation), name cooperation), name changes, ergonomicschanges, ergonomics
VoiceVoice 2-10%2-10% Use with cell phonesUse with cell phones Sensitive to noisy Sensitive to noisy environment, requires environment, requires talkingtalking* Equal Error Rate estimates for relative comparison only. Actual results are highly
dependent on system design, equipment, user population and experience.
Tangible Programming Tangible Programming
Logo CardsLogo CardsBar code
Instruction
Usage
Definition
Example code
35mm slide holder
Students Programming With LOGO Playing Cards
Queuing and Scanning Programs into Computer
Programming Robots with Programming Robots with Playing CardsPlaying Cards
The Robosapian™ robot comes with a remote control to program robot movements.
The remote control is replaced with a deck of instruction cards and bar code reader.
Students choreograph movements by arranging the cards, then program the robot by scanning the cards with the bar code reader.
TelepresenceTelepresence
When you need to be there … but you don’t want to travel there
Dummy Head Electronics (IBM 2003)Dummy Head Electronics (IBM 2003)
Frame and Handle
VGA Camera
MegaPixel Camera
Tilt Gimble
Yaw Motor
Speaker Phone
Battery
Tiny PC w/WLAN
Ethernet AC PowerAnalog Phone Line
Ethernet
DIGITAL HEAD BASE STATION (x-ray view) (100 m range to head)
Digital Head ComponentsDigital Head Components
Techno HeadTechno Head
QVGA ScreenBinaural Mics
20 W SpeakerSeat Clamp
Digital Camera
Head AppearancesHead Appearances
Standards Committee Symposium Trade Show Booth
Breakfast Keynote Board Meeting Talk Conference Panel
Shopping BuddyShopping Buddy
Location-aware shopping cart provide real-time “you are here” navigation map and location-based content-Location-aware shopping cart provide real-time “you are here” navigation map and location-based content-driven offers such as coupons and recipes. driven offers such as coupons and recipes.
Infrared Receiver
Infrared Beacon
Wireless (Bluetooth)Barcode Scanner
WLAN
Museum InterfacesMuseum Interfaces
Sizzling JelloSizzling Jello
Bubble MachineBubble Machine
Bubble Strobe MachineBubble Strobe Machine
Questions???Questions???
Badge Computer (IBM 1998)Badge Computer (IBM 1998)
NZ Airport Travel Card (IBM 1998)NZ Airport Travel Card (IBM 1998)
PDA running XML Browser, linked by 802.11b WLAN
Keypad overlayTouch Pad
Camera with 270 deg swivel Microphones/Structured Light
FM antenna along perimeter Speakers
802.11 WLAN antennas running along each sideColor Screen (no touch)
Multi-Function PDA (IBM 2001)Multi-Function PDA (IBM 2001)
BioPDA Wireless Biometric TerminalBioPDA Wireless Biometric Terminal
Iris CameraFace & Document Camera
InfraRed Lights
802.11b WLAN
Intel XScale Processor
Numeric Keypad
Field Replaceable Biometric Module
Symbol 8000 Handheld Terminal
Fingerprint Reader
Using the Biometric Using the Biometric Wireless TerminalWireless Terminal
Capturing Face and Iris
Capturing Fingerprint
Lining up Iris Camera
lines up and frames Face Camera