HCI and AI

HCI and AI

• HCI is:– Human-computer

interaction– Letting humans and

computers do what they do best

– Overview, zoom and filter, details on demand

• AI is:– Artificial intelligence– Emulating human

behavior using computers

– Performing tasks as a representative of the user

Software Agents

• Work on behalf of users within the electronic world

• Perform repetitive tasks, watch and respond to events, learn from user’s actions

A Simple Agent

• Email filters with if-then rules– if (to=mccricks) then (priority=1)– if (to=cs3724) then (priority=2)– if (to=cs2604) then (priority=3)– if (cc=mccricks) then (priority=4)

• Determine actions performed on certain kinds of mail messages

An Early Agent: EAGER

• Problem: how to display generalizations to user• Solution: anticipate and automate repetitive

tasks– EAGER works invisibly until it detects a pattern in

the user’s actions– Once a pattern is detected, EAGER uses highlighting

to show what it expects the user to do next– Once the user is confident that EAGER knows what

to do, s/he can allow EAGER to complete the task

• Developed by Allan Cypher

What EAGER Does• Observes user actions• Logs high level events• Detects loops • Tries to anticipate user

actions• If the user sees that

EAGER is anticipating the right pattern, EAGER can complete the rest of the task

EAGER Operation• EAGER operates within a

HyperCard stack environment• The user is not necessarily aware

of EAGER’S presence• As the user goes about editing

the HyperCard stack, EAGER logs the high-level events the user performs

• High-level events include typing, cutting, copying, deleting and pasting text, adding, deleting and moving cards, etc.

EAGER Logging

• EAGER looks for similar sequences of high-level events

• Example: creating an index • In this case, the user has

selected the subject line of a message and pasted it onto a new card, called “Subject Lists”

EAGER detects repetitions

• Repetitions can be detected if:– Commands are of the same type– Objects are of the same type– Objects fit a pattern

• Sequences of days of week• Linear sequences of integers• Similarly spaced screen positions• Text that is positioned similarly

within the same fields

EAGER Anticipation

• When similar sequences of events are detected, EAGER assumes that an iteration of a loop took place

• EAGER instantiates the next iteration of the loop

User Feedback

• EAGER shows the user what it thinks the next action will be by highlighting, displaying small popup windows, etc.

• Actions do not require user interaction so as not to disrupt the work flow

EAGER Task Completion

• Once the user is confident that EAGER’S prediction is correct, s/he can click on the EAGER icon to let EAGER take over

• Because of the users’ discomfort with abandoning control when EAGER takes over, several options are provided: – Complete the task– Do one iteration of the loop– Do one step (typing, cutting, pasting,

etc.)

EAGER Summary

• EAGER acts for the user without direct input

• Actions are represented in a way that maps very closely to the way the user performs them

• EAGER minimizes failures– Generalizations are represented through

instantiations that are directly related to the task at hand, making it easier for the user to verify the correctness of the generalizations

– The EAGER icon only appears when it is able to automate the task

Recommender Systems

• Mediate, support, and automate the process of sharing recommendations

• Generates communities of people with common interests

• Examples:– Eat at restaurant with lots of patrons– Rent movie that a friend liked– Buy album voted “Best of the Year”

Issues for Recommenders

• Preferences– Source– How obtained– Explicit or implicit– Incentives

• Roles– Roles for “H” and “C”– Fixed or changing– Distinct or unique

• Algorithms– Recs to use– Weighting/

computation

• HCI– Presentation– Representing weights– Conveying meaning– Lists, visualization,

annotation

Types of Recommenders

• Content-based– Use preferences of the

seeker– Find items similar to

ones user liked in past

• Recommendation support– Tools to help users

share recommendations

• Social data mining– Implicitly find

preferences from activity records (Usenet, system logs)

– Visualize results

• Collaborative filtering– Seekers express

preferences– System matches people

with similar taste

Content-Based Recommender

• Letizia acts as an advance scout for Web browsing:– It watches your Web browsing to try to learn

what topics you are interested in• Formulates “queries” dynamically/incrementally

– While you are reading a Web page, Letizia searches the neighborhood of the page to discover other pages you might be interested in

• Does “search” dynamically/incrementally

Letizia is a “channel surfing” interface for the

Web

Advantages of Letizia

• While you search “wide”, Letizia searches “deep”

• Uses the time that you spend reading a page to anticipate what you might interested in

• Filters out “junk”• Maintains persistence of interest• Good at discovering serendipitous

connections

Collaborative Filtering

• Systems that recommend products to users

• Queries: – What would I like?– Would I like ‘Pulp Fiction’?

• Collaborative filtering– Users provide ratings– Answer queries by relating ratings of user

with those of others

A Recommender Dataset

Movie Abby

Bernie

Charles

Austin Powers

Braveheart Castaway Don Juan Demarco

Emma

A Simple Recommender

• A user can recommend to another if a simple majority of their common ratings agree:– Abby – Charles: 3/4 agree, OK– Abby – Bernie: 0/4 agree, Nope– Bernie – Charles: 1/3 agree, Nope

• Prediction: Charles would like ‘Braveheart’

ConnectionsCharles’ ratings connect him to Abby

Charles’ connection to Abby connects him to ‘Braveheart’

AbbyCharles

D.J. Demarco

Castaway

Emma

Braveheart

Hammocks

Graph structure that indicates commonality of two people’s ratings

Could also show agreement

width – number of common ratings

Nearest Neighbor Algorithms

Only take recommendations from immediate neighbors

Abby ‘Star Wars’

Hammock Paths

Recommendation by “friends of friends”

‘The Cry of the Owl’Abby

length – number of hammocks to artifact

Social Network GraphConnections by common ratings

width 2

width 3

AI/HCI Summary

• Communities often at odds as to the best way to balance tasks

• Several overlapping (complementary?) areas

• AI often generates new problems for HCI researchers

Hypermedia and the Web

Bush’s Hypertext Vision

• Vannevar Bush, 1945 “As We May Think”

• Vision of post-war activities, Memex

• “…when one of these items is in view, the other can be instantly recalled merely by tapping a button”

Nelson’s Hypertext

• Coined “hypertext” in discussing his universal library and docuverse

• Had vision of a Xanadu system with hypergrams (branching pictures), hypermaps (with transparent overlays), and branching movies

• Many concepts adopted in WWW

Early Commercial Systems

• Knowledge Systems’ KMS– One or two frames of text/graphics– Links (tree/annotation) to additional

information

• Xerox PARC’s NoteCards– Cue card metaphor– Resizable but non-scrollable

• Apple’s HyperCard– Deck of cards metaphor– Links to other cards/programs

Hyperties

• Uses electronic encyclopedia metaphor• Indices and table of contents list

contents of information space• History lists show recently visited

pages• No syntactic entry means no error

messages (and less flexibility?)• Used in help systems, books

Shneiderman’s Golden Rules

of HypertextChoose projects where:1. There is a large body of information in

numerous fragments2. The fragments relate to each other3. The user needs only a small fraction

of the fragments at a time

Hypertext Guidelines

• Know the users and their tasks

• Ensure that meaningful structure comes first

• Apply diverse skills• Repect information

chunking

• Show interrelationships

• Ensure simplicity in traversal

• Design each screen carefully such that they can be grasped easily

• Require low cognitive load