stanford hci group / cs376

http://cs376.stanford.eduJeffrey Heer · 26 May 2009

User InterfaceToolkits

Final Project PresentationsTuesday June 9, 3:30-6:30pm, 104 Gates

8 minute presentations 6 min for research, 2 min for questions Start with an overview:

1 sentence statement of your research result 1 slide / 4 sentences of what you did and why

Rest of time on details. Assume audience is familiar with HCI issues: focus on your work

2

cryptonomicon.com/beginning.html

In the beginning…

The Xerox Alto (1973)

Event-Driven UIs

Old model (e.g., UNIX shell, DOS) Interaction controlled by system, user

queried for input when needed by system

Event-Driven Interfaces (e.g., GUIs) Interaction controlled by user System waits for user actions and then

reacts More complicated programming and

architecture

Periodic Table of Motif Widgets

7

Toolkit Example: Java Swing GUI toolkit with a widget set and an

API

8

User Interface Builders

Event Dispatch Loop

Event Queue• Queue of input events

Event Loop (runs in dedicated thread)

• Remove next event from queue• Determine event type• Find proper component(s)• Invoke callbacks on components• Repeat, or wait until event arrivesComponent

• Invoked callback method• Update application state• Request repaint, if needed

Mouse moved (t0,x,y)

Event Dispatch

Window

Panel

TextArea Panel

Button Button

Label

Event Queue• Mouse moved (t0,x,y)• Mouse pressed (t1,x,y,1)

• Mouse dragged (t2,x,y,1)

• Key typed (t3, ‘F1’)• …(queues and dispatches incoming events in a dedicated thread)

/* callback for TextArea */public void mouseMoved(e) { // process mouse moved event}

11

Success of Tools

Today’s tools are highly successful Window Managers, User Interface Toolkits,

and Interface Builders are ubiquitous Most software built using them Are based on many years of HCI research:

Brad A. Myers. “A Brief History of Human Computer Interaction Technology.” ACM interactions. Vol. 5, no. 2, March, 1998. pp. 44-54.

Why Tools?

The quality of the interfaces will be higher.

Why? Rapid prototyping. Easier to incorporate changes motivated

by evaluation. Re-use affords investment in high quality

tools. Consistency of interface design. Enable collaboration among specialists.

12

Why Tools?

The user interface will be easier to create and maintain.

Why? Less code to write due to component re-use. Better modularization, separation of concerns Tools may abstract complex systems or

algorithms. Easier to port an application to different

hardware or software environments if device dependencies are isolated in the user interface tool.

13

14

What should tools do?Help design the interface given a specification of

the tasks.Help implement the interface given a design.Help evaluate the interface after it is designed.

Create easy-to-use interfaces.Allow the designer to rapidly investigate different

designs.Allow non-programmers to create user interfaces.Provide portability across different machines and

devices.Be easy to use themselves.

15

Tools might do: Provide sets of standard UI components Guide the implementation Help with screen layout and graphic design. Validate user inputs Handle user errors Handle aborting and undoing of operations Provide help and prompts Deal with field scrolling and editing Insulate the application from all device

dependencies and the underlying software and hardware systems.

Support features in the interface that allow the end user to customize the interface.

Application Types

What application domains are “deserving” of specialized toolkit support?

16

17

Application TypesEach has own unique UI style, and implementation challenges

Word processors Drawing programs (CAD/CAM) Painting programs Mail readers Spreadsheets Programming environments / code editors WWW Interactive games Visualizations Automated-teller machines (ATM) Virtual Reality Multi-media

Video Animation

Controlling machinery

18

DiamondSpin Toolkit

Toolkit for tabletop user interfaces [Shen, Vernier, Forlines, Ringel] CHI

’04

19

20

Tabletop UI Needs

Multi-user support Identity-aware widgets Multiple menus Public and personal spaces Resolving conflicting actions

Arbitrary orientation of UI elements Techniques to control orientation and

layout Rotation sensitive components

21

iStuff Toolkit Physical UI components for

ubiquitous computing environments (multiple users, devices, and applications)

[Ballagas, Ringel, Stone, Borchers], CHI ‘03

22

iStuff Design

iStuff components Device + proxy (“smarts” are in the

proxy) PatchPanel

Translate between iStuff events and application-specific events

Run-time retargetable events Address dimension mismatches

23

24

Discussion of Themes

Address the useful & important aspects of UIs Narrower tools have been more successful

than ones that try to do “everything” Do one thing well

Threshold / Ceiling Research systems often aim for high ceiling Successful systems often seem to instead

aim for a low threshold Impossible to have both?

25

Threshold & Ceiling

Difficultyof

Use

Sophistication of what can be created

Goal

HyperCard

Visual BasicDirector (v6)

HyperTalk

xCmds

Basic

C Programming

Lingo

C Programming

Programming in C

MFC

Click and

Create

26

Discussion of Themes, cont.Path of Least Resistance

Tools should guide implementers into better user interfaces

Goal for the future: do this more?

Predictability Programmers do not seem willing to release

control Especially when system may do sub-optimal

things

Moving Targets Long stability of Macintosh Desktop paradigm

has enabled maturing of tools

Evaluating User Interface Tools “An API is a user interface where

programmers are the users” ? Evaluate toolkit as you would a UI?

Factors Expressiveness (Ceiling) Development Rate (of skilled user) Learning Rate (to become skilled) Performance Portability

27

28

The Future of Interface ToolsSupporting…

Prototyping Collaboration Evaluation

of interfaces built with tools of tools themselves – how to prototype, test,

iterate?

Emerging interface styles, such as mobile recognition-based UIs (speech, pens, vision) multiple devices

29

Current Research Challenges Complex design space

e.g., do we have to update the toolkit every time someone creates a new sensor or actuator?

Ambiguous input Speech, gestures, computer vision, etc.

aren’t recognized as accurately as mouse clicks. Should the toolkit handle the recognition?

30

Summary Toolkits provide reusable interface

components to simplify UI development Toolkit trap: it’s tempting to only make

UIs that the toolkit makes easy, instead of making what’s best for a specific app

Toolkit types: WIMP (Garnet, Swing, Motif, etc) Specialty (Phidgets, iStuff, Papier-Mache,

DiamondSpin, GroupKit, Peripheral Displays Toolkit, etc)

Next Time: Adaptive Interaction

You should use Sharon’s online paper viewer for the next set of readings.

31