1 tutorial t4 - programming by example techniques brad a. myers brad a. myers human computer...

1

Tutorial T4 - Programming by Example Techniques

Brad A. MyersBrad A. MyersHuman Computer Interaction Institute

School of Computer ScienceCarnegie Mellon UniversityPittsburgh, PA 15213-3891

(412) [email protected]

http://www.cs.cmu.edu/~bam

© - Brad Myers© - Brad MyersCMU - HCI InstituteCMU - HCI Institute 2

VL’2000 Tutorial: PBEVL’2000 Tutorial: PBE

Contents:

Abstract & Bio Slides from Talk Sources for Further Information Articles

Early VP & PBE Taxonomy Demonstrational Interfaces paper Gamut Paper Intelligence in Demonstrational Interfaces

Abstract & Bio Slides from Talk Sources for Further Information Articles

Early VP & PBE Taxonomy Demonstrational Interfaces paper Gamut Paper Intelligence in Demonstrational Interfaces



Abstract

This tutorial covers the broad range of techniques used in programming systems that use "Programming By Example (PBE)." Sometimes called "Programming By Demonstration" or "Demonstrational Interfaces", this refers to systems that allow the user to give examples of the desired behaviors and contents, and the system generalizes from those examples so the result will work in different contexts. Some PBE systems use sophisticated artificial intelligence techniques to generalize, others use simple rule-based heuristics, and still others require the user to perform the generalization. Applications of PBE techniques have been to general-purpose programming, programming of user interfaces, defining animations, creating macros or scripts for text editing, designing business charts, desktop file manipulation, hypertext browsing, programming for kids, etc. The tutorial will include many demonstrations and videos of research and commercial uses of programming-by-example techniques.

This tutorial covers the broad range of techniques used in programming systems that use "Programming By Example (PBE)." Sometimes called "Programming By Demonstration" or "Demonstrational Interfaces", this refers to systems that allow the user to give examples of the desired behaviors and contents, and the system generalizes from those examples so the result will work in different contexts. Some PBE systems use sophisticated artificial intelligence techniques to generalize, others use simple rule-based heuristics, and still others require the user to perform the generalization. Applications of PBE techniques have been to general-purpose programming, programming of user interfaces, defining animations, creating macros or scripts for text editing, designing business charts, desktop file manipulation, hypertext browsing, programming for kids, etc. The tutorial will include many demonstrations and videos of research and commercial uses of programming-by-example techniques.



About the Presenter

Brad Myers is a Senior Research Scientist in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University, where he is the principal investigator for various projects, including Demonstrational Interfaces, User Interface Software, Natural Programming, and the Pebbles Palm Pilot Project. He and his students have created about a dozen systems that use programming-by-example techniques. He is the author or editor of over 190 publications, including "Creating User Interfaces by Demonstration" and "Languages for Developing User Interfaces," and he is on the editorial board of five journals, including the Journal of Visual Languages and Computing. His research interests include User Interface Development Systems, user interfaces, Programming by Example, programming languages for kids, Visual Programming, interaction techniques, window management, and programming environments. He belongs to SIGCHI, ACM, IEEE Computer Society, IEEE, and Computer Professionals for Social Responsibility. He will also be a keynote speaker for VL'2000.

Brad Myers is a Senior Research Scientist in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University, where he is the principal investigator for various projects, including Demonstrational Interfaces, User Interface Software, Natural Programming, and the Pebbles Palm Pilot Project. He and his students have created about a dozen systems that use programming-by-example techniques. He is the author or editor of over 190 publications, including "Creating User Interfaces by Demonstration" and "Languages for Developing User Interfaces," and he is on the editorial board of five journals, including the Journal of Visual Languages and Computing. His research interests include User Interface Development Systems, user interfaces, Programming by Example, programming languages for kids, Visual Programming, interaction techniques, window management, and programming environments. He belongs to SIGCHI, ACM, IEEE Computer Society, IEEE, and Computer Professionals for Social Responsibility. He will also be a keynote speaker for VL'2000.



Topics

What is Programming by Example? Applications of Programming by Example Overview of PBE Systems Issues in PBE Systems

User Interfaces for giving examples User Interface for feedback to show the program Implementation architectures

What is Programming by Example? Applications of Programming by Example Overview of PBE Systems Issues in PBE Systems

User Interfaces for giving examples User Interface for feedback to show the program Implementation architectures



What is Programming by Example?

Direct Manipulation allows properties to be set by directly moving objects with the mouse and setting properties Example: interface builders, Visual Basic Limited to static parts of the interface No way to point at objects that will be drawn by the user How set the color of an object in Visual Basic at run

time?

Direct Manipulation allows properties to be set by directly moving objects with the mouse and setting properties Example: interface builders, Visual Basic Limited to static parts of the interface No way to point at objects that will be drawn by the user How set the color of an object in Visual Basic at run

time?



Demonstrational Tools

Demonstrational Tools allow the user to operate on example objects which represent objects that are created at run-time. Size of the boxes depends on labels Demonstrational systems allow the user to operate on

examples, and then generalize to produce a general-purpose procedure or prototype.

Demonstrational Tools allow the user to operate on example objects which represent objects that are created at run-time. Size of the boxes depends on labels Demonstrational systems allow the user to operate on

examples, and then generalize to produce a general-purpose procedure or prototype.

VL’2000Seattle

2000

IEEE



"Examples"

Draw an example of the objects that will be created at run time

Draw objects in approximately the right places, and systems creates general constraints

Give examples of the steps of the algorithm

Draw an example of the objects that will be created at run time

Draw objects in approximately the right places, and systems creates general constraints

Give examples of the steps of the algorithm

VL’2000Seattle

2000Prototype

IEEE



Generalization

Is a hard problem, which is why you don't see many commercial products that do this.

Research on: Algorithms for generalization

“Artificial Intelligence” User Interfaces to help generalization

Is a hard problem, which is why you don't see many commercial products that do this.

Research on: Algorithms for generalization

“Artificial Intelligence” User Interfaces to help generalization



Motivations

Bring programming to more people Automate repetitive tasks Expand how much of the interface can be specified

interactively. Interactive editors are much faster to use than

programming with toolkits Frameworks improve productivity by factors of 3 to 5,

interactive tools by factors of 10 to 50! It might take an hour to draw an interface interactively,

compared to days to program it. It is much more natural to specify the graphical parts of

applications using a graphical editor.

Bring programming to more people Automate repetitive tasks Expand how much of the interface can be specified

interactively. Interactive editors are much faster to use than

programming with toolkits Frameworks improve productivity by factors of 3 to 5,

interactive tools by factors of 10 to 50! It might take an hour to draw an interface interactively,

compared to days to program it. It is much more natural to specify the graphical parts of

applications using a graphical editor.



Example in Emacs

Useful to create scripts for repetitive actions: Rename all gif files to have OLD at the front of their

names

Useful to create scripts for repetitive actions: Rename all gif files to have OLD at the front of their

names

archesonly.gifcircuitafter.gifcircuitbefore.gifemacsrecord.gifgeneralizeloc.gifgeneralizeobj.gifheightgen.gifinvocationdb.gifleftgen.gifmoveselection.gif

mv archesonly.gif OLDarchesonly.gifmv circuitafter.gif OLDcircuitafter.gifmv circuitbefore.gif OLDcircuitbefore.gifmv emacsrecord.gif OLDemacsrecord.gifmv generalizeloc.gif OLDgeneralizeloc.gifmv generalizeobj.gif OLDgeneralizeobj.gifmv heightgen.gif OLDheightgen.gifmv invocationdb.gif OLDinvocationdb.gifmv leftgen.gif OLDleftgen.gifmv moveselection.gif OLDmoveselection.gif



Example in Microsoft Word

The fillparagraphs macro changes RETURNs into spaces, unless there are 2 in a row.


Lots of short ¶lines¶can be filled and ¶ combined. ¶¶ But two RETURNS¶signal a new ¶paragraph. ¶

Lots of short lines can be filled and combined.¶

But two RETURNS signal a new para-graph.



Applications of Programming by Example

General-purpose programming Programming for kids Programming of user interfaces Creating macros or scripts for text editing Designing business charts Desktop file manipulation Hypertext browsing

General-purpose programming Programming for kids Programming of user interfaces Creating macros or scripts for text editing Designing business charts Desktop file manipulation Hypertext browsing



General Issues for All PBE Systems

Amount of Inferencing None -> Lots Predictability vs. amount of help

If Inferencing: Number of examples Negative Examples Hints or other help from user Feedback on the Inferencing

Control structures: loops, conditionals Variables How show user the resulting program?

Amount of Inferencing None -> Lots Predictability vs. amount of help

If Inferencing: Number of examples Negative Examples Hints or other help from user Feedback on the Inferencing

Control structures: loops, conditionals Variables How show user the resulting program?



Overview of Some PBE Systems

Chronological Bias towards my systems

Easier to get pictures / video!

There are many other systems not covered here See references

Chronological Bias towards my systems

Easier to get pictures / video!

There are many other systems not covered here See references



Pygmalion

David C. Smith’s PhD thesis Watch What I Do, chapter 1.

1975 in Smalltalk No inferencing Work out program using icons and example

values Primary contributions:

Programming with icons Working out program using examples General motivation

David C. Smith’s PhD thesis Watch What I Do, chapter 1.

1975 in Smalltalk No inferencing Work out program using icons and example

values Primary contributions:

Programming with icons Working out program using examples General motivation



Pygmalion Picture



Emacs

R. Stallman, 1987 in Teco, Lisp, C, etc. Easily record a sequence of keystrokes No inferencing Very light-weight and extremely useful

R. Stallman, 1987 in Teco, Lisp, C, etc. Easily record a sequence of keystrokes No inferencing Very light-weight and extremely useful



Emacs Example

^x(â^k"mv "^y" OLD"^y^n^x)û1000^xe

^x(â^k"mv "^y" OLD"^y^n^x)û1000^xe

archesonly.gifcircuitafter.gifcircuitbefore.gif…

mv archesonly.gif OLDarchesonly.gifmv circuitafter.gif OLDcircuitafter.gifmv circuitbefore.gif OLDcircuitbefore.gif…



Emacs Contributions

Primary contributions: Clearly demonstrate benefits of macros by example Quick and easy macro creation for “keyboard macros” Fully programmable using Lisp for further End-User-

Programming (EUP)

Primary contributions: Clearly demonstrate benefits of macros by example Quick and easy macro creation for “keyboard macros” Fully programmable using Lisp for further End-User-

Programming (EUP)



Rehearsal World

William Finzer and Laura Gould, 1984 in Smalltalk Watch What I Do, chapter 4.

No inferencing Metaphor of “actors” on a “stage”

Wings for off-screen computation

PBE by opening an “eye” icon Primary Contributions:

Stage metaphor: on-stage, wings, etc. Eye metaphor: for easy controlling of when recording

William Finzer and Laura Gould, 1984 in Smalltalk Watch What I Do, chapter 4.

No inferencing Metaphor of “actors” on a “stage”

Wings for off-screen computation

PBE by opening an “eye” icon Primary Contributions:

Stage metaphor: on-stage, wings, etc. Eye metaphor: for easy controlling of when recording



Rehearsal World Pictures



SmallStar

Daniel Halbert, 1980-1984, in Smalltalk Watch What I Do, chapter 5.

Create programs in the Star office system Desktop (file) manipulations

No inferencing Introduced notion of “data descriptions”

Specify what data using dialog boxes

Edit straight-line program to add conditionals and loops

video (6 min)

Daniel Halbert, 1980-1984, in Smalltalk Watch What I Do, chapter 5.

Create programs in the Star office system Desktop (file) manipulations

No inferencing Introduced notion of “data descriptions”

Specify what data using dialog boxes

Edit straight-line program to add conditionals and loops

video (6 min)



SmallStar Contributions

Primary contributions Data descriptions concept and UI UI Techniques for control structures Demonstration in a complex environment

Primary contributions Data descriptions concept and UI UI Techniques for control structures Demonstration in a complex environment



Peridot

Brad Myers, 1987, in Lisp Watch What I Do, chapter 6.

Create user interface widgets (buttons, scroll bars) Inferred:

Graphical constraints among the objects. Control structures such as iteration over all the items in a

menu How the mouse affects the graphics, such as that the check

mark should follow the mouse.

Feedback using question and answer No representation of the program video (8 min)

Brad Myers, 1987, in Lisp Watch What I Do, chapter 6.

Create user interface widgets (buttons, scroll bars) Inferred:

Graphical constraints among the objects. Control structures such as iteration over all the items in a

menu How the mouse affects the graphics, such as that the check

mark should follow the mouse.

Feedback using question and answer No representation of the program video (8 min)



Peridot Contributions

Primary Contributions Inferencing using condition-action rules Question-and-answer feedback technique shown not

successful Demonstrational technique for graphical positions Coverage of UI behaviors

Primary Contributions Inferencing using condition-action rules Question-and-answer feedback technique shown not

successful Demonstrational technique for graphical positions Coverage of UI behaviors



Eager Allen Cypher, 1990 in HyperCard

Watch What I Do, chapter 9.

Inferred loops from repeated sequence of actions Had to match actions and look for generalizations of

values “Anticipatory Feedback” of what it thought the user

would do next User can say “do the rest” No way to know what the step after would be Users weren’t comfortable without knowing the stopping

condition

video (4 min)

Allen Cypher, 1990 in HyperCard Watch What I Do, chapter 9.

Inferred loops from repeated sequence of actions Had to match actions and look for generalizations of

values “Anticipatory Feedback” of what it thought the user

would do next User can say “do the rest” No way to know what the step after would be Users weren’t comfortable without knowing the stopping

condition

video (4 min)



Eager Contributions

Primary Contributions Anticipatory Feedback Algorithms for matching actions where parameters differ Showed that people were not comfortable telling it to go

without knowing what it would do and when it would stop.

Primary Contributions Anticipatory Feedback Algorithms for matching actions where parameters differ Showed that people were not comfortable telling it to go

without knowing what it would do and when it would stop.



Garnet

Brad Myers, 1988 - 1992 in Lisp Watch What I Do, chapter 10.

User interface development environment Lapidary: Create UI elements, less inferencing

video (2.5 min) Contributions:

UI for defining graphical constraints More sophisticated behaviors by example

Brad Myers, 1988 - 1992 in Lisp Watch What I Do, chapter 10.

User interface development environment Lapidary: Create UI elements, less inferencing


UI for defining graphical constraints More sophisticated behaviors by example



Garnet, cont. C32

video (2 min) Contributions:

Simple generalizations on copy-paste Gilt: Graphical styles for widgets


Extend text metaphor for tabs and Gilt: transformations of widget values

video (5:15 min) Contributions:

Inferring parameters of procedures from examples

C32 video (2 min) Contributions:

Simple generalizations on copy-paste Gilt: Graphical styles for widgets


Extend text metaphor for tabs and Gilt: transformations of widget values

video (5:15 min) Contributions:

Inferring parameters of procedures from examples



Chimera

David Kurlander, 1990, Lisp+C+Postscript Watch What I Do, chapter 12.

Graphical search and replace Graphical “comic strip” representation for all

actions Generalization of macros selected from the

transcript Explicit specification of control structures

video (15 min)

David Kurlander, 1990, Lisp+C+Postscript Watch What I Do, chapter 12.

Graphical search and replace Graphical “comic strip” representation for all

actions Generalization of macros selected from the

transcript Explicit specification of control structures

video (15 min)



Chimera Contributions

Primary Contributions: Graphical Search and Replace Inferring more sophisticated constraints from examples Comic-book metaphor for command history & programs Editing command history as undo/redo

Primary Contributions: Graphical Search and Replace Inferring more sophisticated constraints from examples Comic-book metaphor for command history & programs Editing command history as undo/redo



DEMO

David A. Wolber and Gene L. Fisher, 1991, 1992 David A. Wolber and Gene L. Fisher, "Demonstrational Technique for Developing

Interfaces with Dynamically Created Objects," Proceedings UIST'91, Nov, 1992, Monterey, CA, pp. 89-97.

Gene L. Fisher, Dale E. Busse, and David A. Wolber, "Adding Rule-Based Reasoning to a Demonstrational Interface Builder," Proceedings UIST'92, Nov, 1991, pp. 221-230.

Create (pieces of) user interfaces First system to support dynamic creation of

objects. Infers graphical relationships by examples of edits Only simple relationships provided No feedback of code

David A. Wolber and Gene L. Fisher, 1991, 1992 David A. Wolber and Gene L. Fisher, "Demonstrational Technique for Developing

Interfaces with Dynamically Created Objects," Proceedings UIST'91, Nov, 1992, Monterey, CA, pp. 89-97.

Gene L. Fisher, Dale E. Busse, and David A. Wolber, "Adding Rule-Based Reasoning to a Demonstrational Interface Builder," Proceedings UIST'92, Nov, 1991, pp. 221-230.

Create (pieces of) user interfaces First system to support dynamic creation of

objects. Infers graphical relationships by examples of edits Only simple relationships provided No feedback of code



DEMO Contributions

Primary Contributions Concept of “Stimulus-Response” programming Dynamic creation of objects

Primary Contributions Concept of “Stimulus-Response” programming Dynamic creation of objects



Tourmaline

Brad Myers and Andrew Werth, 1990-1992. Lisp, Ness, WordBasic


Infer composite styles for text formatting E.g., more than one format in the header Infer roles for each piece of text, and formatting for that

role

Infer tables from a drawing video (7:22 min) Contributions

Roles

Brad Myers and Andrew Werth, 1990-1992. Lisp, Ness, WordBasic


Infer composite styles for text formatting E.g., more than one format in the header Infer roles for each piece of text, and formatting for that

role

Infer tables from a drawing video (7:22 min) Contributions

Roles



Marquise Brad Myers, Rich McDaniel, Dave Kosbie, 1993 in Lisp

(part of Garnet) Myers B., McDaniel, R. and Kosbie, D.. "Marquise: Creating Complete User

Interfaces by Demonstration," CHI'94. Amsterdam, Netherlands, April 24-29, 1993. pp. 293-300.

Do more by demonstration just show the way that the interface should operate.

Demonstrate when the behaviors start & feedback Mouse button does one of 10 things Demonstrate both behavior and conditions

Built-in support for palettes and modes. Feedback using sentences with clickable words video (3 min)

Brad Myers, Rich McDaniel, Dave Kosbie, 1993 in Lisp (part of Garnet) Myers B., McDaniel, R. and Kosbie, D.. "Marquise: Creating Complete User

Interfaces by Demonstration," CHI'94. Amsterdam, Netherlands, April 24-29, 1993. pp. 293-300.

Do more by demonstration just show the way that the interface should operate.

Demonstrate when the behaviors start & feedback Mouse button does one of 10 things Demonstrate both behavior and conditions

Built-in support for palettes and modes. Feedback using sentences with clickable words video (3 min)



Marquise Contributions

Primary Contributions Four modes Ability to demonstrate mouse actions by changing

modes in the middle using the keyboard Introduction of icons for “mouse droppings” Demonstration that text with pop-up embedded menus

was not a great idea

Primary Contributions Four modes Ability to demonstrate mouse actions by changing

modes in the middle using the keyboard Introduction of icons for “mouse droppings” Demonstration that text with pop-up embedded menus

was not a great idea



Pursuit

Francemary Modugno, 1993-1995 in Lisp Watch What I Do, chapter 20.

Generalize desktop programs Inferred generalizations of data descriptions

Comic book representation for programs video (6:43 min)

Francemary Modugno, 1993-1995 in Lisp Watch What I Do, chapter 20.

Generalize desktop programs Inferred generalizations of data descriptions

Comic book representation for programs video (6:43 min)



Pursuit Contributions

Primary Contributions Generalizing the comic-strip metaphor Handling of errors, pop-ups in the macro

Meta-dialog boxes Showing the importance of the representation for the

program

Primary Contributions Generalizing the comic-strip metaphor Handling of errors, pop-ups in the macro

Meta-dialog boxes Showing the importance of the representation for the

program



Gold

Brad Myers, 1994 in Lisp CHI’94, pp. 106-111

Generalize business charts Give one or two examples of desired elements

video (7:00 min) Primary Contributions:

Generalizing graphics with no “program” Feedback and editing by labeling Handling of special items

Brad Myers, 1994 in Lisp CHI’94, pp. 106-111

Generalize business charts Give one or two examples of desired elements

video (7:00 min) Primary Contributions:

Generalizing graphics with no “program” Feedback and editing by labeling Handling of special items



InferenceBear & Grizzly Bear

Martin Frank, 1994-1996 Martin Frank and Jim Foley, "Model-Based User Interface Design by

Example and By Interview,” UIST'93. Atlanta, GA, Nov. 3-5, 1993, pp. 129-

137.

Create UIs by demonstration and direct edit User control through dialog boxes, edit using textual

language: EET

First UI builder with multiple examples Snapshots of before and after Multiple examples Add more positive examples to cause generalization Negative examples to specify exceptions

Martin Frank, 1994-1996 Martin Frank and Jim Foley, "Model-Based User Interface Design by

Example and By Interview,” UIST'93. Atlanta, GA, Nov. 3-5, 1993, pp. 129-

137.

Create UIs by demonstration and direct edit User control through dialog boxes, edit using textual

language: EET

First UI builder with multiple examples Snapshots of before and after Multiple examples Add more positive examples to cause generalization Negative examples to specify exceptions



InferenceBear & Grizzly Bear



InferenceBear Pictures



InferenceBear Contributions

Primary Contributions Inferencing algorithm which can handle linear equations Use of negative examples Studies showing negative examples are difficult

Primary Contributions Inferencing algorithm which can handle linear equations Use of negative examples Studies showing negative examples are difficult



Pavlov

David Wolber, 1995-present David Wolber, "Pavlov: Programming by Stimulus-Response

Demonstration," CHI'96. April 1996. pp. 252-259

Single positive example Little generalization

Score editor for feedback and editing Stimulus from mouse or time-based video

David Wolber, 1995-present David Wolber, "Pavlov: Programming by Stimulus-Response

Demonstration," CHI'96. April 1996. pp. 252-259

Single positive example Little generalization

Score editor for feedback and editing Stimulus from mouse or time-based video



Pavlov Contributions

Primary Contributions Score editor as representation Mouse-based stimulus

Primary Contributions Score editor as representation Mouse-based stimulus



StageCast Creator

(formerly called KidSim, Cocoa) Allen Cypher & David C. Smith;1994-present

http://www.stagecast.com

Formerly research at Apple, now commercial company:

Create animations and rough simulations Program using before and after pictures Can add additional control to rules No automatic generalization Video – on computer (3 min)

(formerly called KidSim, Cocoa) Allen Cypher & David C. Smith;1994-present

http://www.stagecast.com

Formerly research at Apple, now commercial company:

Create animations and rough simulations Program using before and after pictures Can add additional control to rules No automatic generalization Video – on computer (3 min) Video Clip Video Clip



StageCast Contributions

Primary Contributions Before-After rules by example Grouping of rules into sets:

Random ordered, first to succeed Bringing PBD to wider audience

Primary Contributions Before-After rules by example Grouping of rules into sets:

Random ordered, first to succeed Bringing PBD to wider audience



Gamut Rich McDaniel, 1996 - 1999

R.G. McDaniel and B.A. Myers, "Getting More Out Of Programming-By-Demonstration.” CHI'99. Pittsburgh, PA, May 15-20, 1999. pp. 442-449.

Domain: "board games" and educational software Goal: new interaction techniques so can infer more

complex behaviors E.g., how a piece can move in Monopoly / Chess

Reduce number of modes in UI “Do Something”, “Stop That”

New interaction techniques to provide hints Better inferencing algorithms video (4:19 min)

Rich McDaniel, 1996 - 1999 R.G. McDaniel and B.A. Myers, "Getting More Out Of Programming-By-

Demonstration.” CHI'99. Pittsburgh, PA, May 15-20, 1999. pp. 442-449.

Domain: "board games" and educational software Goal: new interaction techniques so can infer more

complex behaviors E.g., how a piece can move in Monopoly / Chess

Reduce number of modes in UI “Do Something”, “Stop That”

New interaction techniques to provide hints Better inferencing algorithms video (4:19 min)



Gamut Contributions

Primary Contributions UI techniques for positive and negative examples that

reduce modes Hints by asking to highlight Most powerful inferencing algorithm Most ambitious coverage by inference

Primary Contributions UI techniques for positive and negative examples that

reduce modes Hints by asking to highlight Most powerful inferencing algorithm Most ambitious coverage by inference



Topaz

Brad Myers, 1998, part of Amulet in C++ B.A. Myers. "Scripting Graphical Applications by Demonstration," CHI'98.

Los Angeles, CA, April 18-23, 1998. pp. 534-541.

Move Emacs techniques to graphical editing Move selection like a cursor Generalizations using dialog boxes video (3:00 min)

Brad Myers, 1998, part of Amulet in C++ B.A. Myers. "Scripting Graphical Applications by Demonstration," CHI'98.

Los Angeles, CA, April 18-23, 1998. pp. 534-541.

Move Emacs techniques to graphical editing Move selection like a cursor Generalizations using dialog boxes video (3:00 min)



Topaz Contributions

Primary Contributions Scripted macros for graphics like have for text Moving graphics selection like text cursor Dialog boxes for generations of position

Primary Contributions Scripted macros for graphics like have for text Moving graphics selection like text cursor Dialog boxes for generations of position

53

General Themes

User Interface for Giving Examples UI for feedback of Programs Implementation



User Interface for Giving Examples

Issue of Modes Hints to help with inferencing One versus multiple examples How specify negative examples

Issue of Modes Hints to help with inferencing One versus multiple examples How specify negative examples



User Interface for Feedback of Program

None (Peridot, Gamut, etc.) “Real” Code (Visual Basic for Word) Special languages

EET language for InferenceBear Comic strip languages (Chimera, Pursuit) “Score” (time-line) view (Pavlov) Before-after rules (Stagecast)

How edit the resulting program? Many only support: give up and start over

None (Peridot, Gamut, etc.) “Real” Code (Visual Basic for Word) Special languages

EET language for InferenceBear Comic strip languages (Chimera, Pursuit) “Score” (time-line) view (Pavlov) Before-after rules (Stagecast)

How edit the resulting program? Many only support: give up and start over



Example in Microsoft Word



Lots of short ¶lines¶can be filled and ¶ combined. ¶¶ But two RETURNS¶signal a new ¶paragraph. ¶

Lots of short lines can be filled and combined.¶

But two RETURNS signal a new para-graph.



Word Visual Basic code For “fillparagraphs” (removes RETURNS unless there are 2 in a row)

Public Sub MAIN()WordBasic.EditReplace Find:="^p^p", Replace:="%$%",

Direction:=0, MatchCase:=0, WholeWord:=0, PatternMatch:=0, SoundsLike:=0, ReplaceAll:=1, Format:=0, Wrap:=0

WordBasic.EditReplace Find:="^p", Replace:=" ", Direction:=0, MatchCase:=0, WholeWord:=0, PatternMatch:=0, SoundsLike:=0, ReplaceAll:=1, Format:=0, Wrap:=0

WordBasic.EditReplace Find:="%$%", Replace:="^p", Direction:=0, MatchCase:=0, WholeWord:=0, PatternMatch:=0, SoundsLike:=0, ReplaceAll:=1, Format:=0, Wrap:=0

End Sub

For “fillparagraphs” (removes RETURNS unless there are 2 in a row)

Public Sub MAIN()WordBasic.EditReplace Find:="^p^p", Replace:="%$%",

Direction:=0, MatchCase:=0, WholeWord:=0, PatternMatch:=0, SoundsLike:=0, ReplaceAll:=1, Format:=0, Wrap:=0

WordBasic.EditReplace Find:="^p", Replace:=" ", Direction:=0, MatchCase:=0, WholeWord:=0, PatternMatch:=0, SoundsLike:=0, ReplaceAll:=1, Format:=0, Wrap:=0

WordBasic.EditReplace Find:="%$%", Replace:="^p", Direction:=0, MatchCase:=0, WholeWord:=0, PatternMatch:=0, SoundsLike:=0, ReplaceAll:=1, Format:=0, Wrap:=0

End Sub



Implementation

Techniques for implementing PBE Especially for inferencing Rule-based heuristics More sophisticated AI algorithms

Pattern matching Decision trees

Techniques for implementing PBE Especially for inferencing Rule-based heuristics More sophisticated AI algorithms

Pattern matching Decision trees



Conclusions

PBE techniques are powerful but still “researchy” Many exciting research questions

A community of researchers addressing them

Some commercial applications now Increasing as computers get “smarter”

PBE techniques are powerful but still “researchy” Many exciting research questions

A community of researchers addressing them

Some commercial applications now Increasing as computers get “smarter”



Sources for Further Information

Special issue of Communications of the ACM on Programming by Example: March, 2000. vol. 43, no. 3.

WWW home page for programming by example: http://lieber.www.media.mit.edu/people/lieber/PBE/index.html

Watch What I Do, edited by Allen Cypher, co-edited by Daniel C. Halbert, David Kurlander, Henry Lieberman, David Maulsby, Brad A. Myers, and Alan Turransky, The MIT Press, Cambridge, Massachusetts.

New book, next year: Henry Lieberman, ed. Your Wish is My Command. 2001: Morgan Kaufmann

Special issue of Communications of the ACM on Programming by Example: March, 2000. vol. 43, no. 3.

WWW home page for programming by example: http://lieber.www.media.mit.edu/people/lieber/PBE/index.html

Watch What I Do, edited by Allen Cypher, co-edited by Daniel C. Halbert, David Kurlander, Henry Lieberman, David Maulsby, Brad A. Myers, and Alan Turransky, The MIT Press, Cambridge, Massachusetts.

New book, next year: Henry Lieberman, ed. Your Wish is My Command. 2001: Morgan Kaufmann

1 tutorial t4 - programming by example techniques brad a. myers brad a. myers human computer...

Documents

n applications of programming

programming of user

topics n

pbe systems n user interfaces

programming systems

examples n user interface

programming languages

visual programming