ai in end-user software engineering
DESCRIPTION
AI in End-User Software Engineering. Henry Lieberman, MIT (presenting) Raphael Hoffmann, Michael Toomim, U. Washington, Seattle. Roles for AI in End-User SoftEng. Supply knowledge end-user doesn't have Recognize user intent Plan generation and plan recognition - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/1.jpg)
Henry Lieberman • MIT Media Lab
AI in End-User Software Engineering
Henry Lieberman, MIT (presenting)
Raphael Hoffmann, Michael Toomim, U. Washington,
Seattle
![Page 2: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/2.jpg)
Henry Lieberman • MIT Media Lab
Roles for AI in End-User SoftEng
Supply knowledge end-user doesn't have
Recognize user intent
Plan generation and plan recognition
Automate common patterns of use
Generalize & Specialize
Manage multiple representations
Error checking and sanity checking
![Page 3: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/3.jpg)
Henry Lieberman • MIT Media Lab
U. Washington SoftBots & more
Planning-based software agents
Version-space Programming by Demonstration
Sequence prediction via Logic & Probability
Recommendation from software libraries
![Page 4: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/4.jpg)
Henry Lieberman • MIT Media Lab
SmartEdit - Lau
![Page 5: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/5.jpg)
Henry Lieberman • MIT Media Lab
Version Spaces
![Page 6: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/6.jpg)
Henry Lieberman • MIT Media Lab
Linked Editing
Alternative to content abstraction facilities
Powerpoint Master Slides
Style Sheets
"Symbols" vs. instances in Flash
Web templates, components
User stays within WYSIWYG paradigm
![Page 7: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/7.jpg)
Henry Lieberman • MIT Media Lab
Linked Editing
End-users "underuse" abstractions
Linked Editing
User edits concrete example
Generalizing over content rather than actions
Finds other instances of description
Performs "analogous" edits
Program code, Web pages, Vector graphics, Sheets
![Page 8: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/8.jpg)
Henry Lieberman • MIT Media Lab
Linked Editing
![Page 9: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/9.jpg)
Henry Lieberman • MIT Media Lab
Linked Editing
![Page 10: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/10.jpg)
Henry Lieberman • MIT Media Lab
Assieme - Customization Recommendation
Goal: Recommending scripts for application extensions: Emacs, Firefox
Collect keystrokes, file names, commands
Provide Web service for recommendation
Collab filtering, learning classifier
![Page 11: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/11.jpg)
Henry Lieberman • MIT Media Lab
Assieme - Customization Recommendation
![Page 12: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/12.jpg)
Henry Lieberman • MIT Media Lab
The crazy dream: Programming in English
Programming languages are too difficult for human beings
People can express procedures in English (or other natural language)
Why not program in natural language?
That was the original idea behind Cobol
(whereas Fortran tried to make programming like math)
![Page 13: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/13.jpg)
Henry Lieberman • MIT Media Lab
Childrens’ description of Pac-Man
![Page 14: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/14.jpg)
Henry Lieberman • MIT Media Lab
But what can you do if you can only partially understand English?
Need to make the interaction fail-soft
Rely on dialogue to resolve ambiguities and errors
Maybe don’t eliminate code entirely
Emphasize correspondence between language and code
An "outliner" for code
Application: Programming MOOs
![Page 15: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/15.jpg)
Henry Lieberman • MIT Media Lab
New resources for Common Sense Reasoning
Open Mind Common Sense knowledge base
ConceptNet Semantic Net
MontyLingua/LangUtils
![Page 16: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/16.jpg)
Henry Lieberman • MIT Media Lab
Metafor Interface
![Page 17: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/17.jpg)
Henry Lieberman • MIT Media Lab
Programmatic Semantics
English to “scaffolding” code
Roughly,
Turns nouns into classes and/or instances;
Verbs into functions
Adjectives into instance variables
Resolves anaphora
Untangles conditional and loops
Propagates context
![Page 18: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/18.jpg)
Henry Lieberman • MIT Media Lab
Narrative stances
I want to make a bar with a customer. (1st person programmer)
There is a customer in the bar. (3rd person objective)
I am a customer sitting on a stool. (1st person program)
The bartender said, “Here is a customer” (3rd person subjective)
![Page 19: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/19.jpg)
Henry Lieberman • MIT Media Lab
Refactoring
There is a bar. (atom)The bar contains two customers. (unimorphic list)It also contains a waiter. (unimorphic wrt. persons)It also contains some stools. (polymorphic list)The bar opens and closes. (class / agent)The bar is a kind of store. (inheritance class)Some bars close at 6pm. (subclass or instantiatable)
![Page 20: AI in End-User Software Engineering](https://reader035.vdocuments.us/reader035/viewer/2022062806/56814f7e550346895dbd30bc/html5/thumbnails/20.jpg)
Henry Lieberman • MIT Media Lab
Shameless Plug