UCI
Knowledge Management for Software Design
David F. Redmiles
David M. Hilbert, Jason Robbins, Michael Kantor, Shilpa Shukla
Information and Computer ScienceUniversity of California, IrvineIrvine, California 92697-3425
[email protected]://www.ics.uci.edu/~redmiles/ or
http://www.ics.uci.edu/eden/
UCI
Overview• Motivation and Problem Statement
• Approach — Human-Centered Software Development
• Usage Scenario
• Systems’ Theory, Implementation, Current Directions– Design: Argo Software Architecture Design
– Usage Monitoring: Expectation-Driven Event Monitoring
– Review: Knowledge Depot Project Awareness Tool
• Conclusions
UCI
Motivation and Problem Statement
UCI
Motivation• Software engineering has become harder because
software is being applied to increasingly complex problem situations.
• Complexity is measured in terms of people and workplace settings.
UCI
The Challenge of Increased Complexity in Software
• Changed Nature– Software applications have moved from predictable, static
applications to unpredictable, interactive tools.
• Expanded Scope– Software applications are no longer intended for a single user or
small group of highly trained users.
• New Problem Domains– Software applications are no longer restricted to “simple” problems
—mathematical models with a few input variables.
• Increased User Sophistication– Wide availability of personal computers has increased users’
awareness of computer potential.
UCI
Changed Nature• Software applications have moved from static to
interactive tools.
• Human behavior is “unpredictable”—people do not always use tools as anticipated by the developers.– Incomplete understandings of tools (system model).
– Incomplete understanding of problem situations (situation model).
– Other cognitive limitations (e.g., attention).
UCI
Expanded Scope• Software applications are no longer intended for a
single user or small group of highly trained users.– General-purpose tool sets are applied in a variety of situations in
mass markets.
– Special-purpose tools cannot assume long training periods because of high personnel turnover and frequent tool updates.
– In any tool, the number and variety of features implies that many are infrequently accessed.
• Moreover, new, cooperative work tools coordinate the activities of several people.
UCI
New Problem Domains• Software applications are no longer restricted to
“simple” problems—mathematical models with a few input variables.
• Technological advances make it possible to put software on every desk.– Tools must be able to coordinate with many dimensions of a
workplace: users’ relying on information from other artifacts in the work setting, other computer tools, and other people.
– New tasks challenge existing computer representations and algorithms.
– Models of tasks must be more rapidly developed.
UCI
Increased User Sophistication• Wide availability of personal computers has increased
users’ awareness of computer potential.
• Attractive interfaces in games and other personal software have raised users’ expectations.
• Competitive markets for many applications enable users to be “discretionary”—more demanding.
UCI
New Demands on Design• Interactive tools require a human-centered, not
artifact-centered, focus.It is more difficult for designers to understand how people will use tools in their workplace than it is for them to understand the domain the tools will support.
• User evaluation is essential to informing tool development.
Fluctuating requirements and other failures should be expected, good “fit” must evolve, new methods for collecting usage data and feedback can make evaluation more tenable and cost-effective.
UCI
New Demands on Design (cont.)• Design is an evolutionary process and continues after
delivery.In new problem domains, complexity unfolds incrementally. Moreover, after tools are introduced into a community, the users develop new approaches to their problems, hence new requirements for tools.
• Users and other stakeholders must share ownership of systems.
Acceptance of systems often fails for social and organizational reasons. Stakeholders must feel their intentions have been fairly negotiated.
UCI
New Demands on Design (cont.)• Developers need information about good design in
new domains.– Developers require knowledge both about problem domains and
potential solutions. Possible solutions grow daily with new software architecture techniques.
UCI
Approach• In a human-centered software development process,
feedback comes from all stakeholders.
• Stakeholders reflect on an artifact that evolves from sketchy designs to partial, and eventually, complete prototypes.
• Feedback from human stakeholders is facilitated by tools; tools can provide data about actual usage and knowledge evolved from previous development experiences.
UCI
Human-Centered Software Development
Design
UsageReview
ARGOEDEM
EDEMKNOWLEDGE DEPOT
incorporatereviews into
deploy prototype withexpectation agents
deposit observations
prototype and expectation agents
prototype with expectation agentsexpectation agents’ reports andother information
perform direct observations,interviews, other manualusability methods
perform other design reviews,including email discussions
design
UCI
Communication and Knowledge in the Lifecycle
• The human-centered lifecycle interprets software development as a process of communication (of knowledge):– Knowledge about what constitutes a good design is provided to
developers.
– Knowledge about actual system usage and user commentary is reported.
– Knowledge about system usage and other related design knowledge is discussed among stakeholders.
• Three software systems support these activities: Argo, EDEM, and Knowledge Depot.
UCI
Usage Scenario
UCI
Developing a Travel Voucher System
• A software developer architects a new application, a travel voucher system.
• Software developers author expectations about the usage of the system; they are deployed with the system.
• The agents monitor system usage and may solicit user comments.
• Usage information, comments, and other design information is collected and reviewed.
• Design knowledge provides feedback for a redesign of the system.
UCI
Design• A software developer architects a new application, a
travel voucher system.
UCI
Design and Usage• Software developers author expectations about the usage of
the system; they are deployed with the system.• This agent fires when a user edits the City or State fields
while the Zip field is empty
Trigger
Guards
Actions
}}}
AuthoredAgents {
UCI
Design and Usage• Selection of events
simplifies agent definition.
• Agents are deployed independently of applications.
UCI
Usage• The agents monitor system usage.
UCI
UsageAgent Notification (Optional)
• Agents may post messages.
UCI
UsageUser Response (Optional)
• Users may provide feedback.
UCI
Review• Usage information,
comments, and other design information is collected and reviewed.
UCI
Redesign• Design knowledge provides feedback for a redesign of
the system.
UCI
Summary• The changing nature of software places new demands on
software engineering.
• An empirically-driven, evolutionary and human-centered model is necessary.
• The human-centered lifecycle interprets software development as a process of communication (of knowledge):– Knowledge about what constitutes a good design is provided to
developers.– Knowledge about actual system usage and user commentary is reported.– Knowledge about system usage and other related design knowledge is
discussed among stakeholders.
UCI
Argo
Software Design Environments
UCI
Underlying Theories of Designers’ Cognitive Needs
• We have tried to identify what makes design challenging for people– Design feedback: Reflection-in-Action [Schoen]
– Design process support: Opportunistic Design [Guindon]
– Design perspectives: Comprehension and Problem Solving [Kintsch]
• We have specialized and applied these theories to software architecture
UCI
Design Feedback — Reflection-in-Action
• Observation: Designers work through the design– Designers rarely produce a design fully formed
– Incrementally frame a partial design; evaluate it; revise
• Suggested requirements on design environments– Support the designer in framing a partial solution
– Support the designer in evaluating it
– Integrate manipulation and analysis
• Support in Argo: Editors, Critics, To Do List
UCI
Construction with Critical Feedback
UCI
Design Process Support - Opportunistic Design
• Observation: Designers deviate from prespecified processes– Emerging issues change the design process
– Cognitive costs explain some task reorderings
• Suggested requirements on design environments– Provide process models as resources
– Use process models to keep design feedback timely
• Support in Argo: First-class decision model, To Do List
UCI
Process Navigation
UCI
Design Perspectives — Comprehension and Problem Solving
• Observation: Designers use multiple mental models– Work through successive levels of refinement
– Restructure their thinking to address different issues
• Suggested requirements on design environments– Present multiple perspectives to match mental models
– Help maintain mappings between mental models
• Support in Argo: Multiple coordinated design perspectives, customizable visualizations
UCI
Multiple Perspectives
UCI
Summary• Tools for designers should support their cognitive
needs.
• We have examined three theories of cognitive needs and how to support them.– Gives us a checklist of needed design environment features.
Critics
First ClassProcess Model
ManagedTo Do List
MultiplePerspectives
DirectManipulation
CustomGraphics
ProcessCritics
PresentationCritics
Re½ection in Action
Comprehension and Problem
Solving
Opportunistic Design
UCI
Argo Future• Apply Argo’s infrastructure to two new domains
– UML: a new object-oriented design notation
– CoRE: a formal state-based representation of behavioral requirements for embedded systems
• Evaluation of impact of cognitive support on design product and process– Feedback from Collins Avionics on CoRE tool
– Planning to use Argo/UML in ugrad class in Spring
• Evaluation of usefulness of Argo infrastructure– Metrics on effort needed to build Argo/CoRE and Argo/UML
– Feedback from other developers using Argo
UCI
BackgroundUML: The Unified Modeling Language
• UML unifies – Popular OO design notations: Booch, OMT, OOSE– Many views of the system: object model, state models, use cases,
deployment model, etc.– Notations used for analysis, design, and implementation
• UML is well defined– UML Meta-model defines what is in a UML model– Object Constraint Language specifies system constraints– Meta-model has about 100 classes and 180 constraints (that might make
good design critics)
• There is broad industrial interest in using UML
UCI
UML Example
(self.robot->size) / (self.worker->size) < 0.1
Company
Worker
Robot «Person»Employee
Trainee
Course
0..*
0..*
1..4
1..*1..*
UCI
CoRE• CoRE is a formal state-based notation for behavioral
requirements of embedded systems, based on SCR– Embedded systems monitor some variables and control others.
The values of controlled variables depends on the current state. State transitions occur in reaction to changes in monitored variables.
• Examples– Traffic lights at a simple 3-way intersection
• Monitors sensing pads, walk buttons, and internal timers• Controls color of lights and walk signs• About 9 states and 12 transitions
– Auto-pilot on a Boeing 777• Monitors buttons and switches in cockpit• Controls flight modes and some cockpit displays• Thousands of states and transitions
UCI
Intersection Overhead View
Crosswalk button
Crosswalk button
1 2
3
4
5
6
7
Cornell Drive
Cam
pus
Dri
ve
UCI
Behavioral SpecificationDiagram
Flow
Warn
Stop
Normal
Flow
Warn
Stop
Allow2
Flow
Warn
Stop
Walk
1
2 3
4
5
6
7
89
10
11
UCI
CoRE Specifications with Feedback
UCI
EDEM
Usage Data Collection Through
Expectation-Driven Event Monitoring
UCI
Expectations in Development• Developers have usage expectations that importantly affect
design decisions.
• Developers' usage expectations are based on their:– knowledge of requirements
– knowledge of the application domain
– knowledge of specific user tasks, practices, and work environments
– past experience developing systems
– past experience using applications themselves
• Developers' usage expectations impact the appropriateness and usability of their designs:– accurate expectations => good designs
– inaccurate expectations => poor designs
UCI
Characteristics of Expectations• Some usage expectations are represented explicitly.
– e.g. those specified as requirements or in "use cases"
• Most usage expectations are implicit.– e.g. those encoded in window layout, toolbar and menu design, key
assignments, and user interface libraries.
• Example usage expectations:– users complete forms from left to right and top to bottom.– frequently used features are easy to access, recognize, and use.
• Because most usage expectations are not represented explicitly, they often:– fail to be tested adequately– fail to be explicitly recognized by developers
UCI
Resolving Mismatches• Detecting and resolving mismatches between developers'
expectations and actual usage can help improve:– design, automation, on-line help, training, and use.
• Once mismatches are detected, they may be resolved in one of two ways:– Developers may modify their expectations to better match actual use,
thus refining system requirements and eventually improving the design.
• e.g., features expected to be used rarely but used often in practice can be made easier to access.
– Users may learn about developers' expectations, thus learning how to use the existing system more effectively.
• e.g., learning they are not expected to type full URL's in Netscape's Communicator can lead users to omit characters such as "http://".
UCI
Expectation-Driven Event Monitoring (EDEM)
UCI
Agent Representation• Agents are instances of a simple Java class w/ the
following members:– Trigger: patterns of user interface (or agent) events
– Guard: boolean expression involving user interface (or agent) state
– Actions: pre-supplied actions or arbitrary code
• Triggers are continually checked as users interact w/ the application.
• Guards are checked if an agent trigger has been activated.
• Actions are performed if the guard evaluates to true.
UCI
Triggers• Triggers specified in terms of the following patterns:
(1) "A or B or . . . "
(2) "A and B and . . . "
(3) "A then B then . . . "
(4) "(A and B) with no interleaving C"
(5) "(A then B) with no interleaving C"
• Where variables A,B,C are filled in by specifying:(1) a Component from the UI plus an Awt or EDEM event on that
component (e.g. "TextField1:LOST_EDIT" which occurs when TextField1 is edited and then input focus shifts and editing begins in another component)
(2) another Agent (e.g. "AddressDone" which occurs when another agent detects that the address section has been completed)
UCI
Guards• Guards specified in terms of the following patterns:
(1) "A or B or . . . "
(2) "A and B and . . . "
• Where variables A,B are filled in by specifying:(1) a Component from the UI and some expression involving its
properties (e.g. "TextField1:value='Married'" or "Button1:count>100")
(2) another Agent and some expression involving its properties (e.g. "AddressStarted:enabled=true" or "AddressDone:count>1")
UCI
Actions• Actions may include arbitrary code, but usually
involve pre-supplied actions such as:– generating higher level events for further hierarchical event
processing
– interacting with users to provide suggestions and/or collect feedback, and
– reporting data back to developers
UCI
Integrating w/ EDEM• void initialize(boolean obtrusive)
– load agents, post EDEM message window if obtrusive
• void addMonitors(Object obj)– recursively add monitors to this component and all subcomponents
• void setName(Object obj, String name)– name components to be monitored that don’t have a unique label
• void processEvent(Event evt)– pass events to EDEM for processing
• void finalize()– remove monitors and send log & summary
UCI
Summary• Usage expectations:
– focus data collection
– raise awareness of implications of design decisions
• Event monitoring architecture:– distributed event filtering and abstraction
– reduction of data to be reported and analyzed
– independent evolution of instrumentation/application
• Extensible event model:– data collected/analyzed at multiple levels of abstraction
– events of interest unavailable in a given event model are added by defining them in terms of available events
UCI
Global Transportation Network (GTN) Application
UCI
Action Sequence Dependency
UCI
Knowledge Depot
Information Review and Project Awareness
UCI
Communication and Group Memory
• The Knowledge Depot, an enhanced version of GIMMe (Zimmermann, Lindstaedt, Girgensohn, 1997), is a group memory that captures email conversations and documents submitted over the web.
• Users may browse through the information to learn or to rediscover why different decisions were made (or are being made), allowing the user to gain some of the context in which those decisions were made..
UCI
UCI Knowledge Depot
UCI
Characterizing Individual and Group Awareness
UCI
Project Awareness• Project Awareness is an approach to providing
communications links between individuals and all of the different aspects of the project that affect those individuals.
• The strength of those communication links should depend on the relevance to each individual.
UCI
Types of Information• Changes to component’s design or interface
• Changes to completion dates
• Changes to higher level design
• Changes to management
• Changes to customer requirements
• The knowledge that the above changes are being discussed
• The concerns that lead to the discussions
UCI
Types of Links
UCI
From Group Memory to Project Awareness
• Group Memories can enhance Project Awareness by allowing users to request periodic summaries of new information in topics of interest. This gives a group memory three key attributes for creating an awareness system:– Information Capture: Information captured or added to the group
memory
– Information Distribution: Emailed to interested users
– Information Display: A list of documents added to the group memory, accompanied by brief descriptions
UCI
Future Work• How well does Knowledge Depot enhance project
awareness relative to existing approaches?
• How accurate are our predictions on the benefits of Project Awareness?
• How can Knowledge Depot more effectively support project awareness?
• How effective are categorization techniques on personal/frequently used mailboxes as well as the less frequently used archives.
UCI
Java Depot: Integration between Project and Individual Depots
UCI
Conclusions
UCI
Research Goals• To develop a model of software development as a
process of on-going communication;
• To support this model through active mechanisms in software tools; and
• To improve the accessibility and acceptance of usability methods by software practitioners.
UCI
Research Approach• To focus on a human-centered software development
process, in which feedback comes from all stakeholders.
• Stakeholders reflect on an artifact that evolves from sketchy designs to partial, and eventually, complete prototypes.
• Feedback is not limited to human stakeholders: tools can provide data about actual usage and knowledge evolved from previous development experiences.