designing the expert system

Designing Expert Systems

• Selecting the appropriate Problem– Why build the expert system?

• Need clear identification of the problem, expert, and users.

– What is the payoff?• What is the return on investment in money,

increased efficiency, or any other advantages.

– What tools are available to build the system?

Designing Expert Systems

– How much will it cost?• The cost is dependent upon the people,

resources, and time devoted to its construction.• Also the costs of hardware and software as well

as training need to be considered.

Phase IProject Initialization

Problem definitionNeeds assessmentEvaluation of alternative solutionsVerification of an ES approachFeasibility studyCost-benefit analysisManagerial issues considerationOrganization of development team

Phase IISystem analysis and design

Conceptual design and planDevelopment strategySources of knowledgeComputing resources

Phase IIIRapid Prototyping

Build small prototypeTesting, improving, expandingDemonstrating and analyzing feasibilityCompleting design

Phase IVSystem Development

Completing the Knowledge baseTesting, evaluating, and improving the knowledge basePlanning for integration

Phase VImplementation

Acceptance by usersInstallation, demonstration, deploymentOrientation, trainingSecurityDocumentationIntegration, field-testing

Phase VIPost-implementation

OperationMaintenance and upgradesPeriodic evaluation

Phase I • Phase I of building an expert system is the project

initialization. – This phase requires:

• Problem definition• Needs Assessment• Evaluation of alternative solutions• Verification of the ES approach• Cost-benefit analysis• Consideration of managerial issues

– This phase ends when the project has been approved in principle.

Problem Definition

• A clear problem definition will help to simplify the remaining tasks and generate a productive program.– The problem definition should include as

much supporting material and information as possible.

Need Assessment

• The Need assessment is a formal study to understand the problem or need.

Alternative Solutions

• One should always consider if there is an alternative solution to a problem.– Lack of knowledge does not necessarily

mean that there is a need for an expert system.

– Can the information be made available from the expert in some other manner?

– Can education and training be used to communicate the desired information?

Alternative Solutions

– Can documentation of the knowledge be used?

– Is this problem better suited for conventional programming techniques?

– Can the information be purchased or obtained from the WWW, or some other external source?

Approach Verification

• Even if there does not appear to be an alternative solution, an expert system may not be the best approach.

Approach Verification

• Waterman [1985] suggested a framework for assessing problem fit.– It is composed of requirements,

justification, and appropriateness.

Approach Verification

• Waterman’s Requirements – The following 12 requirements must be met:

• The task can not require common sense.

• The task requires only cognitive, not physical, skills.

• There must be one genuine expert who is willing to cooperate.

• Experts involved can articulate their methods of problem solving.

Approach Verification• Experts involved can agree on the knowledge and

the solution approach to the problem.• The task is not too difficult.• The task is well understood and is defined clearly.• The task definition is fairly stable.• Conventional (algorithmic) computer solution

techniques are not satisfactory.• Incorrect or non-optimal results generated by the ES

can be tolerated.• The task’s vocabulary has not more than a couple

hundred concepts.

Approach Verification

• The second phase of Waterman’s framework is Justification.– At least one of the following must be

present to justify the ES development:• The solution to the problem has a high payoff.• The ES can preserve scarce human expertise.• Expertise is needed in many locations.• Expertise is needed in hostile and hazardous

environments.

Approach Verification

• The expertise improves performance or quality.• The system can be used for training.• The ES solution can be derived faster than that

which a human can provide.• The ES is more consistent or accurate than a

human.

Approach Verification

• The final element to Waterman’s framework is Appropriateness.– Each of the following factors should be

considered:• Have a symbolic structure and heuristics

should be available for the solution.• Not be too easy nor too difficult for the human

expert.• Have a manageable scope size.

Feasibility Study

• Many times a feasibility study will be used to determine if an expert system should be developed.– Such a study should include the economic

feasibility, technical feasibility, and operational feasibility and impacts.

Economic Feasibility

• Economic feasibility considers– The cost of system development in an

itemized fashion.– The cost of maintenance (usually much

more).– The anticipated payoff.– A cash flow analysis.– A risk analysis.

Technical Feasibility

• Technical feasibility covers:– Interface requirements– Networking issues– Availability of knowledge and data– Security of confidential knowledge– Knowledge representation scheme– Hardware/software availability and

compatibility

Operational feasibility

• The operational feasibility considers:– Availability of human and other resources– Priority as compared to other projects– Need assessment– Organizational and implementation issues– Management and user support– Availability of experts and knowledge engineers.– Legal and other constraints– Corporate culture– User environment

Cost-benefit Analysis

• The purpose of cost-benefit analysis is to determine the viability of a project by looking at the costs versus the anticipated benefits.– This is hard to do for expert systems.– It is best to start this process with a well

defined scope of the project and amount of time required for each participant.

Cost-benefit Analysis• The development costs that should be

considered are:– Development tool cost– Any additional hardware cost– Cost to gather expertise from expert(s)– Cost of knowledge engineer and/or programmer.– Cost of time to test, debug, and maintain

program.– Possibly, cost of outside consultants.

Cost-benefit Analysis• It is usually more difficult to assess the benefits.

– Some benefits are intangible.– Some benefits can not be related to a single cause.– Some actions may occur over long periods of time

and are therefore, difficult to predict.– The valuation of benefits should include an

assessment of both quantity and quality.– Multiplicity of consequences can pose a major

problem for quantification.

Managerial Issues• Managerial support is required to complete any

project. Issues to consider are:– All parties and management are convinced of the project’s

value.– Someone in top management is strongly sponsoring the

project.– End-users are involved early, and receive proper support

and training.– Availability of financing, and other resources.– Legal and other potential constraints.

Phase II

• The second phase is the system analysis and design.– This includes:

• Conceptualization and design• Development strategy• Sources of knowledge• Computing resources

– This phase ends when there is an approved and complete project plan.

Conceptual Design

• The conceptual design should provide a general idea of what the system will look like and how it will solve the problem.

Development Strategy

• There are several strategies that can be used to develop the system.– Do it yourself.

• This implies the organization has the skills and resources.

– Hire an outside developer• Either a consulting firm or a consortium of

companies.

Development Strategy

– Participate in a joint venture• The vendor develops the system while testing

its own products and services.• Sponsor a university research project.

– Merge, acquire, or become a major stockholder in an AI company

Sources of Knowledge

• The sources of knowledge are many, but the primary source will be a human expert. – A human expert should posses more

complex knowledge than can be found in documented sources.

– The more human expertise is required, the longer and more complicated the knowledge acquisition will be.

Sources of Knowledge

• Questions that may need to be answered when selecting an expert are:– Should the expert be compensated for their contributions?– How will it be determined if the experts are truthfully

describing the solutions?– How can experts be assured that they will not lose their

jobs, or that the job will not be de-emphasized?– Are the experts concerned about other people in the

organization who may suffer consequences of the new ES?

Sources of Knowledge

• Attributes to look for in an expert:– Highly developed perceptual attention ability.– Awareness of the difference between relevant and

irrelevant information.– Ability to simplify complexities.– Strong communication skills.– Knowledge of when to make exceptions.– A strong sense of responsibility for their choices.– Selectivity about which problems to solve.

Sources of Knowledge

– Outward confidence in their decisions.– Ability to adapt to changing task conditions.– Highly developed content knowledge about their

area.– Greater automaticity of cognitive processes (tasks

are a habit to the expert.)– Ability to tolerate stress.– Capability to be more creative.– Ability to articulate their decisions processes.

Sources of Knowledge

– Thorough familiarity with the domain, including task expertise built up over a long period of task performance.

– Knowledge and reputation such that the system will have credibility and authority.

– Commitment of a substantial amount of time to the development of the system.

– Capability of communicating his or her knowledge, judgment, and experience.

Sources of Knowledge

– Cooperative, easy to work with, and eager to work on the project.

– Interest in computer systems, even if they are not a computer specialist.

Computing Resources

• There are many software options for developing Expert Systems.– The higher-level the software the less

programming that is required. But, it is less flexible.

Programming Languages

• Expert systems can be developed using various languages.– The languages may be pure AI languages

like PROLOG and LISP or may be more object oriented languages.

– Object-oriented languages have been found useful given the complexity of problems.

• Some problems can be simplified by using inheritance.

Programming Languages

• The best programming languages tend to be those that combine procedure-oriented, rule-oriented, object-oriented, and data-oriented programming.– These tend to be more flexible.

Shells

• A shell is basically an abstract application environment.– The advantage of shells is that knowledge

can be easily represented.– Shells tend to be built once and then

reused for many applications.

Shells

KnowledgeBase

Consultationmanager

KB Editorand Debugger

ExplanationProgram

KB MangmtFacilities

InferenceEngine

Shell

Shells

• While shells are easy to use there are also disadvantages:– They can be inflexible.– It may be difficult to fit into nonstandard

problems and tasks.– Since they are end-user tools, they are

subject to end-user computing problems.

Environments

• Development environments usually support different ways of representing knowledge and handling inferences.

• Hybrid System Environments create a programming environment that enhances building complex specific systems or complex tools.

Environments

• Environments tend to be more specialized than languages so they can sometimes increase development productivity.

• Environments also tend to be more flexible than shells.

Phase III

• The third phase of expert system development is rapid prototyping and a demonstration prototype.– A prototype is a small-scale system that

represents some of the knowledge captured.– Prototypes are important to establish the

structure of the knowledge base before building more rules.

Phase III

• The advantages of prototypes are:– They allow the developers to get a sense of the

feasibility of a full application.– They provide the ability to study the effectiveness of

the knowledge acquisition and representation.– They can be used to identify gaps or important

problems in the proposed system.– They provide a tangible product early in the

development.

Phase III

– They provide an opportunity to demonstrate the system capabilities.

– They allow for midcourse corrections based upon management, consulting expert, and user feedback.

– They provide a system that can be tested.– They may provide a system that can be field

tested.– They accelerate knowledge acquisition.

Phase III

– They facility expert feedback and provide exceptions to the rules.

– They make it easier to win over skeptics.– They help sustain the expert’s interest.– They can help build user support.– They can provide an idea of the value of the

system.– They provide information about the initial

problem domain definition.

Phase IV

• The fourth development phase is the System Development.– During this phase the knowledge base is

built and continuous testing, reviews, and improvements are carried out.

Building the KB

• Building the KB involves acquiring and representing the knowledge in an appropriate computer format.– The first step is to organize the domain

knowledge.• List all possible solutions, outcomes, answers,

choices, or recommendations.

– Then define the input facts by identifying and listing all the data that is required by the system.

Building the KB

– It may be useful to develop an outline of the KB even though the potential solutions and outcomes are known.

– A decision tree may be easily developed as some types of knowledge will organize itself into the tree structure.

– Other types of knowledge will nicely organize into a matrix.

– Finally, once the rules are written they can be entered into the KB.

Testing, Validating, etc.

• Testing and evaluation should occur throughout the entire process.– Begin with the prototype and continue

through the entire system development.– Evaluation questions.

• What characteristics should be evaluated?• How should performance be evaluated?• How should the test problems be selected?• How should the system’s mistakes be evaluated?

Testing, Validating, etc.

• Validation attempts to determine if the system was built correctly or whether the system does what is supposed to do with an acceptable level of accuracy.

• Verification determines that the ES has been built to correctly implement the specifications.

Phase V

• The fifth phase is implementation. – Some implementation considerations are:

• The user’s acceptance of the system depends not only on ease of use and quality but also on behavioral and psychological considerations.

• The system is ready for implementation when it reaches a minimum level of stability and quality.

• Demonstrate as much as possible to the users.• Ensure that the deployment mode is chosen and verified.• User acceptance requires a high level of training.

Phase V

• If the knowledge in the KB is proprietary, steps need to be taken to protect the knowledge.

• Do not forget the documentation.• At the end of Phase V, if the system is a stand alone

system, it is ready for field testing.

Implementation Pitfalls

• Waterman lists many pitfalls during implementation and suggestions for avoiding such pitfalls:– The expert’s knowledge can become

entwined with the rest of the program. Making it impossible to separate it from general knowledge about search and problem solving.

• Use a rule-based organization.

Implementation Pitfalls– Even after a long knowledge acquisition process,

the KB can still be incomplete,or not able to solve simple problems because fundamental concepts are missing or are ill-represented.

• Such issues can be identified early by using incremental development and testing.

– The development environment does not have built-in explanations and it is non-trivial to add such capability.

• Design the system from the beginning with transparency.

Implementation Pitfalls

– The system is slow and unwieldy because of a very large number of specific rules.

• Where possible combine smaller rules into more general rules.

Implementation Suggestions

• Work with a core set of representative problems and keep a library of cases that are presented to the system.

• Separate domain specific knowledge from general problem solving knowledge. Keep the inference engine simple.

• If a rule looks big, it probably should be broken down.

Implementation Suggestions

• If several rules look very similar, look for an underlying domain concept and combine them.

• Group rules into rule sets.

• Adopt a set of programming style conventions to give the program a uniform look.– Implement similar functions the same way

throughout the system

Implementation Suggestions

• Do not be afraid to throw out prototypes and previous implementations.

• Make sure the developers are trained in developing expert systems.

Phase VI

• The sixth stage is the post-implementation. This phase includes:– Operation – Maintenance– Expansion– Evaluation

Operation

• The operation of the system is concerned with:– Delivering the system– Providing user training

Maintenance

• The maintenance of an expert system is never done. It is the nature of the beast.– The system will be revised on a regular

basis.• Knowledge and understanding change.

– Maintenance also includes the traditional tasks of fixing software and hardware bugs and system upgrades.

Expansion

• Expert systems are always changing and expanding as new knowledge is added or old knowledge is updated.– It is best to schedule periodic reviews of

the knowledge.

Evaluation

• Periodic evaluation of the expert system is necessary, but may be needed less frequently if the users provide continuous feedback.

Evaluation

– Questions that are asked:• What is the actual cost of maintaining the

system compared to the benefits?• Is the maintenance sufficient to keep the

knowledge up to date so that the system maintains a high level of accuracy?

• Is the system accessible to all users?• Is acceptance by users increasing?

Why Expert Systems Fail?

• The following are reasons that expert systems have failed either during development or after deployment:– The ES no longer solves a meaningful

problem or can not provide a meaningful solution.

– The maintenance costs are too expensive.– The system is not upgraded to continue to

work with the computing environment.

Why Expert Systems Fail?– The company or industry change focus.– The task domain size was too large and could not

be reasonably represented.– The users do not perceive the problem as critical.– The solutions are incorrect or inaccurate.– User resistance.– No one is willing to provide maintenance for the

system.– Loss of key development personnel.

References

• Expert Systems Principles and Programming, third edition by Giarratano and Riley; 1998.

• Decision Support Systems and Intelligent Systems, sixth edition by Efraim Turban and Jay E. Aronson; 2000.

• D. A. Waterman (1985). A guide to Expert Systems. Reading, MA: Addison-Wesley.