Knowledge Engineeringand Acquisition

Chapter 6 SupplementChapter 6 Supplement

Knowledge Acquisition

Knowledge acquisition is the extraction of knowledge from sources of expertise and its transfer to the knowledge base and sometimes to the inference engine

What are some of the Difficulties in Knowledge Acquisition

Expressing the knowledge: Human knowledge exists in a compiled format. A human doesn’t

remember all the intermediate steps used to in transferring and processing knowledge – representation mismatch

Number of participantsStructuring the knowledge: We must elicit not only the knowledge but also its structure; rules

“Knowers” lack time and unwilling to helpTesting and refining knowledge is hardCollect knowledge from one source but relevant knowledge is dispersedImportant knowledge may be mixed up with irrelevant informationIncomplete knowledge (use one source only)“Knowers” may change their behavior when observedProblematic interpersonal factors

Knowledge Engineering Process Activities

Knowledge Acquisition Acquisition of knowledge from human experts, books,

documents, or computer files

Knowledge Validation Knowledge is validated and verified (using test cases) until the

quality is acceptable

Knowledge Representation Organized knowledge; creation of a knowledge map and the

encoding of knowledge into a knowledge base

Inferencing Design of software to enable the software to make inferences

based on the knowledge and the specifics of the a problem

Explanation and Justification The design and programming of an explanation capability. Why

is this piece of information needed? How was a certain conclusion derived.

Knowledge Engineering Process

Knowledgevalidation(test cases)

KnowledgeRepresentation

KnowledgeAcquisition

Encoding

Inferencing

Sources of knowledge(experts, others)

Explanationjustification

Knowledgebase

Knowledge Sources

Documented (books, manuals, etc.)

Undocumented (in people's minds)From people, from machines

Knowledge Acquisition from Databases

Knowledge Acquisition Via the Internet

Knowledge Acquisition Methods: An Overview

Manual :the knowledge engineer interacts directly with the experts Interviews, tracking the reasoning process (protocol analysis), observing,

brainstorming, conceptual graphs and models

Semiautomatic (Expert-driven): the expert encodes his or her expertise directly into the computer system or the developer uses technology to facilitate the knowledge acquistion

Expert’s self reports, computer aided approaches (visual modeling); graphical development environment where the initial knowledge domain can be modeled and manipulated (decision trees based on business process logic) ex. REFINER+ patient manager

Automatic (Computer Aided - Induction driven) Minimize or eliminate the role of the KE and/or the expert inference engines extract the knowledge from a set of examples

Manual Methods of Knowledge Acquisition

Elicitation

Knowledgebase

Documentedknowledge

Experts

CodingKnowledge

engineer

Expert-Driven Knowledge Acquisition

Knowledgebase

Knowledgeengineer

Expert CodingComputer-aided

(interactive)interviewing

Induction-Driven Knowledge Acquisition

Knowledgebase

Case historiesand examples

Inductionsystem

Manual Acquisition Techniques

Interviewing: two common types are unstructured (conversational) and structured (interrogation/using a script)Verbal Protocol Analysis:

Most of the information necessary to model knowledge is found in the cognitive process the knower uses to solve a problem/do a task

Document the step-by-step information processing and decision making behavior by the knower

Concurrent: Think aloud or verbalize thoughts while doing task

Repertory Grid Method: Maybe manual or computerized

Expert Driven/Computer Aided

Reparatory Grid Analysis May also be employed by the KE Developed by Kelly (1955) who conceived humans as

”personal scientist” each with their own model of the world. the expert compares successive groups of three objects

and tells why two differ from the third Also used to infer similarities in construct beliefs held by

multiple experts Knowledge and perceptions about the world are classified

and categorized by each individual as a personal, perceptual model.

Machine learning/Automated Rule Induction

Training set: example of a problem for which the outcome is known

After given enough examples, the rule induction system can create rules that fit the example cases.

The rules can be used to assess new cases for which the outcome is not known.

For Example: Loan Officer’s tasks: Requests for loans include information about the applicants such as income, assets, age and number of dependents

TABLE 13.6 Case for Induction - A Knowledge Map

(Induction Table)

Attributes

AnnualApplicant Income ($) Assets ($) Age Dependents Decision

Mr. White 50,000 100,000 30 3 Yes

Ms. Green 70,000 None 35 1 Yes

Mr. Smith 40,000 None 33 2 No

Ms. Rich 30,000 250,000 42 0 Yes

From this case, it is easy to derive the following three rules:

If Income is $70,000 or more approve the loan

If income is $30,000 or more, age is at least 40, assets are above $249,000 and there are no dependents approve the loan

If income is between $30,000 and $50,000 and assets are at least $100,000, approve the loan

Multisource Knowledge Acquisition

It is likely that multiple sources will be needed to fully acquire the knowledge for a problem and conflicting views and opinions often arise.Brainstorming/Electronic Brainstorming Goal is to come up with creative solutions. Idea

generation and evaluationConsensus Decision NGTDelphi MethodConcept MappingBlackboarding

Validation and Verification of the Knowledge Base

Quality Control EvaluationValidation Verification

Evaluation Assess an expert system's overall value Analyze whether the system would be usable, efficient and

cost-effective

Validation Deals with the performance of the system (compared to the

expert's) Was the “right” system built (acceptable level of

accuracy?)

Verification Was the system built "right"? Was the system correctly implemented to specifications?

To Validate an ES

Test1. The extent to which the system and

the expert decisions agree2. The inputs and processes used by an

expert compared to the machine3. The difference between expert and

novice decisions

Some validation measures

Accuracy

Adaptability

Adequacy

Breadth

Depth

Face Validity

Generality

Precision

Realism

Reliability

Robustness

Usefulness