Kasabov : CH 1-2P. 65: A General

Approach to Knowledge

Engineering

Methods

• Statistical: Can be used when statistically representable data are available and the underlying type of goal function is known.

• Symbolic : AI rule-based systems can be used when the problem knowledge is in the form of well-defined, rigid rules; no adaptationis possible, or at least it is difficult to implement

Methods/Cont. (p.65)

• Fuzzy Systems: are applicable when the problem knowledge includes heuristic rules, but they are vague, ill-defined, approximate, possibly contradictory.

• Neural Networks are applicable when problem knowledge includes data without having any knowledge as to what the type of the goal function might be; they can be used to learn heuristic rules after trainign with data; can also be used to implement existing fuzzy or symbolic rules; provide a flexible, ,approximate reasoning mechanism.

Methods / Cont.

• Genetic Algorithms:Require neither data sets nor heuristic rules, but a simple selection criterion to start with; they are very efficient when only a little is known to start with (p. 67)

Figure 1.37 p. 66

• A neural network is used to learn fuzzy rules, which are implemented in a fuzzy inference system.

• Symbolic AI machine-learning method is used and the rules learned are implemented in a symbolic AI reasoning machine.

• Symbolic AI rules are combined with neural networks in a hybrid system.

• Genetic algorithm is used to define values for some learning parameters.

P.68 : Part A: Case Example Solution

• Too complicated for our purposes, but main point is that

DIFFERENT TRANSFORMATIONS ARE APPPLICABLE TO SPEECH SIGNALS (p. 69).

P. 68 Practical Tasks

Conclusion (P. 72)

CH: Knowledge Engineering and

Symbolic AI

What is Knowledge?

As distinct from data and

information???

Knowledge is “condensed information” Rules of Thumb (Heuristics).

Major Issues in Knowledge Engineering

1. Representationa. What kind of knowledge?b. Alternative methods?

2. Inference

3. Learning – Through Examples– By being told– By doing

Major Issues in Knowledge

Engineering /cont4. Generalization

5. Interaction

6. Explanation

7. Validation

8, Adaptation

What Kind is Best?

• Symbolic, Fuzzy, and Neural Systems

• See TABLE p. 79;

Separating Knowledge from Data (p.79)

Gives 1. Stability (rules independent)

2. Separates Control

Knowledge can be expanded independently from the inference procedure.

Examples of Separation of Control from

Knowledge:

1. PROLOG: Declarative Language

Knowledge distinct from executive.

2. CLIPS: Production Language For Building Expert Systems

Data Analysis, Data Representation, and Data

TransformationVarieties of DATA

– Quantitative vs. Qualitative

Numerical vs. Symbolic

– Static vs. Dynamic

does not change changes

Varieties of Data/ cont.

Natural vs. Synthetic

Clean vs. Noisy

Data Representation

Requirements:

– Adequateness

– Unambiguity

– Simplicity

– E.g IRIS DATA

(example23 SL = 5.7 SW =4.4 PL = 1.5 PW = 0.4;)

– Shorter Form: ex23 = (5.7 4.4 1.5 0.4)

Major Issue

• Small Dimensional vs

Large Dimentional Data

Problem of Choosing appropriate dimensionality for a problem.

(p. 82)

Visualizing Data: Bar Graphs; Scattered Points

Graphsp. 83.

Data Transformations

• Data Rate Reduction-extract meaningful features,Fourier Transform on speech data,

mel-scale cepstrum Coeff.

• Noise Reduction• Sampling• Discretization

- The process of representing continuous-value data with the use of subintervgals where the real values lie. E.g.

- (5.3 4.7 1.2 3.0) becomes (2 3 1 3) in Fig. 2.3 (p. 84)

Data Transformations/cont.

- Normalization moving the scale of raw data into a predefined scale.

- Linear- Logarithmic- Exponential, etc.- Linear- Gaussian Function (later)- Fast Fourier Transform (FFT)

a special nonlinear transformation applied mainly to speech data to transform the signal taken for a small portion of time from the time-scale domain into the frequency scale domain.

Wavelet Transformation

- Wavelet Transformation is another nonlinear transformation. It can represent slight changes of the signal within the chosen window from the time scale.

- Here, within the window, several transformations are taken from the raw signal by applying Wavelet Basis Functions of the form:

- Wa,b (x) = f(ax –b) where:- F is a nonlinear function, a is a

scaling parameter, and b is a shifting parameter (between 0 and u)

Data Analysis (p.87)

- What are the statistical parameters?

- What is the nature of the process?

- How are the available data distributed in the problem space – clustered into groups, sparse, covering only patches of the problem space and therefore not enough to rely on them fully when solving the problem, uniformly distributed?

Data Analysis/cont/. (p.87)

Are there missing data?How much?

What features can be extracted from the data?

1.Statistical analysis methods Discover the repetitiveness in data

based on probability estimation. Simple parameters, like mean, standard deviation, distribution function, as well as more complex analysis like factor analysis, etc.

Clustering Methods (p. 88)

Find groups in which data are grouped based on measuring the distance between the data items.

(Fig. 2.6

Let us have a set of X of p data items represented in an n-dimensional space. A clustering procedure results in defining k disjoint subsets (clusters), such that every data item (n-dimensional vector) belongs to only one cluster….

Clustering Methods / Cont.

A cluster membership function Mi

Is defined for each of the clusters C1, C2, …., CK:

Mi : X => [0,1},

Mi(X) = 1, if x E Ci,

Information Structures

– Sets, Stacks, Queues, and Lists

– Dynamic vs..Static Queue (p. 92)

– Directed Graphs

– Nodes (vertices), Arcs,

Trees and Graphs

• A graph is a tree with a cycle.

• Hence more than one way to reach a node.

• Spanning Tree

• Euler Path

• Hamiltonian Path

• See p. 95.

Frames, Semantic Nets and Schemata ( p. 96-

97)• Schemata are more general

structures than a semantic network. They are based on representing knowledge as a stable state of a system consisting of many small elements which interact with one another when the system is moving from one state to another.

Variety of Problem Knowledge (p.97-98)

• Global vs. Local• Shallow vs. deep Knowledge• Expicit vs. Implicit• Complete vs. Incomplete• Exact vs. Inexact Knowledge• Hierarchical vs. Flat Knowledge.• Meta-Knowledge

• Frame Problem: What should be changed in a knowledge representation when the situation has changed?

Methods for Symbol Manipulation and

Inference:Inference as Matching

• Generate and Test

• Constraint Satisfaction

• Forward and Backward Chaining (p. 102 – 104)

• Forward (Data Driven)

• Backward (Goal Drive)

See P. 104

Methods of Reasoning

• Monotonic vs. Non-Monotonic

• Exact vs. Approximate

• Iteration vs. Recursion

• Propositional Logic (p. 110-113)

• Predicate Logic: PROLOG (p.1114 – 116)