Knowledge Discovery

Javier Béjar

URL - Spring 2020

CS - MIA

Knowledge Discovery (KDD)

Knowledge Discovery in Databases (KDD)

• Practical application of the methodologies from machinelearning/statistics to large amounts of data

• Problem: the impossible task of manually analyzing all thedata we are systematically collecting

• Useful for automating/helping the process of analysis/discovery

• Final goal: to extract (semi)automatically actionable/usefulknowledge

“We are drowning in information and starving for knowledge”

URL - Spring 2020 - MAI 1/18

Knowledge Discovery in Databases

• The high point of KDD starts around late 1990s

• Many companies show their interest in obtaining the (possibly)valuable information stored in their databases (purchasetransactions, e-commerce, web data, ...)

• The area has moved/integrated/transmuted several times toinclude several sometimes interchangeable terms: BusinessIntelligence, Business Analytics, Predictive Analytics, DataScience, Big Data ...

• The Venn Diagram Wars

URL - Spring 2020 - MAI 2/18

Venn Wars: The funny one

Venn Wars: The job description one

Venn Wars: The scientific one

Venn Wars: The multidisciplinary one

Venn Wars: The reality check one

KDD definitions

“It is the search of valuable information in great volumes ofdata”

“It is the explorations and analysis, by automatic or semiau-tomatic tools, of great volumes of data in order to discoverpatterns and rules”

“It is the nontrivial process of identifying valid, novel, po-tentially useful, and ultimately understandable patternsin data”

URL - Spring 2020 - MAI 8/18

Elements of KDD

Pattern: Any representation formalism capable to describe thecommon characteristics of data

Valid: A pattern is valid if it is able to predict the behaviourof new information with a degree of certainty

Novelty: It is novel any knowledge that it is not know respectthe domain knowledge and any previous discoveredknowledge

URL - Spring 2020 - MAI 9/18

Elements of KDD

Useful: New knowledge is useful if it allows to perform actionsthat yield some benefit given a established criteria

Understandable: The knowledge discovered must be analyzed byan expert in the domain, in consequence theinterpretability of the result is important

URL - Spring 2020 - MAI 10/18

The KDD process

KDD as a process

• The actual discovery of patterns is only one part of a morecomplex process

• Raw data in not always ready for processing (80/20 projecteffort)

• Some general methodologies have been defined for the wholeprocess (CRISP-DM or SEMMA)

• These methodologies address KDD as an engineering process,despite being business oriented are general enough to beapplied on any data discovery domain

URL - Spring 2020 - MAI 11/18

The KDD process

• Steps of the Knowledge Discoveryin DB process1. Domain study2. Creating the dataset3. Data preprocessing4. Dimensionality reduction5. Selection of the discovery goal6. Selection of the adequate

methodologies7. Data Mining8. Result assessment and

interpretation9. Using the knowledge

URL - Spring 2020 - MAI 12/18

Goals of the KDD process

There are different goals that can be pursued as the result of thediscovery process, among them:

Classification We need models that allow to discriminate instancesthat belong to a previously known set of groups (themodel could or could not be interpretable)

Clustering/Partitioning/Segmentation We need to discovermodels that clusters the data into groups with commoncharacteristics (a characterizations of the groups isdesirable)

URL - Spring 2020 - MAI 13/18

Goals of the KDD process

Regression We look for models that predicts the behaviour ofcontinuous variables as a function of others

Summarization We look for a compact description thatsummarizes the characteristics of the data

Causal dependence We need models that reveal the causaldependence among the variables and assess thestrength of this dependence

URL - Spring 2020 - MAI 14/18

Goals of the KDD process

Structure dependence We need models that reveal patternsamong the relations that describe the structure of thedata

Change We need models that discover patterns in data that hastemporal or spatial dependence

URL - Spring 2020 - MAI 15/18

Methodologies for KDD

• Decision trees, decision rules

• Usually are interpretable models• Can be used for: Classification, regression, and summarization• trees: C4.5, CART, QUEST, rules: RIPPER, CN2, ..

• Classifiers, Regression

• Low interpretability but good accuracy• Can be used for: Classification and regression• Statistical regression, function approximation, Neural networks,

Support Vector Machines, k-NN, Local Weighted Regression, ...

URL - Spring 2020 - MAI 16/18

Methodologies for KDD

• Clustering• Partition datasets or discover groups• Can be used for: Clustering, summarization• Statistical Clustering, Unsupervised Machine learning,

Unsupervised Neural networks (Self-Organizing Maps)

• Dependency models• Obtain models of the dependence relations (structural, causal

temporal) among attributes/instances• Can be used for: causal dependence discovery, temporal

change, substructure discovery• Bayesian networks, association rules, Markov models, graphs

algorithms, ...

URL - Spring 2020 - MAI 17/18

Challenges

Open problems

• Scalability (More data, more attributes)

• Overfitting (Patterns with low interest)

• Temporal data/relational data/structured data

• Methods for data cleaning (Missing data and noise)

• Pattern comprehensibility

• Use of domain knowledge

• Integration with other techniques (OLAP, DataWarehousing,Intelligent Decision Support Systems)

• Privacy

URL - Spring 2020 - MAI 18/18