a study of mebn learning based on relational model
TRANSCRIPT
A study of MEBN Learning based on Relational Model
Cheol Young Park, Kathryn Blackmond Laskey, Paulo Costa, Shou Matsumoto
1
History of Statistical Relational Learning (SRL)
Statistical Relational Learning (SRL) is a new branch of machine learning for representing and learning a joint distribution over relational data
Traditionally, there are two groups of machine learning in data perspective First group focuses on uncertainty
– e.g. Incomplete data, Inaccurate data Second group focuses on complex data structure
– e.g. First Order Logic, Relation Database
Both groups have their own learning methods and representing models
2
History of Statistical Relational Learning (SRL)
Both groups realize that the real world data has the both properties Uncertainty Complex data structure
To deal with the both properties of data, Statistical Relational Learning (SRL) is invented Statistical: uncertainty Relational: complex data structure Learning: generating process from data to model
3
List of Statistical Relational Learning (SRL)
Bayesian logic programs BLOG models Logic programs with annotated disjunctions Markov logic networks Multi-entity Bayesian networks (our model) Probabilistic relational models Recursive random fields Relational Bayesian networks Relational dependency networks Relational Markov networks Join Bayes Net
4
Concept of induction and deduction of SRL
Dataset from University A
Dataset from University B
Dataset from University C
Induction Deduction
BN from university A BN from university B BN from university C
University MEBN
5
Survey of SRLs
[A Survey on Statistical Relational Learning. 2010] 6
The goal of initial research of MEBN Learning
Providing a basic learning method 1. MEBN-RM Model
– A bridge between MEBN and RM (Relational Model)
2. A basic structure learning algorithm for MEBN – A ground induction method
Limitation The basic learning method only focuses on
– Discrete variable, complete data, relational model, no aggregating and cycle assumption
7
THE BASIC LEARNING METHOD
1. MEBN-RM Model 2. A basic structure learning algorithm for
MEBN
8
1.1. MEBN-RM Model: Relational Model
9
1.2. MEBN-RM Model: MEBN Model
10
Multi-entity Bayesian networks Model
Bayesian Networks Model
Deduction or reasoning
1.3. MEBN-RM Model
MEBN-RM Model 11
Relational Model MEBN Model
1.3. MEBN-RM Model: Example
12
Relational Model MEBN Model
MEBN-RM Model
2. Basic structure learning algorithm for MEBN
13
CASE STUDY To evaluate the method, we used a dataset which came
from the PROGNOS (Probabilistic OntoloGies for Net-centric Operation Systems: The purpose of the system is to provide higher-level knowledge representation, fusion, and reasoning in the maritime domain.)
The PROGNOS’ simulation provides the ground truth information sampled by a single entity Bayesian Network
85000 persons, 10000 ships, and 1000 organizations with various values of attributes sampled were used for test dataset
14
Test process
15
Single entity Bayesian Network Relational Model/Database
Generated SSBN of PROGNOS MTheory Learned PROGNOS MTheory
1. Sampling
4. Comparing two models 2. Learning
3. Reasoning
Test Result: Learned PROGNOS MTheory
16
Test Result: Generated situation-specific Bayesian Network (SSBN)
17
Test Result: Comparing two models
18
ROC of Learned MTheory and Single Entity Network
AUC of Learned MTheory and Single Entity Network
DISCUSSION AND FUTURE WORK
In this paper As a bridge between MEBN and RM, MEBN-RM Model was
introduced For induction, the Basic Structure Learning for MEBN was
suggested
Future work MEBN learning based on ontology Incomplete and continuous data Solving aggregating and cycle problem
19
References [1] Hassan Khosravi, Bahareh Bina. A Survey on Statistical Relational Learning. In Proceedings of Canadian Conference on AI'2010. pp.256~268 [2] Getoor, L., Tasker, B.: Introduction to statistical relational learning. MIT Press, Cambridge, 2007 [3] Domingos, P., Richardson, M.: Markov logic: A unifying framework for statistical relational learning. In: Introduction to Statistical Relational Learning, ch. 12, pp. 339–367, 2007 [4] Nevile, J., Jensen, D.: Relational dependency networks. In: An Introduction to Statistical Relational Learning
[5] Kersting, K., de Raedt, L.: Bayesian logic programming: Theory and tool. In: Introduction to Statistical Relational Learning [6] Oliver Schulte, Hassan Khosravi, Flavia Moser, and Martin Ester. Join bayes nets: A new type of bayes net for relational data. Technical Report 2008-17, Simon Fraser University, 2008. also in CS-Learning Preprint Archive. [7] Laskey, K. B.,MEBN: A Language for First-Order Bayesian Knowledge Bases. Artificial Intelligence, 172(2-3), 2008 [8] Pearl, J.: Probabilistic Reasoning in Intelligent Systems.Morgan Kaufmann, San Francisco, 1988 [9] Jensen, D., Neville, J.: Linkage and autocorrelation cause feature selection bias in relational learning. In: Proceedings of the 19th International Conference on Machine Learning, 2002 [10] S. Natarajan, P. Tadepalli, T. G Dietterich, and A. Fern. Learning first-order probabilistic models with combining rules. Special Issue on Probabilistic Relational Learning, AMAI, 2009. [11] Paulo C. G Costa, Bayesian Semantics for the Semantic Web. PhD Dissertation, George Mason University, July 2005. Brazilian Air Force. [12] Rommel N. Carvalho, Probabilistic Ontology: Representation and Modeling Methodology, PhD Dissertation, George Mason University, July 2011. [13] Codd, E.F. "A Relational Model of Data for Large Shared Data Banks". Communications of the ACM, 1970 [14] Codd, E.F. "Derivability, Redundancy, and Consistency of Relations Stored in Large Data Banks", IBM Research Report, 1969 [15] Suh, N. P., The Principles of Design, Oxford University Press, New York, 1990 [16] Shannon, C.E., "A Mathematical Theory of Communication", Bell System Technical Journal, 27, pp. 379–423 & 623–656, July & October, 1948 [17] Daphne Koller and Nir Friedman. Probabilistic Graphical Models: Principles and Techniques. The MIT Press, 1 edition, August 2009. [18] D. Heckerman. A tutorial on learning with Bayesian networks. In M. I. Jordan, editor, Learning in Graphical Models. MIT Press, Cambridge, MA, 1998. [19] Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, 2006 [20] D.M. Chickering, Optimal Structure Identification with Greedy Search, 2002. [21] D.M. Chickering, Learning Equivalence Classes of Bayesian-Network Structures, 2002.
20