xcs: current capabilities and future challenges
DESCRIPTION
Martin Butz presents the current state-of-the-union of XCSTRANSCRIPT
XCS: Current capabilities and futurechallenges
Martin V. ButzDepartment of Cognitive Psychology (III)
University of Würzburg, Germanyhttp://www-illigal.ge.uiuc.edu/~butz
02/23/2006
05/16/2006 XCS: Current Capabilities & Future Challenges 2Martin V. Butz
Overview
1. The XCS Classifier System
2. XCS - Capabilities
3. XCS - Future Challenges
4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges 3Martin V. Butz
1. The XCS Classifier System
1. The XCS Classifier System1. Framework
2. Evolutionary Pressures
3. Computational Complexity
4. General Learning Intuition2. XCS – Capabilities
3. XCS – Future Challenges
4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges 4Martin V. Butz
The XCS Classifier System1 The XCS Classifier System
• Learning classifier system– Rule-based representation of condition-action-predictions– Steady-state GA for evolution of conditions– Gradient-based techniques for estimation of predictions
• Major characteristics:– Q-learning based reinforcement learning– Relative accuracy-based fitness– Action-set restricted selection, that is, niche selection– Panmictic (population-wide) deletion
Goal:
Learn a complete maximally accurate,maximally general predictive model.
05/16/2006 XCS: Current Capabilities & Future Challenges 5Martin V. Butz
Rule Evaluation
• Gradient-based techniques for derivation of predictionand error estimates
• Q-learning derived update
• Propagation of reward possible
• Rule quality depends on inverse of error estimate
• Accuracy of rule prediction determines fitness
1 The XCS Classifier System
05/16/2006 XCS: Current Capabilities & Future Challenges 6Martin V. Butz
Evolutionary Algorithm
• Fixed population size
• Steady-state genetic algorithm
• Two reproductions and deletions per iteration– Reproduction in action set based on fitness
– Deletion from whole population based on coverage
• Genetic operators:– Mutation
– Recombination
1 The XCS Classifier System
05/16/2006 XCS: Current Capabilities & Future Challenges 7Martin V. Butz
Learning Suitability
• XCS represents its solution by a collection of sub-solutions(that is, the population of classifiers).
• XCS learns an effective problem space clustering(subspaces) in its conditions.
• Clusters (subspaces) evolve to enable maximally accuratepredictions.– Accuracy can be bounded (error threshold ε0 and population size
relation).– Basically any form of prediction is possible (e.g. reward, next
sensory input, function value).
1 The XCS Classifier System
05/16/2006 XCS: Current Capabilities & Future Challenges 8Martin V. Butz
XCS Power
• Combination of– Gradient-based techniques (to generate predictions)– Evolutionary techniques (to generate features / clusters)
• Advantage:– Usage of gradient-based error-feedback learning where possible– Usage of evolutionary techniques
• Where error-feedback is hard or impossible to propagate into• Where error-feedback learning gets easily stuck in local optima
• Thus:– Combine best approximation technique (error-feedback learning) with
best evolutionary technique (representation and operators)
1 The XCS Classifier System
05/16/2006 XCS: Current Capabilities & Future Challenges 9Martin V. Butz
2. XCS – Current Capabilities
1. The XCS Classifier System
2. XCS – Current Capabilities
1. Binary and Real-world Classification Problems
2. Function Approximation Problems
3. (Multistep) Reinforcement Learning Problems3. XCS – Future Challenges
4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges 10Martin V. Butz
The Multiplexer Problem
……
0110000
0100101
1100010
0101101
0011011
0000111
1001000
0000000
ClassProblem instance
101000010##0#
……………
101000101#1##
100001#1##
100101#0##
101000001#0##
1010001001###
1000001###
1001000###
1010000000###
FεRAC
Optimal solution representation
2 XCS – Current Capabilities
05/16/2006 XCS: Current Capabilities & Future Challenges 11Martin V. Butz
XCS Performance in MP 702 XCS – Current Capabilities
05/16/2006 XCS: Current Capabilities & Future Challenges 12Martin V. Butz
Hierarchical Classification Problem
• Hierarchical problems with low order dependencies(“building blocks”) and further high-order dependencies
• BB structures are re-used on the higher level to deriveproblem class.
• Example: Hierarchical 3-parity, 6-multiplexer problem:
2 XCS – Current Capabilities
05/16/2006 XCS: Current Capabilities & Future Challenges 13Martin V. Butz
XCS/BOA Performance2 XCS – Capabilities
05/16/2006 XCS: Current Capabilities & Future Challenges 14Martin V. Butz
Classification of Real-World Datasets
• Conditions are coded with attributes dependent on type ofattribute in dataset (interval coding or Binary coding).
• Experiments in 42 datasets (from UCI and other sources)• Comparisons with ten other ML systems (pairwise t-test)• XCS learns competitively but it is a much more general
learning system.
24/99/149/189/1514/77/619/125/930/138/095%
23/88/139/179/1113/75/619/125/829/138/099%
SMO
(rad.)
SMO
(pol.3)
SMO
(poly)
IB3IB1PARTNaïve
Bayes
C4.51-RMaj.XCS
2 XCS – Capabilities
05/16/2006 XCS: Current Capabilities & Future Challenges 15Martin V. Butz
Piecewise Linear Function Approximation
0 0.2 0.4 0.6 0.8 1x
0 0.2 0.4 0.6 0.8 1
y
-1-0.8-0.6-0.4-0.2
0 0.2 0.4 0.6 0.8
1
f(x,y)
2 XCS – Capabilities
05/16/2006 XCS: Current Capabilities & Future Challenges 16Martin V. Butz
Performance in 3D Sinusoidal Function2 XCS – Capabilities
05/16/2006 XCS: Current Capabilities & Future Challenges 17Martin V. Butz
Performance Evaluation in Maze62 XCS – Capabilities
05/16/2006 XCS: Current Capabilities & Future Challenges 18Martin V. Butz
Performance Maze6 plus Irrelevant Bits2 XCS – Capabilities
05/16/2006 XCS: Current Capabilities & Future Challenges 19Martin V. Butz
3. XCS – Future Challenges
1. The XCS Classifier System
2. XCS – Performance Demonstration
3. XCS – Future Challenges1. Representation & Operators
2. Niching
3. Dynamic Problems
4. Compactness of solution / population
5. Hierarchical classifier systems4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges 20Martin V. Butz
Representation & Operators
• Different representations of conditions– Binary, real-valued, mixed– Kernels, bases– Combinations, Hybrids
• Different representations of predictions– Constant, Linear, Polynomial– State prediction, property prediction– Control variable prediction
• Most suitable operators for representations– Approximation operators (use best gradient method)– Genetic operators (mind XCS problem bounds)
4.1 XCS –Future Challenges
05/16/2006 XCS: Current Capabilities & Future Challenges 21Martin V. Butz
XCS and Niching
• Currently:– Niching is done occurrence-based
– Number of classifiers in large problem niches unnecessarylarge
– Number of classifiers in small but hard to approximate problemspaces potentially too small -> niche loss
• Additional balancing mechanisms might be necessary!
4.1 XCS –Future Challenges
05/16/2006 XCS: Current Capabilities & Future Challenges 22Martin V. Butz
Dynamic Problems
• Currently: XCS was applied mainly to static problems– Makes iterative, adaptive approach not really necessary
• Dynamic problems– Concept class changes
– Reward distribution changes
– Problem sampling changes
• Question:– How quickly can XCS adapt to these changes?
– Can we improve adaptation?
4.1 XCS –Future Challenges
05/16/2006 XCS: Current Capabilities & Future Challenges 23Martin V. Butz
Compactness of Solution / Population
• Problem in XCS:– Population sizes get rather big.
– Final solution is not very compact
– Solutions indicate overfitting problems in dataminig problems
• Main generalization mechanism purely based onoccurrence frequency
• Borders between different classes are ill-defined.
• How can we efficiently compact the population online orusing post-processing (some approaches available)?
05/16/2006 XCS: Current Capabilities & Future Challenges 24Martin V. Butz
Hierarchical Classifier System
• Local structures are often used by many higher-orderstructures (decomposability of environment, problems, etc.)
• Can we build higher-level classifier structures that build onevolving lower-level structures….
• The hierarchical boolean function problems as a start?
05/16/2006 XCS: Current Capabilities & Future Challenges 25Martin V. Butz
4. Summary and Conclusions
1. The XCS Classifier System
2. XCS – Performance Demonstration
3. XCS – Future Challenges4. Summary & Conclusions
05/16/2006 XCS: Current Capabilities & Future Challenges 26Martin V. Butz
Conclusions
• XCS is designed to– Cluster the problem space to enable– Maximally accurate predictions
• XCS is a highly flexible learning system– Conditions of various types possible– Predictions of various types possible
• Major XCS challenges lie in the further development of– Representation & Operators– Niching– Dynamic Problems– Compactness of solution / population– Hierarchical classifier systems
• XCS has a big potential due to the combination of– Gradient-based update mechanisms– Evolutionary-based feature extraction mechanisms
4 Summary and Conclusions
Thank You for Your Attention