mastergoal machine learning environment phase iii presentation alejandro alliana cis895 mse project...
TRANSCRIPT
![Page 1: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/1.jpg)
Mastergoal Machine Learning Environment
Phase III PresentationAlejandro Alliana
CIS895 MSE Project – KSU
![Page 2: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/2.jpg)
MMLE Project Overview Provide an environment to create,
repeat, save experiments for creating strategies for playing Mastergoal using ML techniques.
Divided in 4 sub-projects. Mastergoal (MG) Mastergoal AI (MGAI) Mastergoal Machine Learning (MMLE) User Interface (UI)
![Page 3: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/3.jpg)
Phase III Artifacts
User Documentation Component Design Source Code (and exe) Assessment Evaluation Project Evaluation References
![Page 4: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/4.jpg)
User Documentation
MMLE UI User manual provided. Install Common use cases
Data fields types and formats Examples
MG, MGAI, MMLE libraries. API generated by doxygen.
![Page 5: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/5.jpg)
Component Design
Project divided in 4 main subprojects. UI uses MMLE, MGAI and MG as
libraries.
![Page 6: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/6.jpg)
Component Design – Design Patterns Used Factory Method: Board, Search Algorithm. Prototype: Strategy. Singleton: All factories, Terms, Fitness
Functions, Selection Criteria, Termination Criteria.
Template Method: Terms, Strategy. Strategy: Search Algorithms – Agents,
Strategy – Search Algorithms. Observer: GameSubject – GameListener,
TrainSubject – TrainListener. Proxy: TrainBridge, UIAgentProxy
![Page 7: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/7.jpg)
Component Design
Deployment Diagrams show packages and files.
![Page 8: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/8.jpg)
Component Design
Deployment diagrams.
Class diagrams Sequence
diagrams. Object Diagrams. Short description
of classes and link to API online documentation.
![Page 9: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/9.jpg)
Source Code
Kept in SVN repository (7 projects). 4 sub-projects. 3 test sub-projects.
Metrics were taken weekly and will be discussed later in the presentation.
![Page 10: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/10.jpg)
Installer and executable. Installer created with NSIS (Nullsoft
Scriptable Install System). UI created with the Windows Forms
GUI API available in the .NET Framework.
All other sub-projects coded in (unmanaged) C++ and are available as libraries.
![Page 11: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/11.jpg)
MMLE Demonstration.
![Page 12: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/12.jpg)
Assessment Evaluation I used the CPPUnit framework to
perform unit testing on the projects. MastergoalTest MastergoalAiTest Mmle-test
Assertions used to test for pre and post-coditions.
I used the Visual Leak Detector system to detect memory leaks .
![Page 13: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/13.jpg)
Assessment Evaluation
Test Plan All test passed*
CPPUnit Regression Bugs. Coding of test cases. Document and debug test cases. Memory Leak Bugs.
![Page 14: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/14.jpg)
Assessment Evaluation – Defect Log.
![Page 15: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/15.jpg)
Size of the test projects. Overall the three test projects
have 1125 lines of code.
Size of each Project (LOC)
0500
100015002000250030003500400045005000
mas
tergo
al
mas
tergo
al-test
mas
tergo
alAI
mas
tergo
alAIT
est
MM
LE
MM
LE-T
est
UI
LO
C
![Page 16: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/16.jpg)
Assessment Evaluation - Metrics
Chart Title
Elaboration, 155
Implementation, 6141
0 1000 2000 3000 4000 5000 6000 7000
Implementation
Elaboration
Implementation 6141
Elaboration 155
Assessment
Sum of Delta _Min
WorkflowPhase
![Page 17: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/17.jpg)
Assessment Evaluation - Metrics
Chart Title
Documentation, 475
Test Assesment, 136
Test Plan, 155
Testing, 5530
0 1000 2000 3000 4000 5000 6000
Testing
Test Plan
Test Assesment
Documentation
Testing 5530
Test Plan 155
Test Assesment 136
Documentation 475
Assessment
Sum of Delta _Min
Workflow
Task
![Page 18: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/18.jpg)
Project Evaluation
Souce code metrics
02000400060008000
100001200014000
1/16
/200
8
1/23
/200
8
1/30
/200
8
2/6/
2008
2/13
/200
8
2/20
/200
8
2/27
/200
8
3/5/
2008
3/12
/200
8
3/19
/200
8
3/26
/200
8
4/2/
2008
4/9/
2008
4/16
/200
8
4/23
/200
8
4/30
/200
8
Time
NOM
LOC
COM
![Page 19: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/19.jpg)
Project Evaluation
Size of each Project (LOC)
0500
100015002000250030003500400045005000
mas
tergo
al
mas
tergo
al-test
mas
tergo
alAI
mas
tergo
alAIT
est
MM
LE
MM
LE-T
est UI
LO
C
![Page 20: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/20.jpg)
Project Evaluation
Metrics 533 hours (13.3 weeks or 3 months over a
period of 10 months) and 11 KLOC. Estimations
FP: Time 10.79 months, 2.79 KLOC. COCOMO: Time 9.24 months, 7.5 KLOC. COCOMO II: Time 9.54 months, 7.5 KLOC.
![Page 21: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/21.jpg)
Project Evaluation - FP Real
11 KLOC and 3 months. Estimates of Function Points
Size 2.79 KLOC, Time 10.79 months. Lack of experience using FP. Some of the user interfaces were more
complex than previously thought No .NET conversion rates A big part of the project is the user interface
which contains automatically generated code Algorithms not well represented.
![Page 22: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/22.jpg)
Project Evaluation - COCOMO Real
11 KLOC and 3 months. Estimates of COCOMO.
Actual size is arbitrary, based on experience. Size 7.5 KLOC, time = 9.25 months Inexperience in C++/ .NET. Conversion rates of the languages.
Estimates of COCOMO II Application Composition model 5.57 Person
months (Object Points / Productivity) Post Architectural Model 9.54 months (7.5
KLOC)
![Page 23: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/23.jpg)
Project Evaluation - Time spent at each phase.
Total
Total, 4814
Total, 21473
Total, 5694
Elaboration
Implementation
Inception
Sum of Delta _Min
Phase
![Page 24: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/24.jpg)
Project EvaluationChart Title
Assessment, 6296
Design, 4144
Environment, 598
Implementation, 14935
Infrastructure, 12
Management, 2686
Planning, 384
Requirements, 645
Research, 2267
(blank), 14
0 2000 4000 6000 8000 10000 12000 14000 16000
Sum of Delta _Min
Workflow
![Page 25: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/25.jpg)
Project Evaluation - Lessons Learned
Implementation: C++ language, memory
management, implementation of design patterns.
Tools and libraries (NSIS, CPPUnit, VLD, Doxygen)
Design: Design Patterns.
![Page 26: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/26.jpg)
Project Evaluation - Lessons Learned Experience on various estimate models. Measurement
Tools (CCCC, Process Dashboard). Testing.
CPPUnit framework. VLD.
Process Iterative process Artifacts
![Page 27: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/27.jpg)
Project Evaluation - Future work Improve performance of search algorithm
and add new algorithms. Add more functionality to the game
playing library and UI. Add more selection mechanisms to the
GA Experiments. Add more learning algorithms. Distributed computation to speed up
training Refactoring of some classes. Add test classes for each feature.
![Page 28: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/28.jpg)
Project Evaluation Results
Learning Level I
0
0.2
0.4
0.6
0.8
1
1.2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Generations
Fit
nes
s max
avg
![Page 29: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/29.jpg)
Project Evaluation Results
Learning Level II
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
1 3 5 7 9 11 13 15 17 19 21 23 25
Generations
Fit
nes
s max
avg
![Page 30: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/30.jpg)
Project Evaluation Results
Learning level III
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
1 2 3 4 5 6 7
Generations
Fit
nes
s
max
avg
![Page 31: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/31.jpg)
Tools Used MS Visual Studio 2005 BoUml Rational Software Architect NSIS (Nullsoft Scriptable Install System) CCCC (C and C++ Code Counter) TinyXML Visual Leak Detector Doxygen Process Dashboard TortoiseSVN
![Page 32: Mastergoal Machine Learning Environment Phase III Presentation Alejandro Alliana CIS895 MSE Project – KSU](https://reader037.vdocuments.us/reader037/viewer/2022103100/56649ee45503460f94bf3722/html5/thumbnails/32.jpg)
References Design Patterns: Elements of Reusable Object-Oriented Software,
Gamma, Erich; Richard Helm, Ralph Johnson, and John Vlissides (1995). Addison-Wesley. ISBN 0-201-63361-2.
Machine Learning, Tom Mitchell, McGraw Hill, 1997 ISBN 0-07-042807-7 BoUML http://bouml.free.fr/ Rational Software Architect
http://www-306.ibm.com/software/awdtools/architect/swarchitect/ http://sourceforge.net/projects/tinyxml/ NSIS (Nullsoft Scriptable Install System)
http://nsis.sourceforge.net/Main_Page CCCC (C and C++ Code Counter) http://sourceforge.net/projects/cccc CPPUnit
http://cppunit.sourceforge.net/doc/lastest/cppunit_cookbook.html TinyXML http://www.grinninglizard.com/tinyxml/ Visual Leak Detector available at http://dmoulding.googlepages.com/vld Doxygen http://www.stack.nl/~dimitri/doxygen/ Process Dashboard http://processdash.sourceforge.net/ TortoiseSVN http://tortoisesvn.tigris.org/