![Page 2: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/2.jpg)
Background
• DISCOM is a tool to record, playback, & print-out Distributed Interactive Simulation PDUs -- a UDP datagram broadcasted on the network
• DISCOM was rewritten from a structured approach to an Object-Oriented approach a year ago.
• DISCOM presents a unique opportunity to analyze quality attributes from both a structured approach as well as an object-oriented approach
![Page 3: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/3.jpg)
DISCOM Explanation
![Page 4: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/4.jpg)
Quality Attributes to Analyze
• Performance/Efficiency
• Maintainability
• Reliability
• Testability
• Reusability
![Page 5: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/5.jpg)
Performance/Efficiency
Definition:• The response time, utilization, and throughput behavior of
the system. As well as fulfilling its purpose without waste of resources
Hypothesis:• Structural Approach is marginally faster.
Measurement Approach:• VTune Performance Analyze both copies with a log file.• Analyze cost of dynamic allocation • Observe memory usage
![Page 6: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/6.jpg)
VTune Performance Analyzer
![Page 7: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/7.jpg)
VTune Results
Module ProceduralTime (μs)
OOTime (μs)
Speedup
Packet Processing 1,479,967 5,842,532 -3.95X
File Writing 277,978 573,210 -2.06X
NetworkListener
261,850 425,221 -1.62X
Graphics 13,168,507 20,231,334 -1.54X
Other 7681627 11,563,422 -1.51X
Sleep Time 33,322,979 17,887,733 N/A
Main 56,192,908 56,523,452 N/A
Total Exec 22,869,929 38,635,719 -1.69X
Table 1. Performance Metrics
1 minute playback file consisting of ~10000 packets/59000 PDUs
![Page 8: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/8.jpg)
With more effort…
The cost of dynamic allocation, C, is given by:
)(* DEPTH
dc ssNC
where N is the number of calls, depth is the number of contained objects that must be created when the class is created, sc and sd are the service time to create the object and to destroy the object respectively.
The cost of context switching, C, is given by:
)(* DEPTH
uw ttNC
where N is the number of calls, depth is the number of contained objects that must be created when the class is created, tw and tu are the winding time and the unwinding time to push to the call stack and pop from the call stack respectively.
![Page 9: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/9.jpg)
Efficiency Metrics
Statistic Procedural OO Factor
Memory Median
10,402K 13,650K -1.31X
ExtendedDurationMemoryUsage
10,402K 24,232K* -2.33X
Table 2. Efficiency Metrics
* May be indicative of memory leak
1 minute playback file consisting of ~10000 packets/59000 PDUs Extended = 1 minute-playback looped 10x
![Page 10: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/10.jpg)
Performance/Efficiency Conclusion
• Performance: – Close to 2X slow-down in using OO-code
• Efficiency: – OO requires more memory. – Probably even more so in a pure-OO language
(like Java) where everything must be instantiated and no concept of pointers
– More prone to hazards (excessive dynamic allocation of objects)
![Page 11: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/11.jpg)
Definition: • The extent to which software facilitates updates to satisfy
new requirements
Hypothesis:• Object-Oriented approach is more easily maintainable
Measurement Approach:• Track resolution time for Discrepency Reports• Track change time for Change Requests• Track prevalance of changes from affected file-listings• Measure cyclomatic complexity
Maintainability/Modifiability
![Page 12: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/12.jpg)
Cyclomatic Complexity
• Cyclomatic Complexity directly measures the number of linearly independent paths through a program's source code.
• Equivalent to CFG Paths (ifs, loops, etc)
• WMC is OO-metric that rolls-up CCs in class
• Use a Tool Called CodeAnalyzerPro to measure CC, WMC, & SLOC
![Page 13: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/13.jpg)
Code Analyzer Pro
![Page 14: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/14.jpg)
Modifiability MetricsParadigm Max
CCMAXWMC
SLOC
UI Procedural 92 N/A 8643
OO 14 93 5235
DIS Procedural 147 N/A 42423
OO 17 185 23563
Total Procedural 147 N/A 51066
OO 17 185 28798
Table 4. Modifiability Metrics
![Page 15: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/15.jpg)
Modifiability Metrics (Cont.)
Paradigm #DRsTotalTime (Hrs.)
AvgResolu
tionTime
Procedural 24 94.5 3.94 Hours
OO 11 29.2 2.65 Hours
Table 4. Discrepency Metrics
Paradigm #CRsTotalTime (Hrs.)
AvgChang
eTime
Procedural 29 204.0 7.03 Hours
OO 19 98.2 5.17 Hours
Table 4. Change Metrics
![Page 16: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/16.jpg)
Modifiability Conclusion
• OO paradigm appears to be more understandable by way of lower method Cyclomatic Complexity and SLOC count
• Change Request / Discrepency Report tracking information seems to support this conclusion
![Page 17: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/17.jpg)
Reliability
Definition:• It can be expected to perform its intended
functions satisfactorily
Hypothesis:• Reliability growth is steeper with OO-Design
methodology
Measurement Approach:• Track Discrepancy Reports• Reliability Modeling of event simulation log files
using CASRE to generate reliability profiles
![Page 18: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/18.jpg)
Reliability Growth Modeling Setbacks
• Reliability is generally very high with many hidden problems that are not recorded.
• Furthermore, I could not find meaningful unclassified data-set for procedural paradigm and while I could have gone ahead with evaluation of OO-Reliability growth, there would have been nothing to compare it to.
![Page 19: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/19.jpg)
Reliability Metrics
Paradigm # DRs #Runs Avg Run Time
#DRs/Minute
Procedural
8 15 52.4 0.01018
OO 5 17 53.9 0.00055
Table 3. Reliability Metrics
Paradigm # DRs
Severity
Average
(1-5)
Avg Resolution Time (Hrs from DR
Entry)
Procedural
8 3.2 73.5
OO 5 2.2 34.2
Table 4. Reliability Growth Metrics
![Page 20: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/20.jpg)
Reliability Conclusion
• OO exhibited higher reliability growth after integration deployment
• Isn’t conclusive proof to support a clear determination on the Reliability attribute
• Exception handling may help with discrepency resolution as well as severity (try/catch)
![Page 21: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/21.jpg)
Testability
Definition:• The extent that software facilitates the establishment of
acceptance criteria and supports evaluation of its performance
Hypothesis:• From a class-level unit-testing standpoint OO design
prevail. From a system level or for debugging issues a structural design.
Measurement Approach:• Measure cyclomatic complexity• Perform unit-testing and document qualitatively the ease
![Page 22: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/22.jpg)
Cyclomatic Complexity Revisited
Paradigm Max CC MAXWMC
SLOC
UI Procedural 92 N/A 8643
OO 14 93 5235
DIS Procedural 147 N/A 42423
OO 17 185 23563
Total Procedural 147 N/A 51066
OO 17 185 28798
![Page 23: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/23.jpg)
Testability Qualitative Assessment
Paradigm Unit TestHooks
DebuggingEase
Procedural HARD [4 unit test hooks inserted --
challenging]
Moderate [Difficult with multiplicity of objects, Easy with 1
object]
OO EASY [10 unit test hooks quickly easilyinserted]
Moderate[Due to several paths]
![Page 24: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/24.jpg)
Testability Conclusion
• Object-Oriented Paradigm is better suited for testability. – More logical coherence– Less paths/more functions for easier unit-test
hook insertion– Less Cyclomatic Complexity within functions– Debugging ease similar across paradigms.
![Page 25: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/25.jpg)
Reusability
Definition:• The likelihood a segment of source code can be
used again to add new functionalities with slight or no modification
Hypothesis:• OO approach is more reusable
Measurement Approach:• Observe opportunities available and opportunities
taken for reuse at both the object level and and the system level
![Page 26: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/26.jpg)
Reusability Approach
• The first step at evaluation was to survey both reuse opportunities available and reuse opportunities taken in the labs.
• Next, reusability was gauged by qualitatively mashing a simplistic Create Entity PDU into both an application with an existing DIS interface as well as one without. This was done utilizing the code from both the procedural application as well as the object-oriented application.
• Measurement was qualitative primarily from a time perspective as SLOC modified is not necessarily indicative of the ease of reuse.
![Page 27: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/27.jpg)
Reusability Metrics
Paradigm Reuse OpsRealized/Year
DifficultyExist App
DifficultyNew App
Procedural 0.2 MED MED/EASY
OO 1 HARD EASY
Table 5. Reusability Metrics
![Page 28: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/28.jpg)
Reusability Conclusion
Existing Applications:
For applications with existing interfaces reusability is moderately smoother if one begins with code from the procedural paradigm. This is due primarily, in my opinion, to ease of understandability. With Object-Oriented Code, one has to understand 2 designs, both the source and the target, in order to adapt it to fit the needs.
New Applications:
For applications without existing interfaces reusability is smoother if one begins with the Object-Oriented Paradigm due to the enhanced modularity and portability of the approach.
![Page 29: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/29.jpg)
References
• Software Performance AntiPatternsConnie U. Smith, Lloyd G. Williams
• Indicators of Structural Stability of Object-Oriented Designs: A Case Study
Mahmoud O. Elish and David Rine• Has Object-Oriented Programming Delivered?
Greg Goth• A Controlled Experiment in Maintenance Comparing
Design Patterns to Simpler Solutions
Lutz Prechelt, Walter F. Tichy,
![Page 30: DISCOM Quality Attribute Impact A Case Study in OO-Design Lewis Sykalski E-mail: lsykalski@smu.edulsykalski@smu.edu](https://reader030.vdocuments.us/reader030/viewer/2022032708/56649e7f5503460f94b83d9b/html5/thumbnails/30.jpg)
Questions/Comments