cots project types cebase cots research ye yang, jesal bhuta, dan port
Post on 21-Dec-2015
218 views
TRANSCRIPT
COTS Definition
• For the purposes of these classifications, we use the following definition of COTS
“A software product, developed by a third party (who controls its ongoing support and evolution), bought, licensed, or acquired for the purposes of integration into a larger system as an integral part, i.e. that will be delivered as part of the system to the customer of that system (i.e. not a tool), which might or might not allow modification at the source code level, but may include mechanisms for customization, and is bought and used by a significant number of systems developers.”
MBASE Milestone ElementsMilestone Element
Life Cycle Objectives (LCO) Life Cycle Architecture (LCA)
Definition of Operational Concept
Top-level system objectives and scope - Shared vision of expected initiatives and outcomes [COTS strategy for reuse and legacy elements] - System boundary [major COTS
component boundaries within system] - Environment parameters and assumptions
[CBS success factors] - Evolution parameters [major COTS
vendor product availability, upgrade cycles]
Operational concept - Operations and maintenance scenarios and parameters [COTS maintenance] - Organizational life-cycle responsibilities (stakeholders) [major COTS vendors]
Elaboration of system objectives and scope by increment
Elaboration of operational concept by increment [COTS operation and use summary, vendor support strategies, COTS adoption impact on organization]
System Prototype(s)
Exercise key usage scenarios [demonstrate key COTS capabilities and validate interfaces, scope use and value of major COTS products]
Resolve critical risks [COTS selection and evaluation plan]
Exercise range of usage scenarios [complete COTS evaluations, initial COTS configuration and training, initial COTS tailoring]
Resolve major outstanding risks [resolve COTS integration issues and level of use]
Definition of System Requirements
Top-level functions, interfaces, quality attribute levels, including:
- Growth vectors - Priorities - Legacy systems and environments - [Establish key COTS evaluation
attributes and screening parameters] Stakeholders’ concurrence on essentials
[mandated COTS packages, suppliers and vendors, identify negotiable and non-negotiable COTS requirements, COTS evaluation win-conditions and constraints]
Elaboration of functions, interfaces, quality attributes by increment
- Identification of TBDs (to-be-determined items) [mapping of COTS capabilities to requirements]
Stakeholders’ concurrence on their priority concerns [concurrence on COTS imposed requirements and 90% re-worked requirements tradeoffs]
MBASE Milestone Elements (2)Milestone Element
Life Cycle Objectives (LCO) Life Cycle Architecture (LCA)
Definition of System and Software Architecture
Top-level definition of at least one feasible architecture
- Physical and logical elements and relationships, including top-level domain factoring and identification of possible design patterns
- Choices of COTS and reusable software elements [mapping of critical system components to COTS products]
Identification of infeasible architecture options
Choice of architecture and elaboration by increment
- Physical and logical components, connectors, configurations, constraints
- COTS, reuse choices [CBS design (mapping of components and system factors to COTS), COTS package configurations]
- Domain-architecture and architectural style choices [design for COTS glue code and wrappers]
Architecture evolution parameters
Definition of Life-Cycle Plan
Identification of life-cycle stakeholders - Users, customers, developers,
maintainers, interpreters, general public, others [COTS suppliers and vendors]
Identification of life-cycle process model - Top-level stages, increments [COTS
deployment – prototyping, evaluation, schedule, risks impact of schedule and cost, stakeholders, license negotiations]
Top-level WWWWWHH* by stage [COTS product lifecycles (releases, delivery dates, costs, training needs)
Elaboration of WWWWWHH* for Initial Operational Capability (IOC)
- Partial elaboration, identification of key TBDs for later increments
- [COTS upgrades, patches] - [License managament (non-strandard
provisions)] - [COTS users, maintainer, and developer
skill set attributes] - [COTS training plan] - [COTS risks realized; integration,
cultural, economic]
Feasibility Rationale
Risk assessment and mitigation plans [COTS vendor issues – product availability, stability, costs]
Assurance of consistency among elements above
- Via analysis, measurement, prototyping, simulation, etc.
- Business case analysis for requirements, feasible architectures [buy versus build rationale, justification of CBS strategy, CBS tradeoffs]
Assurance of consistency among elements above [feasibility of COTS vendor relationships, COTS maintenance costs, organization readiness to implement COTS]
All major risks resolved or covered by risk management plan [completed license contracts, COTS product alternatives and marketplace issues, COTS organization culture issues, vendor commitments]
COTS Based Systems• A COTS Based System (CBS) is any system that
more than 50% of a systems functional requirements are implemented through use of COTS via glue code, tailoring, and assessment effort.– Glue code is the custom development needed to integrate
COTS packages within an application external to the packages themselves
– Tailoring effort is the activities relating to adapting COTS packages internally for use within an application
– Assessment effort is the suitability evaluation and analysis of the desired attributes of the COTS packages for an application
* The above definitions are compatible COCOTS definitions
CBS Approaches
• There are a variety of approaches to developing COTS based systems (CBS). – Find and use COTS packages as is (“turnkey”)– Find and adapt COTS packages as needed (“adaptation”)– Use COTS as components in custom built application
(“integration”)
• These approaches differ considerably and driven primarily by the systems shared vision and desired characteristics & priorities (DC&P’s). – Effort distributed between
• Glue code• Assessment• Tailoring
CS577B COTS Effort Distribution
0%
20%
40%
60%
80%
100%
Team7
Team8
Team11
Team20
Team21
Team9
Team15
577ab2001-2002 Projects COTS effort data
Glue Coding
Tailoring
Assessment
0%
20%
40%
60%
80%
100%
Team 3 Team 6 Team 8 Team 14 Team 21
577ab 2000-2001 Project COTS Effort Data
Glue Coding
Tailoring
Assessment
Source: CS577 Project Effort Data2000-2002
CBS Types
• Roughly, a CBS project can be classified as either
• Assessment Intensive • Tailoring Intensive• Application Intensive
• These classifications provide an overview of the potential risks, attributes, and development effort priorities.
– Use for strategic and tactical development planning
Assessment Intensive
• An assessment intensive CBS project will have
– The DC&P’s covered without much modification (or tailoring) by a single COTS framework or through the simple integration of several.
– The primary effort is focused on identifying collection of candidate COTS products and evaluating (i.e. assessing) a feasible set of products whose existing capabilities directly address a desired operational concept.
Tailoring intensive CBS
• A tailoring intensive CBS project is where
– The DC&P’s are covered by a single (or very few) COTS framework(s).
– The primary effort is on adapting a COTS framework whose existing general capabilities can be customized (i.e. tailored) in a feasible way to satisfy the majority of a systems capabilities or operational scenarios.
Application Intensive
• The application intensive CBS project will have
– Some nontrivial DC&P’s covered by COTS, and some that must be covered by custom development.
– The primary effort is to identify COTS packages that can feasibly used as components to address subsets of the required system capabilities.
– Typically there are significant “buy versus build” decisions
• Critical factors are: available COTS products, schedule limitations, budget limitations, and desired current and future capabilities
Effects of CBS Types
• The CBS type effects many aspects of a project– MBASE development guidelines– Glue code, assessment, tailoring effort– Risk factors– Requirements flexibility– System evolution and maintenance – Cost factors
CBS project Type Guidelines
System shared vision Desired characteristics & priorities (DC&P’s)
System shared vision Desired characteristics & priorities (DC&P’s)
Relation of candidate COTS products to DC&P’s
Relation of candidate COTS products to DC&P’s
Standard MBASE Application Guidelines
Standard MBASE Application Guidelines
Application-Intensive CBS Guidelines
Application-Intensive CBS Guidelines
Tailoring-Intensive CBS Guidelines
Tailoring-Intensive CBS Guidelines
Assessment-Intensive CBS Guidelines
Assessment-Intensive CBS Guidelines
III III IV
CBS project Type Selection Matrix
CBS
Requirements Volatility
Developer Resources
Domain Adaptability
Growth Rate
Evolution Rate
Developer Autonomy
Support Stability
None VL VH M H VH VH H Application L H M H H H H Tailoring M M M-H L L H M Assessment H L H VL VL VL L
Evaluate with respect to business case and organization attributes
Assessment Intensive CBS
• Assumption:
– COTS packages available to cover most of desired system capabilities identified according to system shared vision among stakeholders.
– A considerable number of COTS package combination needs to be assessed against established evaluation criteria.
Assessment Intensive CBS-Characteristics 1
• The primary effort is focused on identifying collection of candidate COTS products and evaluating a feasible set of products whose existing capabilities directly address a desired operational concept. – About 60-80% of COTS related effort is
assessment effort, and accounts for 14% of overall effort.
Assessment Intensive CBS- Characteristics 2
• Need high flexibility in requirements and design due to COTS uncertainty– COTS capabilities compose requirements– Requirements cannot be fixed (or “hard”)
before final COTS selection– We use Desired Characteristics & Priorities
(DC&P’s) to guide requirements gathering and flexibility
Assessment Intensive CBS- Characteristics 3
• System architecture is focused on a COTS package configuration model, since the class model, object model and interaction model are not available for most COTS products. – Example: Rework effort for CS577 2001-2002
team8’s SSAD
Assessment Intensive CBS- Characteristics 4
• A business case analysis for each COTS solution is highly recommended for the detailed COTS assessment – COCOTS Model for effort and schedule estimation
• Assessment, glue code, tailoring effort tradeoffs
– Return on Investment analysis • Value, risk, cost tradeoffs
– COTS specific risks and mitigation costs• vendor volatility, upgrade lifecycle, etc.
• Example: Dot-Project infeasibility in CS577A 2001 team 8
Identifying Assessment CBS
• You have an assessment intensive CBS when you have the previously stated characteristics.– High requirement volatility, high domain
adaptability, low developer resources…– Business case shows feasibility of focusing
effort on COTS assessment (rather than glue code, tailoring, or build from scratch)
Assessment Intensive CBS Project Guidelines
Assessment-Intensive CBS
Assessment-Intensive CBS
Assess COTS candidates with respects to DC&P’s
Assess COTS candidates with respects to DC&P’s
Develop glue code; Tailor COTS mix to DC&P’s
Develop glue code; Tailor COTS mix to DC&P’s
Integrate, test, transition IOC
Integrate, test, transition IOC
Evolve as tailored COTS mix
Evolve as tailored COTS mix
Follow Tailoring-Intensive COTS Guideline
Single COTS
Yes
No, COTS Mix Solution
Example
• Project profile– USC Collaborative Services System (team 8
CS577 2001-2002)• DC&P’S
– Project management– File management– Online Discussion Board– Group Calendar
Example- COTS Assessment Steps• Plan: set the evaluation effort scope and provide a baseline
for evaluation process. • COTS Identification: search for candidate COTS packages
based on the set of DC&P’s • Establish evaluation criteria based on functional,
performance, architectural, and financial characteristics of DC&P’s
• Conduct multiple cycle of evaluation to collect evaluation data against the defined evaluation criteria.
• Perform Hand-on Experiments on COTS to verify vendor’s claim
• Compare and contrast the evaluation data and make COTS final selection.
Major risks-1
• Many new non-technical activities introduced into software engineering process– Establishing evaluation criteria, – Planning and executing evaluation procedures,– Collecting and analyzing data from evaluation
process,– Making COTS recommendations, which
require training and experience.
Major Risks-2
• Within the COTS evaluation process:– You may miss some possible COTS candidates.
• Stay as broad as possible when doing the initial search for candidates.
– You might not include all key aspects for establishing the evaluation criteria set.
• Judgment of weight distributions require experienced and knowledgeable people
– Introducing new COTS candidates is likely and require re-planning or contingency planning.
• Limits on schedule and budget must be accounted for
Major Risks-3
• You may not get all the features of certain COTS products as advertised by its vendor.– Detailed assessment beyond literature review or
vendor provided documentation should be performed in the form of hands-on experiments.
Example - Comparison of results from initial assessment and detailed assessment
Evaluation Requirement COTS candidate 1-eProject
COTS candidate 2-
Blackboard
COTS candidate 3-
iPlanet collaborative package
Initial Detailed Initial Detailed Initial Detailed
Users can upload/modify/download/delete
authorized files ** ***** ** ** * None
Project management, including planning group tasks, timelines and tracking
** ***** ** *** None None
Message Board ** ***** ** **** None None
User Interface ** ***** ** ** ** *
Admin Interface ** ***** ** **** ** None
Group Calendar ** ***** ** ** ** *
Detailed analysis provides greater assurance of COTS characteristics with respect to vendor documentation (although at significant effort)
Major Risks-4
• The organization may not have the ability or willingness to accept the impact on its operation due to imposed COTS requirements. – Example:
• CS577 2001-2002 team 8: – COTS: Blackboard
– Required operational change: User access mode
Major Risks-5
• Difficulty in finding suitable time for key personal meetings to discuss COTS assessment may incur significant and unpredictable schedule delay– Based on analysis from student project weekly
effort reports, team interaction and client interaction are the top-2 activities for assessment intensive CBS projects.
Tailoring Intensive CBS
• Assumption:– COTS packages have some sort of a
tailoring/development interface using which the COTS package can be adapted to the DC&P’s
– Little or no assessment is required (if required it is already complete) in selecting the COTS package
• CS 577 2001 – 2002 – Team 20 (577 MBASE in BORE)– Team 21 (Quality Management Through BORE)
Tailoring Intensive CBS Characteristic 1
• The primary effort is on tailoring the existing adaptable capabilities of the COTS package to satisfy the majority of a systems core capabilities or operational scenarios.
– CS 577 2001 – 2002 – Team 20 (577 MBASE in BORE)
– Team 21 (Quality Management Through BORE)
Tailoring Intensive CBS Characteristic 2
• Need moderate flexibility in requirements and design due to limitations in adaptability of COTS packages– Capabilities (and adaptability) of the COTS packages
may only partially satisfy the system requirements
– System requirements may need to be adapted to fit within COTS limitations
• Significant source of risk, focus on DC&P’s to manage requirements change
• Example: QA in BORE security levels requirement change
Tailoring Intensive CBS Characteristic 3
• The focus of the system architecture depends upon the type of the COTS package tailoring required in order to satisfy the DC&P’s– GUI Based Tailoring (ex. Spearmint)
• No focus on system architecture
– Parameter Based Tailoring (ex. Windows Media Player)
• Little or no focus on system architecture
– Programmable based Tailoring Interface (ex. Hyperwave)
• Moderate focus on system architecture
Tailoring Intensive CBS – Characteristic 4
• The Tailoring intensive CBS development usually require some amount of time for developer training to get acquainted to the COTS package.– A Learning Curve must be accounted for during
the system development process
Identifying Tailoring CBS
• You have an tailoring intensive CBS when you have the previously stated characteristics.– Business case shows feasibility of buying and
tailoring COTS packages to implement major system capabilities
Tailoring Intensive CBS Project Guide
Tailoring-Intensive CBS
Tailoring-Intensive CBS
Tailor COTS Framework to DC&P’s
Tailor COTS Framework to DC&P’s
Test, Transition,IOC
Test, Transition,IOC
Evolve as Tailoring-Intensive CBS
Evolve as Tailoring-Intensive CBS
Get acquainted with the COTS framework
Get acquainted with the COTS framework
Ensure that the core features to be used, function at an acceptable level
Ensure that the core features to be used, function at an acceptable level
Risk 1
• Version change may result in re-tailoring of COTS product– A constant feedback by the client to the vendor
about the features used and enhancements required may help in ensuring that the features used by the current CBS continue in the future versions of the project
• Ex. “The LOGO element specifies a graphic file to display as a logo. Not supported after Windows Media Player version 6.4.”
Risk 2
• Inadequate vendor support may result in significant project delays– Hiring consultants with experience in using
COTS product (Non-CS577 mitigation) – Ensuring an efficient Phone/Email support at
least during the development phase
Risk 3
• Faulty Vendor claims (COTS “surprises”) may result in Significant Delays
• (Ex) Oracle intermedia claims to search for all unicode characters however the search is not carried as specified
– Pre-testing of core features to be used before beginning of the actual development
Risk 4
• COTS package incompatibilities (integration clash)– In case of multiple COTS packages being used
to implement the DC&P’s there may exist incompatibilities between COTS solutions
• Pre-testing of compatibilities between packages to be used before beginning of the actual development
Risk 5
• Added complexity of un-used COTS capabilities– Adaptable and Tailoring capable COTS
packages usually come with additional complexity and cost that is incurred in the process of making the package tailorable
• Ex Microsoft Office Package
Assessment/Tailoring Intensive CBS Life Cycle and Mbase
Inception
Identify the Type of CBSNarrow the list of Vendors depending on available information
Elaboration
Develop and Test Functional Prototypes to ensure the functioning Core Capabilities
Collect Detailed Information and Evaluate the COTS vendors short-list
Construction 1
Obtain a shared Vision amongst stakeholders.Identify why to go CBS (window of opportunity, independence from evolutionary maintenance, …). Develop and collect Information on available COTS product in the market (RFI). Make a list of products which can satisfy the Core requirements and undergo a primary evaluation them
LCO LCA CCD IOC
Construction 2
Train the Developers to use the COTS product
Implement and test the Core requirements of the system
Implement the low priority requirements of the system
Support personnel must be trained to work with the COTS API Maintaining constant contact with the developer in terms of specifying evolution requirements
Identify the COTS product (s) that shall implement the system
Application Intensive CBS
• Assumptions:– COTS packages are available to satisfy
significantly valuable subsets of system capabilities or project limitations dictate use (e.g. schedule, skill, or complexity factors)
• Buy vs. build cost-benefit ROI
– Integration overhead is minimal with respect to custom development
– Low assessment and tailoring effort required
Assessment Intensive CBS-Characteristics 1
• The primary effort is focused on identifying system components and requirements that may be risky to custom implement and mapping them to a collection of candidate COTS products.– CS577 2001-2002 Team 15 DART’s use of
MySQL and Tomcat
Application Intensive CBS-Characteristics 2
• COTS expected to implement system requirements• Low requirements volatility and adaptability• Significant glue code effort expected• COTS packages used are not specialized to a
particular application domain (generic capabilities)• COTS components dependent on application to
function • Many possible COTS packages may apply• Many evolutionary requirements with respect to
custom components and anticipated COTS features
Identifying Application CBS
• You have an assessment intensive CBS when you have the previously stated characteristics.– Business case shows feasibility of buying
components to implement major system capabilities and focusing effort on COTS integration (glue code)
Application Intensive CBS Project Guidelines
Application-Intensive CBS
Application-Intensive CBS
Tailor COTSTailor COTS
Develop glue code, integrate COTS, custom components
Develop glue code, integrate COTS, custom components
Choose best mix of COTS, custom components
Choose best mix of COTS, custom components
Develop custom components
Develop custom components
Integrate, test, transition IOC
Integrate, test, transition IOC
Assess COTS candidates with respects to DC&P’s
Assess COTS candidates with respects to DC&P’s
Evolve as application-intensive CBS
Evolve as application-intensive CBS
Example
• Project profile– CS577
• Team 15 DART 2001-2002– Tomcat, MySQL
• Team 17 Web-mail 2000-2001– Tomcat
• Team 8 Fulltext Title Search 2000-2001– MySQL, Tomcat, Zope
• Team 9 Student/Staff Directories 2001-2002– Zope
Major risks-1
• Spending too much effort locating appropriate COTS products (too little value for cost)
• Overly optimistic expectation of COTS quality attributes
• COTS package incompatibilities (integration clash)• Added complexity of un-used COTS capabilities• Imposed black box testing of COTS components• Inadequate COTS assessments
Major risks-2
• Overly optimistic COTS package learning curve
• Compromising hard requirements to make use of particular COTS
• COTS “surprises”, capability shortfalls
• Application dependent on COTS vendors (legacy support, upgrades, bugs, etc.)
Comparison of CBS project effort sources
0. 00%
5. 00%
10. 00%
15. 00%
20. 00%
25. 00%
1 2 3 4 5 6 7 8 9 10 11 12
Act i vi t i es
%
AssessmentI ntensi ve CBSs (2)Tai l or i ngI ntensi ve CBSs (3)Appl i cat i onI ntensi ve CBSs (2)
1 Team Interaction 7 Feasibility Rationale Document
2 Client Interaction 8 COTS Final Selection
3 COTS Initial Filtering 9 Control and Monitoring
4 Life Cycle Planning 10 COTS Tailoring
5 Project Web-site 11 Transition and Support Planning
6 Training and Preparation 12 Critical Component Implementation
Source: CS577 2001-2002 Project Effort Data
CBS project Type Comparisons CBS
Requirements Flexibility
Glue Code
Domain Specificity
Custom Development
Number of Components
COTS Requirements
Coverage
Evolution Requirements
Degree None Varies None Usually
highly domain
specific for mission
critical apps (exception is general product or
tools)
All Varies None Generally Very High
Application Low Significant Independent of domain
Development needed to implement
custom capabilities
Many possible
Covers effort intensive or risky subsets
High with respect to custom
components and
anticipated COTS features
Tailoring Change non-critical (esp.
UI) requirements
if needed
Some General to domain and
can be refined to specific
organization or mission
Little Few Mostly covered by adapting existing
capabilities
Limited to what is
tailorable
Assessment Change requirements
if needed
None to very little
Generic to domain or COTS is domain specific;
organization will adapt to
COTS
None One or Very Few
Existing capabilities
should satisfy as possible
Generally Very Low, but
included within
assessed products