applying concepts of software reuse to the implementation of data warehouse etl systems jiayang zhou...
TRANSCRIPT
Applying Concepts of Software Reuse to the Implementation of Data Warehouse ETL Systems
Jiayang ZhouNovember 2001
Data Warehouse Loader sd&m
Applying concepts of software reuse to Data Warehouse Loader
Fundamental of software reuse
Description of Data Warehouse Loader
Implementation of Data Warehouse Loader
Reuse Analysis of Data warehouse Loader
Summary
Data Warehouse Loader sd&m
Applying concepts of software reuse to Data Warehouse Loader
Fundamental of software reuse
Description of Data Warehouse Loader
Implementation of Data Warehouse Loader
Reuse Analysis of Data warehouse Loader
Summary
Data Warehouse Loader sd&m
Definition of Software reuse
Software reuse is a process of implementing or updating
software systems using existing software assets.
software components objects software requirements analysis
design model domain architecture database schema
codedocumentation manual standardstest scenario
plan
Data Warehouse Loader sd&m
Importance of Software reuse
Increases productivity
Enhances quality
Saves cost
Reduces software development schedules
Reduces maintenance
Enhances standardization
Increases portability
Data Warehouse Loader sd&m
Fundamental of software reuse
Description of Data Warehouse Loader
Implementation of Data Warehouse Loader
Reuse Analysis of Data warehouse Loader
Summary
Applying concepts of software reuse to Data Warehouse Loader
Data Warehouse Loader sd&m
Architecture of data warehouse application
Data Warehouse Loader sd&m
Features of data warehouse
A data warehouse is a
Subject-oriented
Integrated
Time-variant
Non-volatile
collection of data for management’s decision making process.
The role of Data Warehouse Loader
Extraction: extract raw data from different source system
Transformation: transform data in order to be fit into the architecture of data warehouse
Loading: import data into data warehouse
Data Warehouse Loader sd&m
The requirements of Data Warehouse Loader
Adaptability to any data source systems
Adaptability to any target data warehouse
High operation speed
Ability to deal with large amount of data
Flexibility for data transformation
Portability
Data Warehouse Loader sd&m
The reuse consideration of Data Warehouse Loader
Data Warehouse Loader sd&m
Fundamental of software reuse
Description of Data Warehouse Loader
Implementation of Data Warehouse Loader
Reuse Analysis of Data warehouse Loader
Summary
Applying concepts of software reuse to Data Warehouse Loader
Data Warehouse Loader sd&m
Architecture of Data Warehouse Loader
Data Warehouse Loader sd&m
Loader-engine
Data Warehouse Loader sd&m
Loader-interface
• Extraction-interface: access to data source
• Transformation-interface: different transformation scheme
• Database-interface: access to data warehouse for updating and retrieving
• Record-interface: construct an object structure for each database record
Data Warehouse Loader sd&m
Data Warehouse Loader sd&m
Format of intermediate file
INFO header
FORMAT header
DATA body
Data Warehouse Loader sd&m
Fundamental of software reuse
Description of Data Warehouse Loader
Implementation of Data Warehouse Loader
Reuse Analysis of Data warehouse Loader
Summary
Applying concepts of software reuse to Data Warehouse Loader
Data Warehouse Loader sd&m
Reuse development of Data Warehouse Loader
Thorough examination of application requirements
Good documentation
Choosing of Java as programming language
Concrete separation of specification from a particular application
Data Warehouse Loader sd&m
Advantages of workflow architecture
Easy maintenance
Module reuse
Temporally decoupling
Manageable and controllable programming
Re-composition of operation sequence
Data Warehouse Loader sd&m
Re-composition of operation sequence
Data Warehouse Loader sd&m
Appling concepts of software reuse
Code reuse
Adaptability
Modularity
Interface
Data Warehouse Loader sd&m
Reuse architecture of Data Warehouse Loader
Data Warehouse Loader sd&m
Package overview of Data Warehouse Loader
Data Warehouse Loader sd&m
Fundamental of software reuse
Description of Data Warehouse Loader
Implementation of Data Warehouse Loader
Reuse Analysis of Data warehouse Loader
Summary
Applying concepts of software reuse to Data Warehouse Loader
Data Warehouse Loader sd&m
START-MDB project
Data Warehouse Loader sd&m
Java and C
C
• fast
• migration between different systems
• difficulty for flexibility
Java
• slow and large RAM consumption
• portability and platform-independence
• easy for flexibility
• object-oriented feature
