edws production - vita...enterprise performance for the edw, etl, relational data marts, metadata,...

1
Oracle Audit Vault and Database Firewall SOW p321/p322 Enterprise Performance for the EDW, ETL, relational data marts, metadata, and document management ETL Process (SOW p15) Data acquisition (real-time and scheduled), business and quality rules, and data enrichment. Data Monitoring Change Management Application Lifecycle Management Source Code and Version Control Change Control and Help Desk Reporting System Monitoring Documentation Management and Metadata Tagging and Repository SOW p126/p174 Auditing Quality Assurance and Testing Robert Kowalke ~ Enterprise Architecture ~ [email protected] Relationship Management & Governance (RM&G) @ Virginia Information Technologies Agency (VITA) Commonwealth Enterprise Solutions Center (CESC) PURPOSE: To identify the EDWS from an Optum SOW perspective. In computing, a data warehouse (DW or DWH), also known as an enterprise data warehouse (EDW), is a system used for reporting and data analysis, and is considered a core component of business intelligence. DWs are central repositories of integrated data from one or more disparate sources. They store current and historical data in one single place and are used for creating analytical reports for knowledge workers throughout the enterprise. The data stored in the warehouse is uploaded from the operational systems (such as PBMS, OPSS, PEMS, etc.). The data may pass through an operational data store and may require data cleansing for additional operations to ensure data quality before it is used in the DW for reporting. The EDW maintains a copy of information from the source transaction systems. This architectural complexity provides the opportunity to: 1) Integrate data from multiple sources into a single database and data model. 2) Mitigate the problem of database isolation level lock contention in transaction processing systems caused by attempts to run large, long running, analysis queries in transaction processing databases. 3) Maintain data history, even if the source transaction systems do not. 4) Integrate data from multiple source systems, enabling a central view across the enterprise. 5) Improve data quality, by providing consistent codes and descriptions, flagging or even fixing bad data. 6) Present the organization's information consistently. 7) Provide a single common data model for all data of interest regardless of the data's source. 8) Restructure the data so that it makes sense to the business users. 9) Restructure the data so that it delivers excellent query performance, even for complex analytic queries, without impacting the operational systems. https://en.wikipedia.org/wiki/Data_warehouse Department of Medical Assistance Services (DMAS) – Medicaid Enterprise System (MES) Environment EDWS Solutions Stack DRAFT DOCUMENT REV 2.0 – 031017 VITA Discussion Document Data integration. Data enrichment. Clean, Apply business rules, Data integrity checks, Aggregate or disaggregate, Standardize, etc. Extract Data Data Sources Transform Data Load Data Metadata Extract – Transform – Load (ETL) Data Warehouse Data Acquisition. Data Discovery. Extract data from one or more DMAS sources… Validate data. Raw data loaded weekly. Data warehouse/vault. Single Source of Truth for Analytics. Central repository for all DMAS data. Loads data into specified target systems and/or file formats. Enriched data loaded monthly. Data Storage / Repository Processing Tier Ingestion Tier Real-time / Batch / Scheduled Distillation Tier Data Marts Sandboxes Insights / Consumption Tier Data Analysis / Action Tier / Publish Layer Business Intelligence; Visualization Tools; Reporting (MARS, SURS, FADS) User Access Layer | User Presentation | Data Views Data delivery. Analytics Tableau Dashboards Excel Dashboards SAS Accelerators Scoring and Analytics SOW p337 Committee for Quality Assurance (CQA) Fraud and Abuse Detection System (FADS) Management and Administrative Reporting System (MARS) Surveillance and Utilization Review (SURS) Teradata Active System Management (TASM) Optum™ Triple Aim Analytic Services (OTAAS) FADS and MARS data accessed via Cognos for analysis Access layer of the data warehouse environment used to get data out to the users. It is a subset of the data warehouse and is usually oriented to a specific business line, group, or team within DMAS such as Finance. While transactional databases are designed to be updated, data warehouses or marts are read only. Document Management Data Acquisition Business Architecture Analytic Foundation Suite of Tools SOW p107 Data Enrichment SOW p16/255/299/300/ 372 Replicating databases for developers, testing, etc. Executive Users Business Users Data Analysts Virtualization Application Servers Monitoring SDLC Advanced Security Rules Engine Staging DB Rule Engine SOW p165 of 521 Teradata Cloud Data Warehouse Appliance 2800 56-core / two- node Big Data Analytics Data Aggregation and Consolidation SOW p189 of 521 Informatica for Data Acquisition. SOW p15 of 521 TASM Ad hoc BI Extract Capability SOW p205 Scheduled BI Extracts Teradata database includes support for running SQL in parallel. SOW p240 Geospatial Analytics SQL Multimedia and Application Packages (SQL/MM) SOW p329 Data Loading Utility that uses Teradata SQL SOW p517 Metadata Manager SOW p108/p370/ p373/p375/p376 Enterprise Metadata Repository ETL & Technical Metadata SOW p374 Data Quality Metadata SOW p374 Operational Metadata SOW p374 Security Metadata SOW p374 End-user Metadata SOW p374 Business Metadata SOW p375 Data Feeds EDWS Production SOW p230 EBM Connect SOW p240 Symmetry Episode Treatment Group (ETG) Risk Assessment and Predictive Modeling (ERG) Quality Measurement (EBM Connect)

Upload: others

Post on 28-Sep-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: EDWS Production - VITA...Enterprise Performance for the EDW, ETL, relational data marts, metadata, and document management ETL Process (SOW p15) Data acquisition (real-time and scheduled),

Oracle Audit Vault and Database FirewallSOW p321/p322

Enterprise Performance for the EDW, ETL, relational data marts, metadata, and document management

ETL Process (SOW p15)Data acquisition (real-time and scheduled), business and quality rules, and data enrichment.

Data Monitoring

Change ManagementApplication Lifecycle Management

Source Code and Version ControlChange Control and Help Desk Reporting

System Monitoring

Documentation Management andMetadata Tagging and Repository

SOW p126/p174

Auditing

Quality Assurance and Testing

Robert Kowalke ~ Enterprise Architecture ~ [email protected] Management & Governance (RM&G) @ Virginia Information Technologies Agency (VITA)

Commonwealth Enterprise Solutions Center (CESC)

PURPOSE: To identify the EDWS from an Optum SOW perspective. In computing, a data warehouse (DW or DWH), also known as an enterprise data warehouse (EDW), is a system used for reporting and data analysis, and is considered a core component of business intelligence. DWs are central repositories of integrated data from one or more disparate sources. They store current and historical data in one single place and are used for creating analytical reports for knowledge workers throughout the enterprise. The data stored in the warehouse is uploaded from the operational systems (such as PBMS, OPSS, PEMS, etc.). The data may pass through an operational data store and may require data cleansing for additional operations to ensure data quality before it is used in the DW for reporting. The EDW maintains a copy of information from the source transaction systems. This architectural complexity provides the opportunity to: 1) Integrate data from multiple sources into a single database and data model. 2) Mitigate the problem of database isolation level lock contention in transaction processing systems caused by attempts to run large, long running, analysis queries in transaction processing databases. 3) Maintain data history, even if the source transaction systems do not. 4) Integrate data from multiple source systems, enabling a central view across the enterprise. 5) Improve data quality, by providing consistent codes and descriptions, flagging or even fixing bad data. 6) Present the organization's information consistently. 7) Provide a single common data model for all data of interest regardless of the data's source. 8) Restructure the data so that it makes sense to the business users. 9) Restructure the data so that itdelivers excellent query performance, even for complex analytic queries, without impacting the operational systems. https://en.wikipedia.org/wiki/Data_warehouse

Department of Medical Assistance Services (DMAS) – Medicaid Enterprise

System (MES) Environment EDWS Solutions Stack

DRAFT DOCUMENTREV 2.0 – 031017

VITA Discussion Document

Data integration. Data enrichment. Clean, Apply business rules, Data integrity checks, Aggregate or disaggregate, Standardize, etc.

Extract Data

Data Sources

Transform Data Load Data

MetadataExtract – Transform – Load (ETL)

Data Warehouse

Data Acquisition. Data Discovery. Extract data from one or more DMAS sources… Validate data. Raw data loaded weekly.

Data warehouse/vault. Single Source of Truth for Analytics.Central repository for all DMAS data.Loads data into specified target systems and/or file formats.Enriched data loaded monthly.

Data Storage / RepositoryProcessing Tier

Ingestion TierReal-time / Batch / Scheduled

Distillation Tier

Data Marts

Sandboxes

Insights / Consumption Tier Data Analysis / Action Tier / Publish Layer

Business Intelligence; Visualization Tools; Reporting (MARS, SURS, FADS)

User Access Layer | User Presentation | Data Views

Data delivery.

Analytics

Tableau Dashboards

Excel Dashboards

SAS AcceleratorsScoring and Analytics SOW

p337

Committee for Quality Assurance (CQA)Fraud and Abuse Detection System (FADS)Management and Administrative Reporting System (MARS) Surveillance and Utilization Review (SURS)Teradata Active System Management (TASM)Optum™ Triple Aim Analytic Services (OTAAS)

FADS and MARS data accessed via

Cognos for analysis

Access layer of the data warehouse environment used to get data out to the users. It is a subset of the data warehouse and is usually oriented to a specific business line, group, or team within DMAS such as Finance.

While transactional databases are designed to be updated, data warehouses or marts are read only.

Document Management

Data Acquisition

Business Architecture Analytic Foundation Suite

of ToolsSOW p107

Data EnrichmentSOW p16/255/299/300/

372

Replicating databases for developers, testing, etc.

Executive Users

Business Users

Data Analysts

Virtualization

Application Servers

Monitoring

SDLC

Advanced Security

Rules Engine

Staging DB

Rule EngineSOW p165 of 521

Teradata Cloud Data Warehouse Appliance 2800 56-core / two-

node Big Data AnalyticsData Aggregation and

Consolidation SOW p189 of 521

Informatica for Data Acquisition. SOW p15 of 521

TASM

Ad hoc BI Extract Capability SOW p205

Scheduled BI Extracts

Teradata database includes support for running SQL in

parallel. SOW p240

Geospatial AnalyticsSQL Multimedia and Application Packages (SQL/MM) SOW p329

Data Loading Utility that uses Teradata SQL

SOW p517

Metadata ManagerSOW p108/p370/p373/p375/p376

Enterprise Metadata Repository

ETL & Technical Metadata

SOW p374

Data Quality

MetadataSOW p374

Operational Metadata

SOW p374

Security Metadata

SOW p374

End-user Metadata

SOW p374

Business Metadata

SOW p375

Data Feeds

EDWS Production

SOW p230EBM Connect

SOW p240

Symmetry

Episode Treatment Group (ETG)Risk Assessment and Predictive Modeling (ERG)Quality Measurement (EBM Connect)