information builders enterprise information management ... · iway data integration solution open...
TRANSCRIPT
Transforming data into business value
1
Information Builders Enterprise Information Management Solution
Fateh NAILIEnterprise Solutions Manager
Agenda
Introduction
Information Builders Solutions
Data Integration
Data Quality
Master Data Management
Hadoop and Big Data
Roadmap
Summary
Introduction
Enterprise Information Management
We do Enterprise Information Management!
Information Builders helps align, link and leverage people, processes and technology around information assets across the enterprise.
What do we do?
A complete platform
Integration
Write-Back
Data Discovery Reporting Dashboards
High-PerformanceData Store
Data Quality Data Governance
Master DataManagement
Batch ETL
Integrity
Intelligence
Location Analytics
In-DocumentAnalytics
Casting and Archiving
SearchPredictive Analytics
Sentiment and Word Analytics
Performance Management
Real-Time ESB Hadoop-Based
Natural Language/Narrative BI
ApplicationsLegacy Systems Relational/Cubes Big Data Columnar/In Memory Unstructured Social Media Web Services Trading PartnersIoT
BI Portal Embedded InfoApps™ Mobile
How do we do it?
Well-defined implementation approach
Data
Governance
Share
CommunicateAnalyse
PropagateData Distribution
BuildMatch
MergeData Mastering
ImproveStandardise
EnrichData Quality
GetConnect
OrchestrateData Integration
KnowExplore
ProfileData Analysis
Manage
Control
Data Integration
iWay Data Integration SolutionIntroduction
Unique and powerful Integration Platform Real time and Batch Processing Compose, Orchestrate and Automate
complex integration processes Drag and drop interface for pre-built
components Expose processes as reusable
Services (Web Service, REST …) 300+ Pre-Built Integration Adapters
Bi-directional Data, Application, Industry, etc Use any messaging infrastructure
iWay Data Integration Solution
Open standard-based solution
Backbone for all data processingData Quality, Master Data Management, Data Governance, WebFOCUS can all be stacked on top of this real-time data engine for complete data access.
Native XML and JSON processing Automatically transform A2A and B2B messages (IoT)
Complete end-to-end securityauthentication, authorization, encryption
Full API supportGenerate and consume API/REST
Cloud and on-premise deployment
Key Features
iWay Service Manager
iWayRepositories
iWay Web Console
IWay BAM
IWay Sentinel
iWay Service
Designer
IWay Service
Transformer
iWay Application
Explorer
ApplicationServer
BPMApplication
BI Application
Web ServiceApplication
SWIFT Adapter
MySAPAdapter
OracleAdapter
….
iWay Process Flows
Event DrivenReal Time Processing
Web Service
iWay Adapter Framework
CICSAdapter
Mainframe Data, Applications and
Transactions
Applications,ERP, CRM, …
Files, Excel CSV, XML
SWIFT, HIPAA, EDI Formats
Databases, Data Warehouse,
Design Time
Operational Time Run Time
iWay Integration ToolsiWay Operational Tools
Routing Transform Security XA
iWay Data Integration SolutionTools and components
Data Quality
DQ Rules DQ MonitoringDQ Profiling
Omni-Gen DQ EditionIntroduction
Standalone solution designed to optimize the integrity of informationthroughout the enterprise
Packaged with comprehensive,fully customizable business rulesand a portfolio of libraries
Collaboration
Well defined implementation approach
Remediation
Data Quality WorkbenchProfile the quality of the data
Data profiling shows the presence oferrors with Metrics Information
Profiling Features include Patterns/Masking Custom Business Rules Extremes Quantities Frequency Analysis Foreign Key Analysis Localization Charting Grouping/Aggregate Drilldown/Interactive Displays
Most organizations say data quality issues are detected when reported by employees, customers or prospects. Fewer than 50% of companies conduct proactive data audits to discover data quality issues.
Data Quality RulesImplement Data Quality rules
Data Cleansing and Standardization
Data Enrichment
Data Scoring
Internal/external reference data
Matching / Merging Probabilistic and Deterministic Levenshtein distance Golden record
Id column for NamesAlter Format
Remove titlesStrip Titles
Parse NamesGuess Name Surname
Prepare out valuesColumn Assigner
Remove columnsAlter Format
𝒚=𝒇 𝒙
“John”“Smith”
Web-based Business tool
Allow organizations to monitor and tracktheir data overtime and map them to thebusiness objectives and risks
Defined by user and relevant to data context
DQ Key Performance Indicators
Complex computation Dimensions and Measures
Data Quality MonitoringAnalyze data overtime
Operational Dashboard for Data Stewardsand Data Supervisors
Provide an ability to correct data
Cleansing, Matching, Merging, Unlinking…
Improved reporting, filtering and searches
Track data quality issues in real time
End-to-end audit trail of all data changes
Workflow interface
Data Quality RemediationManual correction
Data Quality Firewall
Avoid dissemination at root (Upstream vs Downstream correction)
Extend the reach of your Data Quality Initiative
Create a firewall protecting your systems from bad data
Extending Data Quality
Master Data Management
Why Master data Management?
Many silos of data often out-of-sync
Poor data quality affecting BI and reporting
Integration complexity
Point-to-point integration is costly (interfaces, “language”)
Inevitable manual process
Error-prone and inconsistency
No way to audit
No way to rollback changes
Different people involved(Updates “interpreted” by systems experts)
Time and resource consuming
The road to Master Data Management
What are the benefits?The road to Master Data Management
Consistent and comprehensives information across the enterpriseKnow your customer, Single view of citizen …
Support data governance and compliance requirements between cross-functional teams
Consolidate and rationalize information from different systems across the enterprise to improve transactional and analytical processes
Enforces Data best practicesData modeling, Quality, Policies, Governance, Hierarchical and Standardize data
Data of the highest quality to feed into their operational and analytical systems
Omni-Gen MDM EditionProduct View
Omni-Gen
Profiling Integration
Integration Edition
Cleansing
Data Quality Edition
Mastering
Master Data Management Edition
Omni-Gen MDM Edition
Top-down approach centered on business problems
Collaboration between the business and IT
Simplifies data integration and mapping of source systems
Pre-packaged mapping documents
Automates key steps in the MDM process
From data integration to Data Quality processing
Facilitates the implementation of mastering templates and rules
Advanced business user interfaces
Master view, source and instance records for full resolution and 360 °view
Introduction
Omni-Gen MDM EditionOmni Designer
Model driven and multi-domain developmentframework (Eclipse based)
Generate the various componentspart of your MDM hub
Data Model configuration
Data Quality setting
Reference Data setting
Workflow customization editor
Omni-Gen MDM Edition360 Viewer
360⁰ view of Data Subjects.
Centralised view of all attributes belongingto a mastered record.
View all instance informationand full history.
LifecycleOmni-Gen MDM Edition
Cloud-based systems
Legacy assets
ERP applications
Business partners
Cloud-based systems
Downstream applications
Data warehouses
Business partners
Hadoop and Big Data
iWay Big Data Integrator
28
100% Run “in” Hadoop architecture
Simplifiedinterface
Native Hadoopscript generation
Process mgmt. & governance Simplified easy-to-use interface
to integrate in Hadoop
Marshals Hadoop resources and standards
Takes advantage of performance and resource negotiation
Includes sophisticated process management and governance
Sqo
op
, Flum
e…A
vro, JSO
N…
Traditional applications and data stores
iWay Big Data Integrator
Simplified, modern, native Hadoop integration
Big Data Hadoop
Any distribution, Any data
1. Data Ingestion – Enterprise Data Hub2. ETL / ELT3. Predictive Analytics - RStat
4. Business Intelligence - WebFocus5. Low-cost storage of large data volumes
iWay Big Data Integrator
We are now promoting iWay Big Data Integrator Developer’s Edition
6-month limited non-production version
Designed for developers who are looking to test the features of Hadoop and Spark, and accelerate value for big data projects
Explore how you can simplify and automate Hadoop data management through a visual UI
iWay Big Data Integrator Developer’s Edition
Roadmap
iWay Integration Platform 8.0
Simplicity, Usability & Accessibility at all levels
Optimized the automated deployment and maintenance
All-in-one Application Projects supporting Team Work
Micro-services and API Management platform
Blockchain support
Product direction
iWay Integration Platform 8.0Simple and Streamlined Design Time
Modern Look & Feel with updated and customizable development environment
Rich Object Pallet enables quick access to components
Intuitive action based approach
“Just tell us what you want to do”
iWay Integration Platform 8.0
Improved native JSON support Provide added flexibility and extend its reach into the light-weight
communication format
Updated Database services Provide robust data access for batch processing as well as traditional
transactional data access
Ability to run scripts as part of the process flow Extend the flexibility and enable user to reuse the existing components
across various projects
Multitenancy support with remote access to externalized resources Extend flexibility for a service oriented architecture
Runtime Updates
iWay Integration Platform 8.0
Application is broken into smaller independent components enabling
Scalability on component level, modular approach
Increased Productivity utilizing agile development
Resilience by separation of runtimes
Micro-services
System A
System B
System C
Service A
Service B
Service C
REST API
REST API
REST API
API GATEWAY
Consumer WebApp
REST API
WebUser
App
iWay Integration Platform 8.0
Construct APIs based on RAML open standard (Restful language)
A single point to access services (or micro-services)
Secure API access including Authentication and Authorization based on OAuth, IP lists and others
Enforce to the consumers SLAs on exposed APIs
Provide lifecycle management and versioning of the APIs
Provide monitoring and metrics such as usage resources, connections …
Enables Micro-services architecture
API Management and key features
iWay Integration Platform 8.0
Intuitive user interaction for API Design
API Management and key features
iWay Integration Platform 8.0
iSM offers integrated support for Blockchain (eg. Hyperledger Fabric)
iSM serves as a client supporting post & query operations
Blockchain Support
Peer
Peer
Peer
Ledger
Ledger
iSM
New transaction
Ledger
1. Send proposal to multiple peers
2. Peers run chaincode and validate
3. Chaincode results send to Orderer
4. Orderer Updates Ledgers
Peer
Peer
PeerLed
ger
Ledger
iSM
ResponseGood/Bad
Ledger
Fabric
iSMPeer
Peer
Peer
Ledger
LedgerLed
ger
Orderer
iWay Integration Platform 8.0Release Roadmap 2017
Release 7.0 SP7• Service Pack Release• Customer Issue resolutions• FTP Updates• AVRO Enhancements• MQ/JMS Dynamic Queues• OAuth Updates (support 2.0)
Q1 Q2 Q3 Q4
Release 8.0
• New L&F Intuitive Design Environment• Multitenancy support (Component Library)
• New process flow palette & new objects• Action/Verb approach for process configuration
• API Management via RAML• Native JSON support and new services
• Blockchain support• Python/JS support as part of process flow
• Support for Tradacoms EDI standard
Release 8.0 SP1
• View of the payload across process objects
• Basic and Advance user configuration options
• Extend API Management Support including Swagger• Inherited runtime configurations
• Self contained application projects
• SWIFT 2017 support
• HIPAA Updates
Omni-Gen IntegrityRelease Roadmap 2017
Rel 3.0 (MDM Edition)• Performance – Bulk processing• New Management Console
• Simplified configuration• End-to-End monitoring
Rel 3.1 (MDM & DQ Editions)• Data Quality Workbench (DQW)
• Nontechnical user interface• Browser-based profiling & cleansing
• Data Quality Monitor (DQM)• Data Quality Insight• Business Impact
Rel 3.1.1 (MDM & DQ Editions)• DQW enhancements• Data Governance enhancements• Healthcare vertical deployments
Rel 3.2 (MDM & DQ Editions)• DQW enhancements• Single Sign On support• Improved management tools• Data Governance enhancements
Q1 Q2 Q3 Q4
Omni ConsoleSystem Administration and Management
New Web-based Monitoring andManagement console
Simplify deployment and overall operations: Provide a centralized location for
starting and stopping services. Enable generation of sample data
for testing purposes. Provide detailed monitoring of all
data processes with KPIs. Enable rapid issue isolation and
resolution.
Data Quality WorkbenchData Profiling and Data Cleansing
Web-based Data Profiling and Cleansing
Business User Friendly Interface
Profile complete Subject Areas suchas Patient, Customer, Product …
Compare Data Quality Improvements
Multi Functional Data Grid for easyData Discovery
Create Data Quality Rules inSimple Business Friendly Language
Data Quality MonitorData Insights & Business Impact
Allow organizations to monitor and tracktheir data overtime and map them to thebusiness objectives and risks
Web-based Business tool
Provide out of box dashboard for KeyPerformance Indicators (KPI) includingdrill downs …
Dimensions and Measures configuration Dollars associated to Data Quality
Summary
Data Governance It’s a process and it’s iterative
Enable the process via technology
Start with Data Integration and grow into MDM
Understand that requirements will change overtime
Know that Information Builders can help you
Summary