Download - iWay Data Profiler
![Page 1: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/1.jpg)
Anand Maheshwari
Product Highlights
5.0 GA
iWay Data Profiler
1
![Page 2: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/2.jpg)
What is iWay Data Profiler
2
iWay Data Profiler provides the ability to look at data quality from a business user perspective with dashboard templates to view, monitor, compare, and report by leveraging the turnkey business intelligence technology through semantic analysis, complex business rules and data drill-down.
![Page 3: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/3.jpg)
Features
1. Ability to profile data against a set of standard (pre-defined) data quality rules and metrics
2. Ability to create custom organization specific data quality indicators3. Advanced comparison of data profiling and quality indicators4. Ability to create and execute data quality projects against varying payloads5. Ability to rollup data with drill through6. Advanced administration capability7. Advanced auditing capability8. Easier migration from earlier versions9. Ability to track background processes for faster troubleshooting and status10. Use of current standards like JQuery for enhanced user experience11. Support for all leading database platforms12. Support for wide variety of OS platforms13. Support for German, French, Spanish and English locales14. Increased performance through several tuning efforts
3
![Page 4: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/4.jpg)
4
Features 4.1 5.0
Create profile with drill-through
Export profile report to: AHTML and PDF format AHTML
XLS ×
Email profile using SSL
Create profile report for: Data Analyses
Business Rules
Primary Key
Foreign Key
Summary
Hierarchical folders ×
Uploading profile in background ×
View process result ×
Rollup profile with drill-through ×
Profile file size 100MB 300MB
1. Ability to profile data against a set of standard (pre-defined) data quality rules and metrics - Comparison
![Page 5: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/5.jpg)
1. Ability to profile data against a set of standard (pre-defined) data quality rules and metrics – New Features
5
• Uploading profile in background
• View result to display uploading progress
• Export profile report in XLS format
![Page 6: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/6.jpg)
6
2. Ability to create custom organization specific data quality indicators – Comparison
Features 4.1 5.0
Create and view DQIs report
Export PDF report
Folder Hierarchical division ×
Comparison Across Expression ×
Visualization for success and failure counts ×
More user interaction using AHTML Format ×
![Page 7: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/7.jpg)
2. Ability to create custom organization specific data quality indicators – New Features
7
Easy to understand the data quality indicators through multiple reports
Flexibility to compare the expressions View Result to display progress for the background processes
![Page 8: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/8.jpg)
2. Ability to create custom organization specific data quality indicators – New Features
8
Easy to understand the data quality indicators through multiple reports
![Page 9: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/9.jpg)
2. Ability to create custom organization specific data quality indicators – New Features
9
Flexibility to compare the expressions
![Page 10: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/10.jpg)
2. Ability to create custom organization specific data quality indicators – New Features
10
View Result to display uploading progress
![Page 11: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/11.jpg)
11
Features 4.1 5.0
Quick compare of profiles/DQIs
Create Dataset & Template
Graphical representation of profile/DQI compare
Default Graph Line 3D Bar
Displaying table report with visualization ×
Hierarchical folders division ×
3. Advanced comparison of data profiling and quality indicators - Comparison
![Page 12: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/12.jpg)
3. Advanced comparison of data profiling and quality indicators – New Features
12
Additional table report and chart type
• Display of both success and failure count and percentages
• Informative table report display with charts
![Page 13: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/13.jpg)
4. Advanced comparison of data profiling and quality indicators – New Features
13
New design of data displayed in dataset table, for Profile & DQI
Modified default report as 3D Bar
![Page 14: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/14.jpg)
4. Ability to create and execute data quality projects against varying payloads – New Features
14
• Data Quality Projects for Profiles/DQIs enables running a DQC plan for generating a Data Profile/DQI and loading into database repository• Create data quality project• Manage data quality project
• Reusability of plan - The same plan can be executed against different inputs• View Result for different level of progress
![Page 15: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/15.jpg)
4. Ability to create and execute data quality projects against varying payloads – New Features
15
Create project – option to upload Profiling plans and DQI plans
Plan can be with drill-through and also with Rollups
![Page 16: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/16.jpg)
4. Ability to create and execute data quality projects against varying payloads – New Features
16
Manage project
![Page 17: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/17.jpg)
4. Ability to create and execute data quality projects against varying payloads – New Features
17
Reusability of plan : The same plan can be executed against different inputs
![Page 18: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/18.jpg)
4. Ability to create and execute data quality projects against varying payloads – New Features
18
Note:
Plan is supported with TXT and CSV Input, Other Input in DQC like XML, Database is not supported in the current version
Output can only be profile steps (or DQI steps depending on the type of DQ project)
The following input settings of the input file can be modified without DQC GUI:• Number Of Lines In Header • Field Separator • Line Separator • String Qualifier • Encoding
![Page 19: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/19.jpg)
4. Ability to create and execute data quality projects against varying payloads – New Features
19
View Result for different level of progress
![Page 20: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/20.jpg)
5. Ability to rollup data with drill through – New Features
20
Roll up is a part of iWay Data Profiler that enables a user to profile view a specific subset of a data.
Rollups for: Data Analysis Primary Key Business Rules.
![Page 21: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/21.jpg)
5. Ability to rollup data with drill through – New Features
21
Enabled drill-through for the subset of data
![Page 22: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/22.jpg)
6. Advanced administration capability – Comparison
22
Features 4.1 5.0
Create Roles, Groups and Users
LDAP User Authentication and LDAP Role Authorization
Option to create, delete and manage folders ×
Group and Role details can be viewed while creating a user ×
Groups have access to one or more Virtual Folders ×
Base Role : Admin, Power, Technical and Analytical with different privileges
×
iSM Tracking privilege ×
DQ Project privilege ×
Rollup privilege ×
![Page 23: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/23.jpg)
6. Advanced administration capability – New Features
23
• Hierarchical folder structure to manage profiles, DQIs, datasets, templates Grouping related things together based on the user choice Sharing and easy to compare Created a group called “Admin” and linked all folders to it Folder structure for a user
o Private o My Groupo Customo Public
• Created default base roles: admin, power, technical and analytical with varying authorizations
![Page 24: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/24.jpg)
6. Advanced administration capability – New Features
24
Manage Folders
![Page 25: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/25.jpg)
6. Advanced administration capability – New Features
25
Manage Roles
![Page 26: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/26.jpg)
6. Advanced administration capability – New Features
26
Manage Groups
![Page 27: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/27.jpg)
6. Advanced administration capability – New Features
27
Manage Users
![Page 28: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/28.jpg)
7. Advanced auditing capability – New Features
28
Audit (Tracking user name, IP address, login time and logout time) with different databases
• Enhanced user security
• Filter audit information
• Track log in and log out time in iDP with IP Address
• Track invalid login & iDP server manager login
![Page 29: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/29.jpg)
7. Advanced auditing capability – New Features
29
Audit information for a user
![Page 30: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/30.jpg)
7. Advanced auditing capability – New Features
30
Filter audit information
![Page 31: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/31.jpg)
8. Easier migration from earlier versions – New Features
31
IncludesUpgrade script is for migrating from the older version to newer version
4.1 5.0Available for:
MySql MSSQL Oracle
There is no loss of data exists in database “idp_data”
Does not includeSynonym creationDrill-through adapter creation in WebFOCUS
![Page 32: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/32.jpg)
9. Ability to track background processes for faster troubleshooting and status – New Feature
32
Interface for the user to monitor the back-end process Multilingual support for view process result – Available for Profile, DQI, DQ
Project and Custom Reports
![Page 33: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/33.jpg)
iWay Data Profiler : Architecture
33
![Page 34: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/34.jpg)
iWay Data Profiler - Component Architecture
34
![Page 35: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/35.jpg)
Benefits
Web enabled thereby single deployment and access anywhere Aids in the identification of data quality issues Expertise and support of a mature business intelligence product –
WebFOCUS Data quality comparison over time Documents and audits quality-control processes for review Streamlines offline maintenance activities to ensure optimum performance Share the reports using a secured connection Mobile support
35
![Page 36: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/36.jpg)
Anand Maheshwari
Thank You !
5.0 GA
iWay Data Profiler
36
![Page 37: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/37.jpg)
Appendix – Additional Features
37
![Page 38: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/38.jpg)
Multilingual support
38
iWay Data Profiler supports multi-languages English German Spanish French
Note: ISM Tracking Edit page language will change corresponding to the language choose in ISM configured not based on the IDP chosen language.
![Page 39: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/39.jpg)
Performance tuning
39
JAVA code has been restructured for better performance
• Complete recoding using XPATH and transaction based multiple coding being undertaken as processing of complicated Rollups found to crash the JVM
• The maximum size of the Profile XML file, which can be uploaded into DP, has been increased from 1MB to 300 MB
![Page 40: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/40.jpg)
Revamped iWay Data Profiler with JQuery
40
JQuery Functionalities• Enhance look and feel• Styling• Cross browser compatibility• Support all mobile devices• Grouping related things together• Sharing• Easy to compare• Fast loading• Alerts and alignment • Expand All/ Collapse All functionality• More/ Less functionality• Pagination• Homepage carousel – to show case different graphs of iDP• Landing page accordion layout – to display different iDP modules
![Page 41: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/41.jpg)
Modified both Windows and Linux installer
41
Updated both Windows and Linux Installer scripts for DQ Project to get the Drill-through DB credential from the installer
Installer is available in English, German, French, Spanish Same Linux build is supported for AIX platform
![Page 42: iWay Data Profiler](https://reader036.vdocuments.us/reader036/viewer/2022081503/56815841550346895dc59888/html5/thumbnails/42.jpg)
Backward Compatibility with iWay Data Quality Center
42
Modified iDP for Profiling DQI
Support across versions DQC 7.0.1 & higher Input data Reports