processing*large*files*with* …...copyright©*2014...
TRANSCRIPT
Processing Large Files with Oracle Managed File Transfer Dave Berry -‐ Oracle Senior Manager – Integra?on Products Pandurang Ranjalkar -‐ Larsen & Toubro Infotech Limited Oracle Fusion Middleware Prac?ce Head
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
CON 7826
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement The following is intended to outline our general product direc?on. It is intended for informa?on purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or func?onality, and should not be relied upon in making purchasing decisions. The development, release, and ?ming of any features or func?onality described for Oracle’s products remains at the sole discre?on of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Oracle Managed File Transfer Overview
Use Cases for Large File
Partner Introduc?on
Partner Solu?on and Demo
Summary and Q & A
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Fusion Middleware The Cloud Pla6orm for Digital Business
Cloud On-‐Premise
DIGITAL ENGAGEMENT
APPLICATION & DATA INTEGRATION IDEN
TITY
MAN
AGEM
ENT
SYSTEM
S MAN
AGEM
ENT
APPLICATION INFRASTRUCTURE & TOOLS
BUSINESS PROCESS MANAGEMENT BUSINESS ANALYTICS CONTENT & COLLABORATION
Web Mobile Social Internet of Things
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Oracle Managed File Transfer Overview
Use Cases for Large File
Partner Introduc?on
Partner Solu?on and Demo
Summary and Q & A
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Uncontrollable problem of file exchanges
Ø ProliferaAon of FTP servers & clients Ø No central control on exchanges of sensi?ve informa?on
Ø No global visibility Ø Not integrated with enterprise security standards: FTP servers are rarely integrated with directories
Ø Single points of failure: FTP servers rarely offer HA capabili?es
Lack of control, visibility, security & reliability
Product specifica?ons
Manufacturing Design partners
Enterprise
Tax informa?on
Administra?on Legal, PR
Contracts, Media
Employee Data
Service Providers
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Managed File Transfer (MFT)
• Automate & schedule file transfers • Highly-‐Available, clusterable solu?on • Complete visibility & auditability • Support for very large files (500+GB) & pass by reference
• Secure: SSH,SSL, PGP Encryp?on • Integrated with SOA Suite, B2B, ODI & IdM • Works with std FTP clients (mul?threaded) • Extensible: compression, custom ac?ons
send receive
DOCUMENT STORE
Audi?ng Monitoring
Temporary store
enterprise
internet
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MFT Func?onal Use Case Paeerns Sources
Fan Out Direct
Transfers Targets Local FTP
Remote FTP
Local File
SOAP
Pass-‐By-‐Reference
Hybrid Integra<on Cloud
Local FTP
Remote FTP
Local File
SOAP
* includes sFTP and FTPs
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MFT Simple Design of Transfers Browser Based Environment • Configure Transfers
– Reusable Sources and Targets – Schedules, filters, compression, encryp?on – Secure endpoint file access
• Endpoint Types – Event: Embedded FTP, SOAP, SOA, B2B, OSB, ODI – Polling: File and remote FTP adapters
• Extreme Ease of Use – Drag and drop, no developer required – Transfer rename, duplicate, filtered transfer list – Deployment details, export, import – Reusable custom ac?ons: no?fica?ons, file conversion …
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MFT – Global Visibility Metrics, Diagnos<cs & Extensive Repor<ng • Metrics Dashboards
– Fine and course grained metrics – Metrics for every ar?fact: Source, Target and Transfer
• Totals, file size, speed, elapsed ?mes, failure ra?o
• DiagnosAcs & Error Handling – Find files by date, filename, status, user, ar?fact name – Errors dashboard to diagnose problems – Real-‐?me ac?ve/recent file deliveries
• Extensive ReporAng – End to End instance reports – Course and fine grained ar?fact Transfer reports – Deployment, history and dependencies
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MFT – Enterprise Security Out of the Box End to End Security using Mul<ple Keystores • File EncrypAon – PGP
– Encrypt/decrypt Ac?ons on all ar?facts – Cer?ficates keystore managed by WLST commands
• FTP and sFTP Embedded Servers Access – Inbound file upload – Pass-‐by-‐reference outbound access
• Source & Target Web Service Policies – Standard OWSM policies – SOA, SOAP, OSB, ODI, B2B, HealthCare
• Web Console SSL and Role Based Access – Admin(All), Deploy(r/o monitor), Monitor(r/o design) – Managed from WebLogic Console/API’s
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MFT – Scheduling Embedded Oracle Enterprise Scheduler (ESS) • Core Scheduler FuncAonality
– Flexible frequencies: Once, Custom, Daily, Weekly … – Repeatable week(s) of month or day(s) of week – Start/stop date/?mes – Generate a schedule and customize as needed
• MFT Usage – Extended to include range or dura?ons – Use in Sources and Targets
• Not used in Web service Sources types: SOA, SOAP, OSB …
• External Scheduler Support – Jobs can invoke MFT Web Service to ini?ate a Transfer – MFT invoke external scheduler using custom callout or BPEL
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MFT – Extensibility Callouts and SOA Tooling Integra<on • Tight SOA/OSB/BPM/B2B IntegraAon
– MFT SOA binding for inbound or outbound SOAP – Extend to BPEL for complex integra?on paeerns – Human task, protocol transla?on, partner management
• Java Custom Callouts – Source, Pre and Post Target – Fully parameterized and reusable – Managed by WLST: createCallouts
• Custom Callout Use Cases – File Newline Conversion: Unix <-‐> DOS * – No?fica?ons, CSV Debatch, header manipula?on * – Virus scan on cri?cal files
* Available as samples
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MFT – SOA Integra?on • Use Cases
– SOA 11g/12c Reference invokes MFT Source – SOA 11g/12c Service invoked by MFT SOA Target – Process very large files by passing an FTP or file reference – Leverage SOA manual tasks, orchestra?on, transforma?ons … – OSB: virtualize file transfers, addi?onal protocols, JMS …
• New 12c JDeveloper Support for MFT – New MFT WS Binding for Composites and BPEL – Process inline payload, aeachments or pass-‐by-‐reference – Easy header processing: File name, size, user, transfer name
• Integrated MFT and EM Monitoring – Bi-‐direc?ons cross linking of consoles – Shared ECID, Correla?on Id
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Oracle Managed File Transfer Overview
Use Cases for Large File
Partner Introduc?on
Partner Solu?on and Demo
Summary and Q & A
1
2
3
4
5
2
1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MFT –Large File and Cloud Use Cases • IniAal Bulk and Cloud Load
-‐ Ini?al Applica?on Sync -‐ Ex: MFT delivers file from eBiz to ODI
• ApplicaAon Sync -‐ On Premise to SaaS Applica?ons -‐ Ex: Oracle eBiz à RightNow
• Big Data -‐ Connect local warehouse to Cloud BI Service -‐ Ex: …
• Design Documents and Media Files -‐ Daily Sync across geographies for round the clock opera?ons -‐ External partners delivering daily updates
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MFT – Processing Large Files Limited only by disk space and network speed • SOAP File Pass-‐By-‐Reference
-‐ Avoids parsing payload in the DOM -‐ Reference types: File, FTP, sFTP, Directory -‐ Includes headers: filename/size, user, linkback … -‐ Inline and aeachment op?on for small files
• Out of the Box File Compression/ExtracAon -‐ Compress/uncompress single large file Ex:1000% -‐ Extract contents to a dir and process files individually -‐ Compress contents of a dir and forward single file
• Batch/De-‐batch large XML,CSV files -‐ Inbound: strip files to dir using custom ac?on -‐ Outbound: append to exis?ng file
Pass-‐by-‐reference payload sample
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confiden?al – Internal/Restricted/Highly Restricted 19
MFT SOA
MFT Debatching Demo – MFT à SOA • Directory Pass Directory Ref to SOA
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Oracle Managed File Transfer Overview
Use Cases for Large File
Partner Introduc?on
Partner Solu?on and Demo
Summary and Q & A
1
2
3
4
5
3
1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle MFT Partner Solu?on Real World Use Case and Demo
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
L&T Infotech Ltd L&T Infotech OFMW Overview
300+ FMW consultants
50+ Global project rollouts
25 AcAve clients
Beta Partners
SOA (40%) WC (25%) IDM (20%) ODI (15%)
Manufacturing Insurance
Banking
Energy and Petrochemical Telecom
Industry focused tools/ accelerators
q Part of 14.3billon Larsen & Toubro Group of Company
q Provide Business to IT connect services to 30 Fortune 500 Client
q Oracle Pla?num and Validate Integrator Partner
q 30+ Specializa?ons in Oracle applica?ons and technologies
q Oracle Fusion Middleware Innova?on partner for 2013
q Focus on Industry solu?on, Accelerators and Frameworks
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Oracle Managed File Transfer Overview
Use Cases for Large File
Partner Introduc?on
Partner Solu?on and Demo
Summary and Q & A
1
2
3
4
5
4
1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Partner Solu?on and Demo
Any FTP Client Clearing .csv
sFTP/FTP
Secured BackUp System
Clearing System
ChargeBack System
Clearing Files
Rejected
Approved
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Partner Solu?on and Demo • Customer Profile
Leading Financial Services, Payments Solu?on Providers, Informa?on & Digital security Services. Plays Key role as partners for banks, businesses & retailers throughout the Nordic Region One of the most extensive & innova?ve product porsolios in Europe Is an unique partner in crea?ng & opera?ng solu?ons for banking sector, merchants & public sector. Key Business Areas: Cards, e-‐Security, Merchant Solu?ons, Payment & informa?on services Fusion Middleware Implementa?ons: 11g /12c SOA, BPM , OSB & 12c MFT solu?ons.
• Problem Statement Need of a single file transfer solu?on for all internal and external file transfer needs. Monitoring Mul?ple Merchant remote FTP servers & their financial files status. Generated Merchant files are huge in size due to large transac?on volume. Development & maintenance costs for the exis?ng custom file transfer solu?ons are too expensive. Sensi?ve & confiden?al data needs to be protected., Brings all file transfer communica?ons into compliance with PCI Standards. Support for mul?ple plasorms, including Windows, UNIX , Linux, AS/400, & the Mainframe .
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Partner Solu?on and Demo • Key Requirements
q Clearing Files received from Issuer Banks & Further handed over to Acquirer Bank for Clearance
§ Integra?on solu?on must be PCI-‐DSS Compliant
§ Clearing Files need to be PGP Encrypted, Compressed & Archived to Secured File System
§ Decrypt, Read & Validate File, Spliung of Valid & Disputed Card Transac?ons,
§ Valid Card transac?ons are sent out to Clearing System database for Clearance.
§ Disputed Transac?on are sent to Chargeback System for Manuel Verifica?on.
Acquirer Bank
Ckearing Files (.csv)
Clearing System
ChargeBack System
Check Disputed TransacAons
Secured BackUp System
ü Encrypt ü Compress ü Archive
ü Tx ValidaAon, ü Split Records ü Monitoring ü No6icaAons
Approved
Rejected Issuer Banks
Issuer Banks
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Partner Solu?on and Demo • How They Solved It?
• Benefits of the SoluAon Reduc?on in ?me to market & total cost of ownership Single file transfer solu?on to all internal and external recipients Unified monitoring consoles for audits and adherence :Monitoring Dashboards, Designing & Administra?ons Faster processing of Large files & Security implement ion is easier Just 1-‐Click processing of Failed /Lost files Easy deployment process from one environment to another Pre-‐Post Processing ac?ons are extendable and can be easily configured . Scheduling and Auto – Retry is much easier to setup Easy use by non-‐technical staff, and offers extensive repor?ng to provide status on file transfer
Acquirer Bank
Ckearing Files (.csv)
Ckearing System
Check Disputed TransacAons
Secured BackUp System
ü Encrypt ü Compress ü Archive ü Tx ValidaAon, ü Split Records ü Monitoring ü NoAficaAons
Approved
Rejected Issuer Banks
Issuer Banks
ChargeBack System
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Oracle Managed File Transfer Overview
Use Cases for Large File
Partner Introduc?on
Partner Solu?on and Demo
Summary and Q & A
1
2
3
4
5 5
1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |