processinglargefileswith oracle… · ̶ just1;click*processing**offailed/lostﬁles* ̶...

Processing Large Files with Oracle Managed File Transfer Dave Berry -‐ Oracle Senior Manager – Integra?on Products Pandurang Ranjalkar -‐ Larsen & Toubro Infotech Limited Oracle Fusion Middleware Prac?ce Head

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

CON 7826


Safe Harbor Statement The following is intended to outline our general product direc?on. It is intended for informa?on purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or func?onality, and should not be relied upon in making purchasing decisions. The development, release, and ?ming of any features or func?onality described for Oracle’s products remains at the sole discre?on of Oracle.


Program Agenda

Oracle Managed File Transfer Overview

Use Cases for Large File

Partner Introduc?on

Partner Solu?on and Demo

Summary and Q & A

1

2

3

4

5


Oracle Fusion Middleware The Cloud Pla6orm for Digital Business

Cloud On-‐Premise

DIGITAL ENGAGEMENT

APPLICATION & DATA INTEGRATION IDEN

TITY

MAN

AGEM

ENT

SYSTEM

S MAN

AGEM

ENT

APPLICATION INFRASTRUCTURE & TOOLS

BUSINESS PROCESS MANAGEMENT BUSINESS ANALYTICS CONTENT & COLLABORATION

Web Mobile Social Internet of Things


Program Agenda with Highlight



Partner Introduc?on


Summary and Q & A

1

2

3

4

5


Uncontrollable problem of file exchanges

Ø ProliferaAon of FTP servers & clients Ø No central control on exchanges of sensi?ve informa?on

Ø No global visibility Ø Not integrated with enterprise security standards: FTP servers are rarely integrated with directories

Ø Single points of failure: FTP servers rarely offer HA capabili?es

Lack of control, visibility, security & reliability

Product specifica?ons

Manufacturing Design partners

Enterprise

Tax informa?on

Administra?on Legal, PR

Contracts, Media

Employee Data

Service Providers


Oracle Managed File Transfer (MFT)

• Automate & schedule file transfers • Highly-‐Available, clusterable solu?on • Complete visibility & auditability •  Support for very large files (500+GB) & pass by reference

•  Secure: SSH,SSL, PGP Encryp?on •  Integrated with SOA Suite, B2B, ODI & IdM • Works with std FTP clients (mul?threaded) •  Extensible: compression, custom ac?ons

send receive

DOCUMENT STORE

Audi?ng Monitoring

Temporary store

enterprise

internet


MFT Func?onal Use Case Paeerns Sources

Fan Out Direct

Transfers Targets Local FTP

Remote FTP

Local File

SOAP

Pass-‐By-‐Reference

Hybrid Integra<on Cloud

Local FTP

Remote FTP

Local File

SOAP

* includes sFTP and FTPs


MFT Simple Design of Transfers Browser Based Environment •  Configure Transfers

–  Reusable Sources and Targets –  Schedules, filters, compression, encryp?on –  Secure endpoint file access

•  Endpoint Types –  Event: Embedded FTP, SOAP, SOA, B2B, OSB, ODI –  Polling: File and remote FTP adapters

•  Extreme Ease of Use –  Drag and drop, no developer required –  Transfer rename, duplicate, filtered transfer list –  Deployment details, export, import –  Reusable custom ac?ons: no?fica?ons, file conversion …


Oracle MFT – Global Visibility Metrics, Diagnos<cs & Extensive Repor<ng •  Metrics Dashboards

–  Fine and course grained metrics – Metrics for every ar?fact: Source, Target and Transfer

•  Totals, file size, speed, elapsed ?mes, failure ra?o

•  DiagnosAcs & Error Handling –  Find files by date, filename, status, user, ar?fact name –  Errors dashboard to diagnose problems –  Real-‐?me ac?ve/recent file deliveries

•  Extensive ReporAng –  End to End instance reports –  Course and fine grained ar?fact Transfer reports –  Deployment, history and dependencies


Oracle MFT – Enterprise Security Out of the Box End to End Security using Mul<ple Keystores •  File EncrypAon – PGP

–  Encrypt/decrypt Ac?ons on all ar?facts –  Cer?ficates keystore managed by WLST commands

•  FTP and sFTP Embedded Servers Access –  Inbound file upload –  Pass-‐by-‐reference outbound access

•  Source & Target Web Service Policies –  Standard OWSM policies –  SOA, SOAP, OSB, ODI, B2B, HealthCare

•  Web Console SSL and Role Based Access –  Admin(All), Deploy(r/o monitor), Monitor(r/o design) – Managed from WebLogic Console/API’s


Oracle MFT – Scheduling Embedded Oracle Enterprise Scheduler (ESS) •  Core Scheduler FuncAonality

–  Flexible frequencies: Once, Custom, Daily, Weekly … –  Repeatable week(s) of month or day(s) of week –  Start/stop date/?mes –  Generate a schedule and customize as needed

•  MFT Usage –  Extended to include range or dura?ons –  Use in Sources and Targets

•  Not used in Web service Sources types: SOA, SOAP, OSB …

•  External Scheduler Support –  Jobs can invoke MFT Web Service to ini?ate a Transfer – MFT invoke external scheduler using custom callout or BPEL


Oracle MFT – Extensibility Callouts and SOA Tooling Integra<on •  Tight SOA/OSB/BPM/B2B IntegraAon

– MFT SOA binding for inbound or outbound SOAP –  Extend to BPEL for complex integra?on paeerns –  Human task, protocol transla?on, partner management

•  Java Custom Callouts –  Source, Pre and Post Target –  Fully parameterized and reusable – Managed by WLST: createCallouts

•  Custom Callout Use Cases –  File Newline Conversion: Unix <-‐> DOS * –  No?fica?ons, CSV Debatch, header manipula?on * –  Virus scan on cri?cal files

* Available as samples


Oracle MFT – SOA Integra?on •  Use Cases

–  SOA 11g/12c Reference invokes MFT Source –  SOA 11g/12c Service invoked by MFT SOA Target –  Process very large files by passing an FTP or file reference –  Leverage SOA manual tasks, orchestra?on, transforma?ons … –  OSB: virtualize file transfers, addi?onal protocols, JMS …

•  New 12c JDeveloper Support for MFT –  New MFT WS Binding for Composites and BPEL –  Process inline payload, aeachments or pass-‐by-‐reference –  Easy header processing: File name, size, user, transfer name

•  Integrated MFT and EM Monitoring –  Bi-‐direc?ons cross linking of consoles –  Shared ECID, Correla?on Id





Partner Introduc?on


Summary and Q & A

1

2

3

4

5

2

1


Oracle MFT –Large File and Cloud Use Cases •  IniAal Bulk and Cloud Load

-‐  Ini?al Applica?on Sync -‐  Ex: MFT delivers file from eBiz to ODI

•  ApplicaAon Sync -‐  On Premise to SaaS Applica?ons -‐  Ex: Oracle eBiz à RightNow

•  Big Data -‐  Connect local warehouse to Cloud BI Service -‐  Ex: …

•  Design Documents and Media Files -‐  Daily Sync across geographies for round the clock opera?ons -‐  External partners delivering daily updates


Oracle MFT – Processing Large Files Limited only by disk space and network speed •  SOAP File Pass-‐By-‐Reference

-‐  Avoids parsing payload in the DOM -‐  Reference types: File, FTP, sFTP, Directory -‐  Includes headers: filename/size, user, linkback … -‐  Inline and aeachment op?on for small files

•  Out of the Box File Compression/ExtracAon -‐  Compress/uncompress single large file Ex:1000% -‐  Extract contents to a dir and process files individually -‐  Compress contents of a dir and forward single file

•  Batch/De-‐batch large XML,CSV files -‐  Inbound: strip files to dir using custom ac?on -‐  Outbound: append to exis?ng file

Pass-‐by-‐reference payload sample

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confiden?al – Internal/Restricted/Highly Restricted 19

MFT SOA

MFT Debatching Demo – MFT à SOA • Directory Pass Directory Ref to SOA





Partner Introduc?on


Summary and Q & A

1

2

3

4

5

3

1


Oracle MFT Partner Solu?on Real World Use Case and Demo


L&T Infotech Ltd L&T Infotech OFMW Overview

300+ FMW consultants

50+ Global project rollouts

25 AcAve clients

Beta Partners

SOA (40%) WC (25%) IDM (20%) ODI (15%)

Manufacturing Insurance

Banking

Energy and Petrochemical Telecom

Industry focused tools/ accelerators

q  Part of 14.3billon Larsen & Toubro Group of Company

q  Provide Business to IT connect services to 30 Fortune 500 Client

q  Oracle Pla?num and Validate Integrator Partner

q  30+ Specializa?ons in Oracle applica?ons and technologies

q  Oracle Fusion Middleware Innova?on partner for 2013

q  Focus on Industry solu?on, Accelerators and Frameworks





Partner Introduc?on


Summary and Q & A

1

2

3

4

5

4

1



Any FTP Client Clearing .csv

sFTP/FTP

Secured BackUp System

Clearing System

ChargeBack System

Clearing Files

Rejected

Approved


Partner Solu?on and Demo •  Customer Profile

  Leading Financial Services, Payments Solu?on Providers, Informa?on & Digital security Services.   Plays Key role as partners for banks, businesses & retailers throughout the Nordic Region   One of the most extensive & innova?ve product porsolios in Europe   Is an unique partner in crea?ng & opera?ng solu?ons for banking sector, merchants & public sector.   Key Business Areas: Cards, e-‐Security, Merchant Solu?ons, Payment & informa?on services   Fusion Middleware Implementa?ons: 11g /12c SOA, BPM , OSB & 12c MFT solu?ons.

•  Problem Statement   Need of a single file transfer solu?on for all internal and external file transfer needs.   Monitoring Mul?ple Merchant remote FTP servers & their financial files status.   Generated Merchant files are huge in size due to large transac?on volume.   Development & maintenance costs for the exis?ng custom file transfer solu?ons are too expensive.   Sensi?ve & confiden?al data needs to be protected.,   Brings all file transfer communica?ons into compliance with PCI Standards.   Support for mul?ple plasorms, including Windows, UNIX , Linux, AS/400, & the Mainframe .


Partner Solu?on and Demo •  Key Requirements

q  Clearing Files received from Issuer Banks & Further handed over to Acquirer Bank for Clearance

§  Integra?on solu?on must be PCI-‐DSS Compliant

§  Clearing Files need to be PGP Encrypted, Compressed & Archived to Secured File System

§  Decrypt, Read & Validate File, Spliung of Valid & Disputed Card Transac?ons,

§  Valid Card transac?ons are sent out to Clearing System database for Clearance.

§  Disputed Transac?on are sent to Chargeback System for Manuel Verifica?on.

Acquirer Bank

Ckearing Files (.csv)

Clearing System

ChargeBack System

Check Disputed TransacAons


ü  Encrypt ü  Compress ü  Archive

ü  Tx ValidaAon, ü  Split Records ü Monitoring ü  No6icaAons

Approved

Rejected Issuer Banks

Issuer Banks


Partner Solu?on and Demo •  How They Solved It?

•  Benefits of the SoluAon   Reduc?on in ?me to market & total cost of ownership   Single file transfer solu?on to all internal and external recipients   Unified monitoring consoles for audits and adherence :Monitoring Dashboards, Designing & Administra?ons   Faster processing of Large files & Security implement ion is easier   Just 1-‐Click processing of Failed /Lost files   Easy deployment process from one environment to another   Pre-‐Post Processing ac?ons are extendable and can be easily configured .   Scheduling and Auto – Retry is much easier to setup   Easy use by non-‐technical staff, and offers extensive repor?ng to provide status on file transfer

Acquirer Bank

Ckearing Files (.csv)

Ckearing System

Check Disputed TransacAons


ü  Encrypt ü  Compress ü  Archive ü  Tx ValidaAon, ü  Split Records ü Monitoring ü  NoAficaAons

Approved

Rejected Issuer Banks

Issuer Banks

ChargeBack System





Partner Introduc?on


Summary and Q & A

1

2

3

4

5 5

1

processing*large*files*with* oracle… · ̶ just1;click*processing**of*failed*/lostﬁles* ̶...

Documents

processinglargefileswith oracle… · ̶ just1;click*processing**offailed/lostﬁles* ̶...