© 2009 STREAM FRBC

1

Conference of State Bank Supervisors

IT Training

STREAM Technology Lab Overview23-June-2009

Federal Reserve Bank of ChicagoS&R Technology Lab

Presented byChristopher Olson

Federal Reserve Bank of ChicagoChristopher.Olson@chi.frb.org

© 2009 STREAM FRBC

2

Agenda

• What is Risk?• Bank Operations Simulation• Asset Liability Management Modeling• IT Topic: Virtualization• Instructor Subject Matter Experts• Technology Lab History and Build-out

© 2009 STREAM FRBC

3

What is Risk?

• Webster's dictionary: "the possibility of a loss". – Future event– Uncertainty of occurrence; probability

• Probability is greater than 0 and less than 1 (or greater than 0% and less than 100%)

– Uncertain outcome or impact– Favorable and unfavorable outcome

© 2009 STREAM FRBC

4

Risks Are Interactive

Reputational

Market

Operational Legal

Credit Liquidity

© 2009 STREAM FRBC

5

Operational Risk Defined

“The risk of loss from inadequate or failed internal processes, people, and systems, or from external events.” – Basel, “Sound Practices for the Management & Supervision of Operational Risk”

Translation: Everything that’s not credit and market risk.

© 2009 STREAM FRBC

6

Operational Risks:

PeopleProcessesSystems

External Events

Insufficient staffUnsafe work placeFraudSecurity breachesBusiness disruptionProduct flawsCustomer unsuitabilityImproper practicesUnsafe work placeProcessing errorsDocumentation errors

Why focus on Operational Risk?

CreditMarket

LiquidityLegal

Reputational

Operational Risks:

PeopleProcessesSystems

External Events

© 2009 STREAM FRBC

7

Scandals Galore

Nick LeesonBarings

Kim Woo-choongDaewoo

Mark Swartz/Dennis KozlowskiTyco

Ken Lay, Jeff Skilling, Andy Fastow, Lou PaiEnron

© 2009 STREAM FRBC

8Examiner Responsibilities

Internal Control Activities

You’re Doomed!

© 2009 STREAM FRBC

9

Control Activities

Bank performance reviews in each business line

Physical and logical controls Separation of duties Conflicts of interest Compensating controls Approvals and authorizations Verifications and reconciliations Information processing

© 2009 STREAM FRBC

10

Bank Operations Simulation Course

© 2009 STREAM FRBC

11

• Cash and Teller Operations• Check Operations• NSF Processing and Transaction Input• Proof and Transit• Back Office Routines• ACH Operations• Investment Operations• Loan Operations• Wire Transfer Operations

BOpS Course Modules

© 2009 STREAM FRBC

12

BOpS Course Modules (continued)

• System and Security Access • Accounts Payable• Fixed Assets• Correspondent Bank Account Reconciliation• Payment System Risk• Call Report Review• Daily Statement Review• Extensive Hands On Training!

© 2009 STREAM FRBC

13

Bank Operations Simulation Course

• Provides core curriculum and training in bank operations.

• Target audience is all Safety and Soundness examiners who are looking for bank operations training!

© 2009 STREAM FRBC

14

Other Application Classes

• BSA/ AML Hands On Lab• Asset Liability Management Model Lab

We call this the “ALM” class

© 2009 STREAM FRBC

15

ALM Course

© 2009 STREAM FRBC

16

Course Background

Effective IRR model reviews require a specialized set of examination tools

• Regulatory Market Risk Knowledge- PALM (f.k.a. FIRRM)- ALM 1, ALM 2

• Understanding of financial instruments- Options Institute- PALM- ALM 1, ALM 2

© 2009 STREAM FRBC

17

Course Background (continued)

• Fundamental understanding of financial modeling– Vocabulary– Internal controls– Technical implementation options, risk, and

limitations• Understanding of moderate simulation and

valuation techniques supported or not supported by model vendors– Baker Group, ProfitStars, Compass, Sendero, Bancware

© 2009 STREAM FRBC

18

ALM Model Vendor Usage—Member Banks

• 68 IRR models or consultants represented• QRM

17 banks with $1.4 trillion in total assets. 15 QRM firms have total assets > $10 billion

• Bancware27 banks with $613 billion in total assets

• Sendero 114 Banks with $413 billion in total assets

• Plansmith / Intercept92 banks with $22 billion in total assets

2004 FRS Board of Governors Survey

© 2009 STREAM FRBC

19

ALM Model Vendor Usage

• IPS Sendero ALM is used at the largest number of FRS member institutions (114)

• BancWare ALM4 and ALM5 are widely used at our largest institutions and many regional banks

© 2009 STREAM FRBC

20

Course Objective

ALM Model class provides examiner the ability to assess:

• The appropriateness of the general model setup • The appropriateness of specific complex instrument

setups• The accuracy and reasonableness of critical model

assumptions• Whether critical assumptions have been correctly

implemented in a model• Common model risk control weaknesses• The overall adequacy of model risk management

practices

© 2009 STREAM FRBC

21

IRR Identification and Management

• Objectives:– Identify four primary sources of IRR Discuss the

modeling process and the types of models most commonly used by banks

– Learn what questions to ask your management team– Discuss supervisory expectations and best practices

for strong IRR management

© 2009 STREAM FRBC

22

Interest Rate Risk

• Mismatch Risk– The risk that interest rates change and assets and liabilities

re-price at different times• Yield Curve Risk

– The risk of non-parallel shifts in the yield curve• Basis Risk

– The risk that rates on instruments with the same or similar maturities will not move together as the general level of interest rates changes

• Options Risk– The risk that changes in interest rates will cause asset or

liability holders to exercise explicit or embedded options

© 2009 STREAM FRBC

23

What Should IRR Models Do?

• The IRR modeling process should:– produce reasonably accurate risk measures

– capture all risks material to the institution

– provide clear and useful information to senior management and board of directors

© 2009 STREAM FRBC

24

What Should Drive the Model Decision?

• Complexity of:– Bank and Organizational Structure– Products and Services– Positions Held– Markets

• Cost versus Benefit• Materiality of Risk• Exposure to Risk Factors

© 2009 STREAM FRBC

25

Information Technology Classes

• e-Banking• IS Vulnerability Management• Network Security• Operating Systems• Supervisory Themes

© 2009 STREAM FRBC

26

IT Topic: Virtualization

© 2009 STREAM FRBC

27

What is Virtualization

• An application and its base operating system combined together in a single compact package

© 2009 STREAM FRBC

28

What is Virtualization?

• Resources are shared between the host systems according to demand

• Resources: CPU, Memory, Network and Disk space

© 2009 STREAM FRBC

29

What is Virtualization?

• Virtualization works by allowing multiple operating systems to be installed on a single physical server– Hypervisor is software

that makes each Virtual Machine appear as a standalone server

Virtual Machine 1

Virtual Machine 2

Hypervisor (Software)Enables CPU,

Memory, Network and Disk sharing

© 2009 STREAM FRBC

30

Two Attack Scenarios

• External Attacker: A vulnerable VM is attacked from an outside attacker– Phase 1: Vulnerability– Phase 2: Exploitation– Phase 3: Extend Control

• Internal Attacker: An attacker compromises the hypervisor (“hyperjacking”)– Hypervisor Rootkit– Off-Host Attack

© 2009 STREAM FRBC

31

Attack Phase 1: Vulnerability

• VM 1 is un-patched and vulnerable

• VM 2, 4, 5 and 6 are patched and compliant

• VM 3 is running with a known vulnerability due to application requirements

• VM 3 not externally available (private)

Attacker is in control of VM 1

© 2009 STREAM FRBC

32

Attack Phase 2: Exploitation

• External attacker launches attacks against other VMs

• Port scans are not detected by the network monitoring device

• No IP traffic traverses the physical NIC on the host

Attacker is in control of VM 1

© 2009 STREAM FRBC

33

Attack Phase 3: Extend Control

• VM 1 and VM 3 are under the control of an external attacker

• Attacker uses trusted production server VM 3 to probe for vulnerabilities in other hosts

• Attacker discovers and exploits VM 6

© 2009 STREAM FRBC

34

Two Attack Scenarios

• External Attacker: A vulnerable VM is attacked from an outside attacker– Phase 1: Vulnerability– Phase 2: Exploitation– Phase 3: Extend Control

• Internal Attacker: An attacker compromises the hypervisor (“hyperjacking”)– Hypervisor Rootkit– Off-Host Attack

© 2009 STREAM FRBC

35

Hypervisor Rootkit

• Hypervisor root kit is inserted on the running hypervisor from a trusted guest

• Attack vector is a known vulnerability on VM 3

Hypervisor Rootkit attacks VM 3

© 2009 STREAM FRBC

36

Attack from Outside of the VM

• A direct attack on the hypervisor comes from an outside the VM

• Attack vector is either from a network connection or from physical access (insider attack)

Outside source attacks hypervisor

© 2009 STREAM FRBC

37

Result: Hyperjacked Host

• All communication to the guest VM’s is compromised

• Guest VMs have no way of knowing that the hypervisor is compromised

• On-guest security tools have no way to “see” the compromise

Hyperjacked Host

© 2009 STREAM FRBC

38

Lessons Learned from the Attack

• A vulnerable VM leads to intra-host risk and potential compromise

• The intra-host (“inside-out”) risk results from running public and private servers in the same environment

• The risk of intra-host (“inside-out”) attacks increases– The financial institution must think through the

security considerations of their architecture

© 2009 STREAM FRBC

39

Implementation Principle #1

• The Bank must understand and document their virtualization solution– Use documentation from the Vendor– Leverage open initiatives (DISA, CISecurity.org,

SANS)– Document physically and logically where

Virtualization fits in the bank

– The Financial Institution must allocate time for training, testing and documentation

© 2009 STREAM FRBC

40

Implementation Principle #2

• Ensure that changes are documented and implemented successfully– Patch Management– Help Desk and Configuration Management

• Change Management is a necessity for incident response– Why? It helps to determine whether an authorize

or unauthorized change led to the event/incident

© 2009 STREAM FRBC

41

Implementation Principle #3

• Plan the Dive and Dive the Plan– Proper planning is essential– Perform a test in a laboratory environment– Define requirements and architect the supporting

solution• Iterate

– Remember Security, but focus on process

© 2009 STREAM FRBC

42

80 % Process, 20% Technology

• Updated Management Processes• Patching of Offline Systems• Access to New Management Tools• Configuration Standards

© 2009 STREAM FRBC

43

Updated Management Processes

• Handling of virtual disks– State is saved as a file (VM disk Image) that can

be copied– The VM disk Image can be analyzed—used by an

attacker / rogue administrator– Treat the File (VM disk image) as a high-security

object– DO NOT store the VM disk image on USB sticks,

portable drives, desktops or other insecure places

© 2009 STREAM FRBC

44

Patching of Offline Systems

• Problem: Offline Virtual Machines (VMs) lag behind on updates– Patching, Anti-Virus and other tools are agent based– Agents don’t work when the VM disk image is offline– Offline images become security risks

• Solution: Don’t let the VMs lag– Adopt tools that can update (patch, Anti-Virus, etc.) the

VM while offline– Adopt tools that scan the VM when they boot

© 2009 STREAM FRBC

45

Access to New Management Tools

• Access Control Life Cycle—Physical Environment– How is server access currently managed– Request, Approve, Provision, Review (RAPR)

• Access Control Life Cycle—Virtual– Enhance the physical management to include

virtual tools

© 2009 STREAM FRBC

46

Configuration Standards

• Problem:– Easy VM disk image copying facilitates easy

replication of security vulnerabilities

• Mitigation:– Ask if the financial institution has adopted

templates

© 2009 STREAM FRBC

47

Case Study—Virtualization

© 2009 STREAM FRBC

48

STREAM Technology Lab Classes

• E-banking• Network Security• IS Vulnerability Management• Operating Systems• Supervisory Themes• Bank Operations Simulation• Asset Liability Management Modeling• Bank Secrecy Act / Anti-Money Laundering

© 2009 STREAM FRBC

49

Course Attendance: 2000-2008

Course attendance continues to increase. 2007 and 2008 shows continued overall growth with near-capacity attendance in each of the three IT Application courses.

STREAM Technology Lab: Student Hours of Training(including 2008 Estimates)

0

2000

4000

6000

8000

10000

12000

2000 2001 2002 2003 2004 2005 2006 2007 2008

Stud

ent H

ours

Application

IT Classes

© 2009 STREAM FRBC

50

Course Participant Affiliations

2008 Projections

FRS56%

Agency9%

State25%

International10%

Course participants have diverse affiliation from across the Federal Reserve System, FFIEC agencies, state regulators and international central banks.

© 2009 STREAM FRBC

51

Outreach and Partnerships

• States: Conference of State Bank Supervisors, Sebastien Monet

• Federal Reserve: Board of Governors, Districts, Center for Online Learning, SDS, Compliance and Consumer Affairs, Payments

• FFIEC: FDIC, OTS, OCC, NCUA• International: Liaison program through Board of Governors,

International IT Steering Group• Industry: Financial Crimes Task Force, FBI/Infragard• Academic: DePaul University

District, Federal Reserve, FFIEC, States, International and Industry outreach.

© 2009 STREAM FRBC

52

We welcome State SMEs!

• IT and Bank Operations experience• States-only dedicated classes??• Details:

– See course schedules– Instructing experience or FFIEC Instructor Training– Minimum 2 weeks of time in first year– Instructor Conference week of Feb 2, 2009

© 2009 STREAM FRBC

53

Questions!

• What is Risk?• Bank Operations Simulation• Asset Liability Management Modeling• IT Topic: Virtualization• Instructor Subject Matter Experts• Technology Lab History and Build-out