1

Vulnerability management – 3i’s journey

Tom Kingtom.king@3i.comA presentation to: Qualys Security ConferenceNovember 2009

www.3i.com

2

Contents

Vulnerability management

About 3i

Our journey – the eras of vulnerability management

Challenges & gotchas

Benefits

Conclusions

3

About 3i

3i plc – the company• A world leader in private equity

• Focus on buyouts, growth capital, infrastructure

• €8.5 billion assets under management, with offices in 12 countries

3i plc – IT• Serving all internal users (circa 750 users)

• Very high expectations of “IT service”

• Largely a Microsoft house, try to avoid bleeding edge technologies3i plc – Information security

• Small security team (two), operational security with other teams

• Good synergy with other internal teams, e.g. Compliance, Risk

• Use ISO 27001/2 as backbone of InfoSec program

4

The eras of vulnerability management

Ad-hoc/ reactive

Microsoft patching

External vulnerability

scanning

Internal vulnerability

scanning

Risk view on vulnerabilities

The past

The future

5

First era – ad-hoc and reactive

First era – ad-hoc and reactive

• Little clarity on threats, vulnerabilities, risks

• Reactive approach

• Annual penetration test against external IP’s?

• Widespread media attention around a malware threat, e.g Nimda

• Main focus on network perimeter – keep the bad guys/ stuff out

• Number of threats and vulnerabilities were snowballing exponentially..

Growth in vulnerabilities and malware - (sources NIST/ F-Secure)

0

1000

2000

3000

4000

5000

6000

7000

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

Year

Vuln

erab

ilitie

s

0

100000

200000

300000

400000

500000

600000

Mal

war

e

Vulnerabilities

Malware

6

Second era – Microsoft patching

Second era – Microsoft patching

• “Monster” worms were continuing to hit companies – not 3i!

• Blaster – global cost (lost/ productivity) - $1.3 billion

• SQL slammer – infected most vulnerable hosts on the Internet in minutes not hours

• Anti-virus helped but was not a panacea – often did not prevent an infection

• Important defensive measure was to ensure timely application of Microsoft patches

• Simple edict from CIO – apply all relevant Microsoft patches..

• Geared up processes and technology to deal with “Patch Tuesday”

• Started to track and report missing patches

7

Third era – external vulnerability scanning

Third era – external vulnerability scanning

• Some pressure from auditors to deploy intrusion detection

• Personal view – great as a burglar alarm, but has challenges..

• Proposed a different direction – improved vulnerability management

• “Let’s find our weak spots, and fix them”. How simple!

• Purchased a well-known SaaS vulnerability scanning solution

• Only scanned Internet-accessible machines – web servers, mail servers, remote access etc.

• Simple KPI and incident process agreed with IT management

• Monitoring, trending, reporting

8

Fourth era – internal vulnerability scanning

Fourth era – internal vulnerability scanning

• Extended SaaS solution to scan machines on internal network

• Great to see what the real picture is, but..

• Huge number of vulnerabilities found

• Herculean task to make any improvement

• Gartner advocate dealing with “high severity” vulnerabilities first, still difficult!

• Ad-hoc pressure from security team to fix certain vulnerabilities

• E.g. on critical machines, on machines with sensitive data

• Monitoring, trending, reporting

9

Fifth era – risk view of vulnerabilities

Fifth era – risk view of vulnerabilities

• Using a simple risk framework

• Risk = threat * vulnerability * asset value

• Makes much more sense of vulnerability data, e.g

• Does it matter if a machine has vulnerabilities if its asset value is low?

• If a machine is in a hostile environment and is valuable, any significant vulnerability is a big issue..

• Tracking over time, monthly reporting to IT management team

• Gives a more meaningful view of the issue – allows better prioritisation of remediation resource.

10

Vulnerability reporting – through the eras

Vulnerability reporting – through the eras

• Ad-hoc/ reactive – little reporting, maybe detailed technical pen-tests

• Microsoft patching – some more detailed data, difficult to see what is important (and why)

• External vulnerabilty scanning – useful focus on Internet-facing vulnerabilities. Simple KPI & incident response process worked well

• Internal vulnerability scanning – whoah! Information overload..

• E.g. scanning 300 machines, each machine has a vulnerability report of ~150 A4 pages!

• Focus initially on critical vulnerabilities per “service”

• Risk view of vulnerabilities – simple RAG table..

Microsoft Patching Index

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

Feb Mar Apr May Jun JulDate

Financial Systems

InvestmentSystems

EU Desktop -Citrix, data etc

Messaging

EU Workstations

EU Laptops

AP Servers

USA Servers

AP Clients

All machines

Potential and confirmed vulnerabilities on Internet-facing machines

0

2

4

6

8

10

Dec Jan Feb Mar Apr May Jun Jul

Date

Num

ber o

f vul

nera

bilit

ies

Confirmed

severity 5

Confirmed

severity 4

Potential

severity 5

Potential

severity 4

Critical vulnerabilities index per service

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

Jun Jul Aug Sep Oct Nov Dec Jan Feb

Date

Group systems

Investment systems

EU Servers

Messaging

Network

EU Workstations

EU Laptops

11

Vulnerability management – monthly KPI’s

Security KPI’s – reported monthly

• Monthly security management pack produced

• 13 KPI’s (cost, resourcing, malware, security incidents, policy compliance, Microsoft patches, vulnerability scanning etc.)

12

Benefits – vulnerability management

Benefits

• Finally getting a holistic view of vulnerabilities

• Across entire estate – internal & external machines

• Not just focussed on Microsoft vulnerabilities

• Risk focussed – threat, vulnerability asset value all considered

• KPI’s and reporting shaping into something useful

• Can determine what issues to address, in what order

• Secondary benefits emerging – e.g. machine comparison

• Journey has not ended

• Not enough visibility on (web) application level vulnerabilities

• Need to address medium risk areas – aim for all green

13

Challenges and gotchas – vulnerability management

Challenges and gotchas

Challenge What we’ve done

Sheer number of vulnerabilities Risk view to help prioritise

False +ve’s Very infrequent, but YMMV

Disruption of live services Generally not an issue, with smart timing and low intensity scans

Timely remediation Risk view helps. Defined and agreed response process helps

Vulnerability landscape changes frequently

Frequent scans

14

Questions/ Answers/ Discussion

Questions/ Answers/ Discussion