Security monitoring boxes

Andrew McNab

University of Manchester

2 June 2005 Security monitoring boxes

Outline

● Background● What we could do● Constraints● “The Idea”● Syslog / RSS / Management● Security Monitoring WG● Next Steps

2 June 2005 Security monitoring boxes

The Background

● Sites get compromised already.

● We can reduce this a lot by keeping sites up to date with

OS patches etc.

● We can mitigate it further by watching for intrusions.● All of our sites use almost identical versions/configurations

● We're very vulnerable to repeat of an attack at many sites● The whole point of the Grid is to provide methods for cross-

institutional access

● Stolen credentials give “legitimate” access to more sites.

2 June 2005 Security monitoring boxes

What we could do

● What can GridPP do about this?● Currently, we “encourage” site admins to keep up to date

and to look for intrusions.● To provide “centralised” Tier-2/Tier-1/??? assistance on top

of this, we need to have a monitoring mechanism.

● Ideally, something we can base automated alarms on.● Inter-site monitoring is also needed to detect “legitimate”

cross-site attacks, using stolen credentials.

● Need to look for unusual usage patterns (cf credit cards)

2 June 2005 Security monitoring boxes

This is a bit like...

● ...monitoring the data from an HEP experiment● You have alarms/warnings generated by each

subdetector DAQ system● You have a central way of collecting, logging and

basing alerts on these messages● You also have higher level monitoring to spot data

quality issues like miscalibration

● “Is the distribution still flat in phi?” etc

● Only get that once you've collected things

centrally

2 June 2005 Security monitoring boxes

Constraints

● Do something quickly – don't start writing lots of code● Don't over burden the site admins – want it everywhere● Keep it detached from site – avoid being compromised,

and provide a hardened “safe” for copies of log files

from attack● Use standard protocols – don't create an oddball system

that lives in it's own world● Interoperate with existing components on sites

● Kernel and OS daemons (sshd etc)

● Grid Middleware

2 June 2005 Security monitoring boxes

So ... “The Idea”

● We provide site admins with an installation DVD that

installs a “Security Monitoring Box” ● Do not use RHEL/SLC to keep kernel / sshd / etc versions

different – Fedora Core probably different enough● Box gathers local messages via the syslog protocol● Command-line admin is done by local site, as root● Monitoring managed via web interface (using GridSite)● syslog messages are republished via RSS feeds● Tier-2/Tier-1/?? watch SecMonBox feeds for trigger signals

2 June 2005 Security monitoring boxes

Inputs to syslog

● syslog is the default logging system on Unix boxes● syslog consists of a

● system call / command line tool for injecting

messages

● syslogd running on each machine records and/or

forwards messages based on filters and syslog.conf● sshd etc use syslog by default● Globus, Apache etc can use it too● Linux kernel can be made to log things like network

probes via syslog

2 June 2005 Security monitoring boxes

RSS

● RSS is widely used to allow clients to pull

categorised, chronological data (like news headlines)

out of webservers, in a programmatic way● As such, it is also well matched to transporting syslog

type alert messages● We can offer multiple channels depending on syslog

service (“sshd”) and severity (“critical”) to provide

coarse filtering● Since RSS is XML text, can search for patterns with

XML or stream tools (like Perl or even grep)

2 June 2005 Security monitoring boxes

Management

● Need to keep boxes themselves patched – use yum● Need to update our software on SecMonBox – yum again?● Provide management interface via GridSite

● Site admin + remote access by Tier-2/Tier-1/??? staff● But aim for minimal configuration: disk space management,

log file expiration, triggering updates, access rights, ...● Want to be able to rebuild a box rapidly – if site is attacked,

may want to give the SecMonBox hard drive to the police

● All choices stored in one config file + idempotent scripts?

2 June 2005 Security monitoring boxes

Security Monitoring

WG

● Being organised by Romain Wartel at RAL● Aims to define:

● what to monitor at sites / on the wider Grid

● recommend what tools to install to monitor that

● how to use the results● “Security Monitoring Box” would provide one set of

local and central tools to base monitoring on● Romain is also using RSS to syndicate security

announcements to websites

2 June 2005 Security monitoring boxes

Next steps

● Produce a prototype Fedora 3 / SecMonBox DVD

(image)

● Installs on “sensible” hardware

● Installs GridSite + RSS service + basic config● Deploy at some volunteer sites● Demonstrate central collection of logging messages● Co-ordinate with Security Monitoring WG

recommendations

● on what to log

● and on how to filter / pattern match for attacks