The True Cost of Running a SIEM By Guest Blogger, David Humphrey, Harvard Pilgrim HealthCare

In this day and age of; “plug-and-play”, the need for instant online gratification, and appliances that

encapsulate all of the functionality you need in one convenient shiny wrapper, the idea of installing a

SIEM into the corporate environment is not met with that much trepidation. After all, how bad can it

be? Everything is made to work with everything else out there, and implementing a SIEM should not be that much of a challenge.

The perception that much of our IT management have is that implementing new IT functionality will

not impact the IT service management model significantly, and should always be easier than it once

was in the recent past for other IT projects. Yet, while a SIEM installation is not rocket science, it is

prudent to recognize just what the true cost of running and maintaining a SIEM will be. And the first step to take along this path is to set the correct level of expectations on just what the project is...

One should consider it to be on the level of replacing the piston rings in the engine of your car.

Now, if you think that clearing the space in your garage for such a project, acquiring the tools to

properly address this engineering feat, and mustering the patience to tackle a lot of unknown issues

with your very-familiar vehicle might be daunting - you’re on the right plane of where your level of

expectations should be for putting a SIEM into the environment. Sure, there are plenty of guys out there that can replace a head gasket, but for most of us, that task is well over our heads.

Most of us are going to pay the dealership cost to have them do it. And that cost will be far more that

we want to pay for maintaining our vehicle. But this is because we never put the maintenance cost into

the evaluation matrix for when we originally purchased said transportation. When the time came to

select that new VW bug over the Honda Accord, well the VW won out on all the cool features. Now

comes the cost of maintenance…

I still remember the vendor meetings at my company when I brought the “technology” in to meet the IT

staff. Oh, the questions they asked! Here is a wonderful example: since logs are, by default, available

on the network, what needs to be done to this new SIEM to suck these down and begin reporting on the

environment? What will it cost for the vendor to configure their new toy to work with application ‘X’? Does it work with application ‘Y’?

The answers were succinctly typical in respective order; it comes with a lot of reporting functionality right out of the box; ‘yes’ and ‘yes’. It will work with all of your IT systems.

And that was true.

What no one realized needed to be asked; was not how well the SIEM could be used in the

environment, but what needed to be done in the environment to use a SIEM.

All of the Unix systems need a centralized logging system to be developed for the SIEM – the logs were

not magically available in the ether to be sucked off the network. The domain controllers needed a

GPO update to determine what was going to be logged, and whether or not storage was going to be

sufficient, locally, to wait for the SIEM to query them. The Oracle databases had to be configured to

use Audit logging, and 30 or more configuration statements would need to be tuned to create pertinent

audit logs, including securing access to those logs, and an automatic purging stored procedure to dump

logs older than several days so that the database did not wipe out all available disk space. All of those

shiny appliances needed to be administered so that they too would log directly to the SIEM, and each

web-server would need a new process installed on the server to monitor the web logs so that those lines

too could be sent to the SIEM. In short, the work necessary for the environment to log to the SIEM

involved three separate management teams, hundreds of change management entries, and a bit of network / storage re-engineering to just begin the process of getting logs.

Don’t get me started on what it takes to log from AWS and O365.

So what is the first thing to note about the true cost of owning and operating a SIEM? It was this: when

you determine a SIEM is necessary for your environment, your true cost of integration is directly

proportionate to the ITSM management cost of your entire IT infrastructure. It will take a strong IT

wizard at the level of your BMW mechanic, to get to all of the silos in your organization and talk

technical about the needs and methods to extract the logs that the system needs to work. It will take

time and effort to get things set up to log to the SIEM, and this is going to be a manpower initiative proportional to the complexity of your organization.

For starters.

Now we need to fill in the rest of the story for your TCO on this functionality!

Why? Because you’re growing and changing. You cloned your production database to upgrade your

back-end hardware and switched over the instance to a new TNS identifier for the database itself. It all

went smoothly. But the SIEM didn’t know this. And now your Oracle logs are gone. When you

upgraded the NAS mount for your logging host, and turned off the “RSyslogd” daemon to “RSync” the

old filesystem to the new one, you forgot to restart the daemon. The SIEM didn’t know this, and now

your UNIS logs are gone. You went with the newest cloud service provider to support your AntiVirus

implementation, and realize that you don’t have logs from the cloud to the SIEM. You’ll need to

provision a logging relay in the DMZ to get those logs into the SIEM, and… yes, now you no longer have

endpoint logging anymore.

And we haven’t even gotten to reporting and analysis yet.

There was a reason to put a SIEM into the environment, and whether it was for audit compliance,

centralized logging, automatic report generation of events, incident alerting to your SOC monitor, or

complex user analytics - once the data is in there, you have to be able to do something with it. And

once you start doing things with the data, you will need personnel to act and respond to the heuristics

coming out of that tool. Someone is going to review your “dial-in” VPN access logs to see if one UserID

is used to get on the network from Bangalore India at the same time that it was used to login in from

the Eastern Ukraine; someone is going to figure out if 10 or more password attempts on an account

within 1 minute is normal or not. In short, the SIEM will deftly be able to produce this output for you,

but someone has to know enough about your global IT footprint to act upon it. And they should also be

able to act upon data anomalies in those report – did the logging on the IPS get turned off for a reason, or did some hacker turn it off to cover his tracks. They need to be fairly IT knowledgeable.

The true cost of owning a SIEM? Wisdom. Your management should recognize the high level of IT

personnel resources needed to maintain this component, and invest in that personnel. The return is

enormous. But directly proportional to this personnel investment. Your environment is dynamic with a

lot of moving IT parts. The SIEM has to keep up, it also has to be maintained with patching and

hardware refresh. You will need some part of personnel to maintain and expand the data flows as they

change and grow. You will need some part of personnel to analyze the data, and update the reporting.

And while this may no longer require your BMW mechanic, it is certainly going to require a good garage

mechanic. In other words, you need to staff accordingly, provision the manpower, and pro-actively

monitor your data sources daily. And of course, only you have the resources for that.

There are multiple ways to accomplish this. One option is to outsource your SIEM completely to the

cloud. In this model you get what you pay for; a logging repository with a helpdesk to turn to when you

figure out what changes you need to have made. Another option is to keep the SIEM on your premises

but have high value alerts monitored 7x24 by a MSSP; you are still responsible for making sure the

event information makes it to the MSSP, and without business context and intelligence you will still

with lots of false positives from your provider. The third option is to keep the SIEM on premise, have a

trusted partner, like ThetaPoint, care for the SIEM at your location, allowing you to focus on the high

value requirements of the SIEM, be it use case/report creation/monitoring, or incident handling. In this

scenario, you have complete control over the business context and intelligence, and eliminate the

headaches and burden associated with care and feeding of your SIEM. Regardless of your path, the

most important thing to remember as you look at the true cost of your SIEM is to make sure you have a

desired state and that everything that you do supports that effort. If not, you will end up with a science project that will need to be continually justified and a cost model that can never be supported.