Eliminated 1,000+ hours of human labor spent on repetitive, low-value tasks.

Reduced MTTR by 90% with self-healing automation

Improved security through higher password management accuracy.

RESULTS

SUMMARY

The Liberty Mutual Insurance network security team uses SaltStack intelligent automation to create self-healing Juniper firewalls and orchestrate remediation of Splunk-created security events.

SOLUTION

● SaltStack Enterprise automation platform● Event-driven orchestration● Multi-site deployment● Splunk integration● SaltStack professional services

Liberty Mutual Insurance Customer SpotlightSelf-Healing Security Automation for Juniper Firewalls and Beyond

The Liberty Mutual Insurance (LMI) network security team manages approximately 100 Juniper firewalls, in addition to thousands of network devices from Cisco and Palo Alto. Before SaltStack, LMI relied on a process whereby Splunk would detect out-of-configuration Juniper firewalls and generate alerts, which then required manual intervention. Once detected, resolution took an average of 5 minutes per issue. At any given time, there could be 500 issues outstanding—resulting in over 40 hours of dedicated resolution work per week.

Greg Fraize and Chris Hutchins, two of LMI’s principal network security engineers, understood that the only path forward was automation and chose SaltStack Enterprise to dramatically improve his team’s efficiency and their ability to harden and protect the network.

Self-healing firewalls for auto remediation

For their first SaltStack project, The LMI network security team decided to use SaltStack event-driven detection and automation to auto-resolve firewall issues and maintain predefined security policies for over 150 Junos configuration options.

By replacing inconsistent bash and shell scripts with unified SaltStack automation, the team eliminated over 100 lines of code per firewall and reduced the time to detect and resolve issues by 90%—from 20 minutes down to 2.

SaltStack self healing automation dramatically improved LMI’s firewall hardening and security and allowed the network security team to shift from a manual, reactive approach of triage and mitigation to an automated, proactive system of finding and fixing issues in real time.

Using SaltStack Proxy Minions to control network hardware

In order to create a self-healing automation system to Juniper firewalls, LMI needed a way to maintain persistent visibility and control without having an agent on box. While SaltStack offers both agent and SSH-based agentless controls options, the team decided to utilize a third powerful option known as the SaltStack Proxy Minion.

Proxy Minions connect a device to SaltStack via API, allowing the user to take advantage of the SaltStack lighting fast and persistent event communication system without running a SaltStack Minion (agent) on box. While SaltStack SSH also waves the need for an agent, it operates by establishing a temporary SSH connection and is therefore incapable of detecting changes or providing automated response. SSH options are best suited for simple, ad-hoc automation (e.g. day 0 deployment).

SaltS

tack

Cus

tom

er S

potli

ght

01 “On average, within about two minutes SaltStack was able to see there was an issue, determine how to fix the issue... and then push that remediation back up to the Juniper firewalls. Whether I’m fixing one issue or 159, it’s two minutes because SaltStack fixes them all at once.”

Greg FraizePrincipal Network Security Engineer at Liberty Mutual

Figure A: SaltStack is the only automation platform that offers agent, agentless, and proxy agent control options for the management of every type of IT infrastructure at scale.

Improving visibility with SaltStack and Splunk

Once LMI had automated the control of their Juniper firewalls with SaltStack, the next step was to integrate SaltStack into their environment and use event-driven orchestration to extend and amplify existing tools—including Splunk, Slack, and BMC Remedy.

SaltStack is the only automation platform that maintains a persistent connection with each agent via a lightning-fast event bus. This allows the SaltStack Minion (or Proxy Minion) to detect the status of a device and pass that information back to SaltStack or third-party systems in real time.

The LMI team uses the Splunk integration to pass system information collected by SaltStack into Splunk and provide real-time device health metrics including box load per second, packets per second size, active sessions, and CPU load. With accurate, up-to-date system data flowing from SaltStack, LMI’s operations and leadership teams can quickly identify, troubleshoot, and fix issues before they have time to impact downstream users and processes.

This integration also allows the team to create Splunk alerts based on SaltStack logs, trigger Slack notifications to the team and create tickets in BMC Remedy.

SaltS

tack

Cus

tom

er S

potli

ght

02

Figure B: by expressing firewall configuration as code, LMI can quickly update global configuration and provide color-coded tracking via an internal repo for auditors and compliance teams.

“Firewalls-as-code” for change control and easy auditing.

Another material improvement to the LMI Juniper firewall management process was the ability to use simple YAML files to express firewalls as code and then store the resulting logs in an internal git repo.

Before SaltStack, the LMI team downloaded a configuration status text file for each firewall every night. If they needed to determine what changes were made, the team would need to manually sift through each file and compare them against the previous day. With SaltStack, LMI can now track exactly what changed. In addition, LMI audit and PCI teams can now access the repo portal to get instant visibility into the status of the system and use simple color coding to track changes made.

Figure C: The LMI team uses Splunk to track the time between when a change occurs on a Juniper router (blue) to when SaltStack detects that change (yellow) and automatically resolves it (red).

_time

12:00 AM 2:00 AM 4:00 AM 6:00 AM 8:00 AM 10:00 AM

JunosOperation

SaltCommitSaltReactor

What’s next for Liberty Mutual and SaltStack

Greg and Chris describe their work so far as a first small demonstration of the massive value SaltStack intelligent automation can deliver at LMI.

Here is a glance at a few of the projects they plan to tackle next:

● Automating Palo Alto Firewalls and F5 load balancers. After a successful deployment of SaltStack onto Juniper firewalls, the team has decided to bring their Palo Alto and F5 resources under SaltStack control as well. In addition to the same types of checks and remediations currently being performed, LMI will also use SaltStack to manage the virtual machines on which the Palo Alto firewalls reside.

● Password automation and coordination across Cisco, Juniper, and Palo Alto. Currently, passwords are generated by CyberArk and Cisco Stealthwatch and then manually applied to the network devices, resulting in 8-16 hours of human labor per week and the potential for human error. The LMI network security and operations teams will use SaltStack orchestration to coordinate the generation of passwords and ensure they are applied correctly across all endpoints.

● Additional integrations and service ticket management. At present, LMI is leveraging the existing Splunk / Remedy connection. Moving forward, LMI plans to integrate SaltStack directly into more of their existing stack, including the automated creation, disposition, and resolution of Remedy tickets with SaltStack.

For Greg, Chris and the rest of the LMI team, the combination of SaltStack Enterprise software, training, and professional services and partner support are helping drive the future of network security at LMI.

SaltS

tack

Cus

tom

er S

potli

ght

© Copyright SaltStack, Inc. 2019

SaltStack, Inc.2801 N Thanksgiving Way, #150Lehi, UT 84043USA

+44 7771 812188 info@saltstack.comwww.saltstack.com

Produced in the United States of America THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. SaltStack products are warranted according to the terms and conditions of the agreements under which they are provided. Statements regarding the future direction and intent of SaltStack are subject to change or withdrawal without notice, and represent goals and objectives only. ♲ Please Recycle