ibm security guardium v9.5 features and updates tech …€¦ · · 2015-03-24value proposition...
TRANSCRIPT
© 2015 IBM Corporation
IBM Security
1© 2015 IBM Corporation
IBM Security Guardium v9.5
Features and Updates Tech Talk
Luis Casco-AriasProduct Manager
IBM Security Guardium
Also with support from: Guy Galil, Lior Solomon and Oded Sofer
© 2015 IBM Corporation
IBM Security
2
This tech talk is being recorded. If you object, please hang up and
leave the webcast now.
We’ll post a copy of slides and link to recording on the Guardium
community tech talk wiki page: http://ibm.co/Wh9x0o
You can listen to the tech talk using audiocast and ask questions in
the chat to the Q and A group.
We’ll try to answer questions in the chat or address them at
speaker’s discretion.
– If we cannot answer your question, please do include your email
so we can get back to you.
When speaker pauses for questions:
– We’ll go through existing questions in the chat
Logistics
© 2015 IBM Corporation
IBM Security
3
Link to more information about this and upcoming tech talks can be found on the InfoSphere Guardium developerWorks
community: http://ibm.co/Wh9x0o
Please submit a comment on this page for ideas for tech talk topics.
April 7th, 2015: Encrypting Data at Rest in IMS using InfoSphere Guardium Data Encryption
Speaker: Dennis Eichelberger, IMS IT Specialist, IBM
Register here! https://ibm.biz/BdE7tR
Reminder: Upcoming Guardium Tech Talks
April 22nd: Part 2 – Overview of InfoSphere Guardium Data Encryption
Speaker: Ernie Mancill, Executive IT Specialist, IBM
Register here! https://ibm.biz/BdXxhx
© 2015 IBM Corporation
IBM Security
4
Guardium community on developerWorks
bit.ly/guardwiki
Right nav
© 2015 IBM Corporation
IBM Security
55
Information, training, and community
InfoSphere Guardium Tech Talks – at least one per month. Suggestions welcome!
InfoSphere Guardium YouTube Channel – includes overviews, technical demos, tech talk replays
developerWorks forum (very active)
Guardium DAM User Group on Linked In (very active)
Community on developerWorks (includes discussion forum, content and links to a myriad of sources, developerWorks articles, tech talk materials and schedules)
Guardium on IBM Knowledge Center (was Info Center)
Deployment Guide for InfoSphere Guardium Red Book
Technical training courses (provided by IBM Business Partners)
InfoSphere Guardium Virtual User Group. Open, technical
discussions with other users. Not recorded!
Send a note to [email protected] if interested.
5
© 2015 IBM Corporation
IBM Security
6
Agenda
Guardium 9.5 ContentNew Platforms S-TAP automatic load balancer
• New GuardAPI commands
Quick Search Infrastructure Update •Enterprise Scope •Investigation Dashboard •Topology Navigator
Outlier Detection enhancementUpgrade path recommendations
Demo
© 2015 IBM Corporation
IBM Security
7
InfoSphere Guardium – Data Security & Privacyone-stop-shop to protect against unauthorized access to data, and reduce the cost of compliance
Value Proposition•Prevent data breaches•Ensure data Privacy•Reduce the cost of compliance•Identifying security risks•Enable secure sharing of data
On Premise On Cloud
Data at Rest
Stored(Databases, File Servers, Big Data, Data
Warehouses, Application Servers, Cloud/Virtual ..)
Over Network(SQL, HTTP, SSH, FTP, email,. …)
Data in Motion
Data Repositories
Sensitive Documents
© 2015 IBM Corporation
IBM Security
88
Expand Platform Coverage
InfoSphere Guardium v9.5 (1Q15)
DAM pricing and packaging changes
•DAM Big Data new part
•DAM Data Warehouse new part New Platform Support: AsterDB, Pivotal, MariaDB, MS APD/PDW
Updates: Teradata 15, GreenplumDB 4.3, Ubuntu 14
Enhance
Activity Monitoring
Scalability &Reduce Op. Costs
Automatic STAP load balancing (phase 1)
Quick Search analytics improvement
•Investigation Dashboard
Outlier Detection performance and scoring
New monthly based pricing
Cloud deployment
Security Integrations New GuardAPI commands
VulnerabilityAssessment
VA parts consolidation
Entitlement Reporting included in base services
Monitoring Teradata encrypted logon and traffic
(A-TAP)
* Adjust appliance pricing currency
© 2015 IBM Corporation
IBM Security
10
Comprehensive support for structured and unstructured sensitive data:Databases, Data Warehouses, Big Data Environments and File Shares
InfoSphere BigInsights
Guardium
DATABASES
Exadata
D A T A B A S E
HANA
Optim
Archival
Siebel,
PeopleSoft,
E-Business
Master Data
Management
Data
Stage
CICS
z/OS Datasets
Pure Data Analytics
FTP
with BLU Acceleration
DB2®
with BLU Acceleration
DB2®
DB
© 2015 IBM Corporation
IBM Security
11
Netezza
Teradata
2011
Netezza
Teradata
BigInsights
Cloudera
2012
Netezza
Teradata
BigInsights
Cloudera
MongoDB
CouchDB
Cassandra
GreenplumHD
HortonWorks
2013
SAP/HANA
GreenplumDB
V8
V9
V9p50
V9.1
Netezza
Teradata
BigInsights
Cloudera
MongoDB
CouchDB
Cassandra
GreenplumHD
HortonWorks
2014
SAP/HANA
GreenplumDB
AsterDB
Pivotal
V 9.1 updates
Platform support Updates in v9.5
• Greenplum DB 4.3 • Hortonworks HDP 2.2 •Teradata 15
New Platform Support in v9.5 :
• Aster DB• MariaDB• MS APD/PDW• Pivotal
OS Support Updates in v9.5
•Ubuntu 14
New Platforms and Updates in v9.5
© 2015 IBM Corporation
IBM Security
13
Installing an STAP on a DB server requires the customer to assign a managed unit
appliance for STAP to connect to.
On large sites (> 100’s of managed units), finding an available managed unit requires
skill and time.
Dedicating a collector to an STAP involves periodically consolidating several
indicators (per collector).
The 'STAP Load Balancer’ automates this process
Overview
© 2015 IBM Corporation
IBM Security
14
Load Balancer Process Running on the Central Manager
Collects and maintains a 'Load Map' of all the Managed Units (MU).
Allocating least loaded Managed Unit to newly installed STAP(s).
CM
Load
Balancer
MU 3
MU 2
MU 1
DB
Server
Load MapMU 1 : LoadedMU 2 : VacantMU 3 : Loaded
Report load
STEP 2 : Allocate MU Request
STEP 4 : Connect to MU 2
STEP 1 : Install STAP
STEP 3 : Find least loaded MU
Periodic background activity
*MU generally represents a Collector
Load Balancer Highlights
© 2015 IBM Corporation
IBM Security
15
Load Collection Mechanism
Periodically collects load information from all the managed units.
Load information relies mostly on 'UNIT UTILIZATION LEVELS' and
'STAP INFO' reports/Tables.
Load Map
Specific load indicators are used to build a 'Load Map' in the Load
Balancer memory.
MU Allocation
Upon receiving a request from STAP to allocate an MU, the Load
Balancer will query the Load Map, and assign an MU for the requesting
STAP.
Load Balancer Function - General
© 2015 IBM Corporation
IBM Security
16
Default dynamic load collection Intervals.
The more MU‘s, the bigger the load collection intervals. (Controlled by the
parameter DYNAMIC_LOAD_CHECK_INTERVAL)
Load indicators are used in order to estimate the load level on each MU over
a period of time.
Load indicators are defined by the STAP Load Balancer parameter
LOAD_INDICATORS which is set by default with the following indicators:NO_OF_RESTARTS_UL (Number of Sniffer restart)
ANALYZER_QUEUE_UL (Size of Sniffer analyzer queue)
LOGGER_QUEUE_UL (Size of Sniffer logger queue)
MYSQL_DISK_USAGE_UL (Disk usage)
Load Sampling Interval is critical in order to get an accurate picture of the
load per MU. Larger load sampling interval means we can capture a more accurate snapshot of
load behavior (busy and non-busy hours) rather than capturing spikes.
Load indicators are defined by the STAP Load Balancer parameter
LOAD_SAMPLE_PERIOD which is set by default to 24 hours.
Load Balancer Function – Load Collection
© 2015 IBM Corporation
IBM Security
17
After evaluating the load from all load indicators, MU's are placed within the Load Map
in 3 load “levels” : NOT-LOADED (LOAD_LEVEL_1),
MEDIUM-LOADED (LOAD_LEVEL_2),
VERY-LOADED (LOAD_LEVEL_3).
In between load collections, the Load Map is dynamically updated upon each
allocation of an MU to a requesting STAP.
MU's will be replaced in a new load “level” if the number of STAPs per single MU
exceeds the threshold defined by the Load Balancer parameters: MAX_STAPS_PER_MU_THRESHOLD1 (default : 20), MAX_STAPS_PER_MU_THRESHOLD2 (default :40)
> grdapi get_stap_load_balancer_current_load_mapLOAD_MAP:
LOAD_LEVEL_1 MU's:
MU=qa-vm18.guard.swg.usma.ibm.com_5: C_STAPS=1, I_STAPS=0, C_LOAD_LEVEL=1, I_LOAD_LEVEL=1-->
MU=qa-vm18.guard.swg.usma.ibm.com_4: C_STAPS=1, I_STAPS=0, C_LOAD_LEVEL=1, I_LOAD_LEVEL=1-->
LOAD_LEVEL_2 MU's:
MU=qa-vm21.guard.swg.usma.ibm.com_4: C_STAPS=2, I_STAPS=2, C_LOAD_LEVEL=2, I_LOAD_LEVEL=1-->
MU=qa-vm22.guard.swg.usma.ibm.com_3: C_STAPS=2, I_STAPS=2, C_LOAD_LEVEL=2, I_LOAD_LEVEL=2-->
MU=qa-vm24.guard.swg.usma.ibm.com_6: C_STAPS=6, I_STAPS=6, C_LOAD_LEVEL=2, I_LOAD_LEVEL=2-->
LOAD_LEVEL_3 MU's:
MU=qa-vm22.guard.swg.usma.ibm.com_4: C_STAPS=2, I_STAPS=2, C_LOAD_LEVEL=3, I_LOAD_LEVEL=3-->
MU=qa-vm24.guard.swg.usma.ibm.com_4: C_STAPS=6, I_STAPS=6, C_LOAD_LEVEL=3, I_LOAD_LEVEL=2-->
MU=qa-vm24.guard.swg.usma.ibm.com_5: C_STAPS=6, I_STAPS=6, C_LOAD_LEVEL=3, I_LOAD_LEVEL=3-->
STAP REQUESTS CACHE:
1.1.1.1=1
LAST MU USED PER STAP CACHE:
1.1.1.1=qa-vm18.guard.swg.usma.ibm.com_6
C_STAPS : Current # STAPS assigned to MUI_STAPS : Initial # STAPS assigned to MUC_LOAD_LEVEL: Current Load Level of MUI_LOAD_LEVEL : Initial # load level of MU
Load Balancer Function – Load Map
© 2015 IBM Corporation
IBM Security
18
Load Balancer Function - Allocating MU's The Load balancer allows usage of groups in order to assign a pool of STAP(s) to a pool of MU.
Guarantee that not every available MU can be assigned to a requesting STAP. Caveat: Have to define an STAP IP in some group prior to installing it on the DB server
Upon receiving an MU allocation request from STAP, the load balancer will:1. Check if it's assigned to some group of MU. 2. If it find such a group, it will look in the load map for the least loaded MU in that
group. 3. Otherwise, the first available MU in the load map will be allocated.
Once an MU is allocated, the load balancer caches the last allocated MU for the requesting STAP.
If an additional request(s) comes from the same STAP, the load balancer will allocate a different MU (if there is one).
Allocated MU's are re-positioned at the end of Load Map “bucket” queue.If the allocated MU is of LOAD_LEVEL “2” or “3” (“Medium/High Load”), an event is going to be recorded in GDM_EXCEPTION table.
© 2015 IBM Corporation
IBM Security
19
STAP Load Balancer GRDAPI
Assign MU group to an STAP group
grdapi assign_stap_load_balancer_mu_tap_group muGroupName=<MU group>
stapGroupName=<STAP group>
Get Current Load Map
grdapi get_stap_load_balancer_current_load_map
Get Load Balancer parameters
grdapi get_stap_load_balancer_params
Set load balancer parameter
grdapi set_stap_load_balancer_param paramName=<param name>
paramValue=<param value>
Un-assign STAP group from an MU group
grdapi unassign_stap_load_balancer_mu_tap_group muGroupName=<MU
group> stapGroupName=<STAP group>
© 2015 IBM Corporation
IBM Security
20
STAP Load Balancer GUI – Associating Existing Groups
All Groups of type 'STAP' will be automatically displayed
All Groups of type 'Managed Unit' will be automatically displayed
© 2015 IBM Corporation
IBM Security
21
STAP Load Balancer GUI – Associating New STAP Group
A List of available STAP hosts will be automatically available for selection.
© 2015 IBM Corporation
IBM Security
22
STAP Load Balancer GUI – Associating New MU Group
All the managed units hosts will be automatically displayed
© 2015 IBM Corporation
IBM Security
26
Quick Search Infrastructure UpdateContinue to improve Quick Search Functionality in Collector environment, change infrastructure from
Lucene to Solr
Benefits :
Real Time Distributed Search
Performance Improvements
Built-in Analytics functions
Caveat :
This engine change may consume extra memory (Requires 24GB of RAM then 16 GB previously)
Investigation Dashboard – Technology PreviewProvides interrelated charts that help reveal patterns, anomalies, and relationships across your data
Best-practice view includes data source-to-user behavior, data source-by-time behavior, data source-to-source
program behavior, and other essential relationships.
© 2015 IBM Corporation
IBM Security
27
Enhancing Quick Search for Enterprise Wide Scope
• Search scope spanning Central Manager controlled environment– No need to understand or concern about topology, aggregation, or load balancing
schemes
– Search requests are sent to all nodes, results are gathered, consolidated, and sorted
according to the request, and then the results are displayed to the user centrally
• Leverage familiar Quick Search glass– Real-time
– Forensic
• New additional Investigation Dashboard– Pivot like facility to extract data activity insights
• focus on any specific context :specific data source, user, date, etc
– reveal patterns, anomalies, and relationships across your data
– best-practice default views:
• data source-to-user behavior
• data source-by-time behavior
• data source-to-source program behavior
• other essential relationships.
• New topology navigation facilitator– Narrowing of scope for search (local, distributed, selected sub-set)
Coming
soon
© 2015 IBM Corporation
IBM Security
31
Investigation Dashboard Overview
Business Goals
– Ability to grasp high-level view in a multi-dimensonal environment
– Quick way to inspect different aspects of a forensic case
– Browsing audit data related to a specific context
Investigation Dashboard Solution
– A dashboard containing a combination of interactive heat maps
– Leverage fast indexing engine in Quick Search
– Each heat map offers two dimensional view , related to other two dimensional
views
– Each change filters the other charts to drill down on a specific case
– Highly configurable
© 2015 IBM Corporation
IBM Security
32
Investigation Dashboard
•X-axis contains DB users•Y axis contains databases•Intersection shows usage of DB by each user•Color depth represents Intensity of usage•Hover cells for details•Click for interactive filtering
© 2015 IBM Corporation
IBM Security
34
Enhancements related to :
• New Features • New scorer
• Performance Improvement • User clustering• Multi-Threaded training• New cleaning mechanism for old data from the internal MySQL tables
• Fixes• Improved anomaly scoring• Filtering inputs according to the filtering patterns specified by the user in the GUI• Inputs collected during training are now analyzed
Outlier Detection
© 2015 IBM Corporation
IBM Security
36
Transition to Guardium V9.0p500 GPU
Target
Source
32-bit 32-bit 32-bit 64-bit 64-bit 64-bit
V8.2 V8.2 to V9.0p150 bundle patch
(IBM Fix Central) + V9.0p200 (or
later) 32-bit GPU patch
Rebuild with V9.0p200 (or later) 64-bit ISO
(IBM Passport Advantage)
N/A
V9.0 (GA) V9.0p200 (or later) 32-bit GPU patch
(IBM Fix Central)
Rebuild with V9.0p200 (or later) 64-bit ISO
(IBM Passport Advantage)
N/A
V9.0p02 V9.0p200 (or later) 32-bit GPU patch
(IBM Fix Central)
Rebuild with V9.0p200 (or later) 64-bit ISO
(IBM Passport Advantage)
N/A
V9.0p50 V9.0p200 (or later) 32-bit GPU patch
(IBM Fix Central)
Rebuild with V9.0p200 (or later) 64-bit ISO
(IBM Passport Advantage)
V9.0p200 (or later) 64-bit GPU
patch
(IBM Fix Central)
>V9.0p300 V9.0p300 (or later) 32-bit GPU patch
(IBM Fix Central)
Rebuild with V9.0p300 (or later) 64-bit ISO
(IBM Passport Advantage)
V9.0p300 (or later) 64-bit GPU
patch
(IBM Fix Central)
Newly built
appliance
Install V9.0p180 32bit ISO +
V9.0p200 (or later) 32-bit GPU patch
N/A V9.0p200 (or later) 64-bit GPU
patch
(IBM Fix Central)
Enterprise Upgrade Strategy
© 2015 IBM Corporation
IBM Security
37
Central Managerlevel
Collectorslevel
Aggregators level
Guardium agentslevel
Upgrading 64-bit system
V9.5p500 (or later) 64-bit
V9.1p300 64-bit
Upgrade
Upgrade
Live Update
Upgrade
Enterprise Upgrade Strategy
© 2015 IBM Corporation
IBM Security
38
External link for information on Guardium p500 GPU
http://www-01.ibm.com/support/docview.wss?&uid=swg27045362
Make the following selections on Fix Central:
Product Group: Information Management
Product: InfoSphere Guardium
Installed Version: 9.0/9.5
Platform: Linux
Heading: Appliance Patch (GPU and Ad-hoc)
Click "Continue", then select "Browse for fixes" and click "Continue" again.
How to Access p500 GPU
© 2015 IBM Corporation
IBM Security
40
Enterprise (and local) Search is based on Apache Solr
A widely used, highly scalable, open source
enterprise search platform from Apache.
Solr runs as a separate web application under tomcat.
Data is indexed and searched on shards.
A zookeeper is responsible for distributing the
indexing and the search queries to the relevant shards.
–Zookeeper runs on CM in a managed environment.
Search Infrastructure Update
© 2015 IBM Corporation
IBM Security
41
On a Guardium collector, data is extracted every 2 minutes
using the same mechanism (datamart) that was used in
previous Quick Search.
This data is indexed into 5 indexes.
Indexing is performed through the zookeeper –
Note that if CM is down indexing is not performed.
The actual index is local to the collector that collects the
data.
Old data is purged using the Purge Object mechanism,
default age is 3 days.
CLI command can be used to modify that age.
Data Flow
© 2015 IBM Corporation
IBM Security
42
On upgrade of a Guardium appliance to GPU500, if Quick
Search is enabled, hardware requirements for Enterprise
Search are met and old (Lucene) indexes are present there is
a transition period.
To prevent data loss in the transition period, data is indexed
on both the old engine and the new one.
Searches are performed using old engine.
A clear message is displayed stating "Upcoming new search
options, in ${date}."
The data is calculated to be the date when old index data is
purged and all data is indexed by new engine.
Transitioning to new Search
© 2015 IBM Corporation
IBM Security
43
Enterprise Search is not supported on 32 bit
Collectors.
A 32 bit CM can be the zookeeper when it manages
64 bit collectors as long as it complies with the
following HW requirements.
The hardware requirements for Enterprise Search are at
least 4 CPU cores and 24 Gig RAM on a collector
On a CM 4 CPU cores and 24 Gig RAM are required.
Hardware Requirements
© 2015 IBM Corporation
IBM Security
44
Solr uses port 8983 for communications. This port must be
bidirectionally open between CM and MUs.
The ports are opened in our internal firewall on registration to
CM and closed on un-registration.
The port is closed to the world by default to preserve the
security of our system.
search data.
Ports
© 2015 IBM Corporation
IBM Security
45
cli>grdapi enable_quick_search – if hardware requirements
are met then new search engine is enabled.
cli>grdapi disable_quick_search – Quick Search is disabled.
A new optional parameter all – if api function is invoked on
CM and all=true then Quick Search is disabled on all
managed units.
CLI commands