The Foundation for Operational Success The value of log data technology for today’s telecommunications company

w h i t e p a p e r

Table of Contents

3 | Introduction

4 | Coping with Data Extremes in the Network Environment

5 | Log Data: Ground Zero for Effective Network Management

6 | The Capabilities of a Comprehensive Log Data Solution

8 | Next Stop: TIBCO LogLogic

w h i t e p a p e r

3

For telcos, network infrastructure is the foundation of business. If it’s not up to the task, nothing else matters. Customer engagement strategies, data-driven pricing models, more efficient billing and collection systems, and new product offerings can optimize business performance only if network operations can effectively handle an enormous volume of voice and data traffic 24x7.

Collecting, centralizing, and intelligently consuming (analyzing and using) the most basic network data – log data – are the keys to ensuring stable network operations. And telco networks generate massive volumes of log data – big data. What’s more, extracting its value and using the data proactively are even more challenging than ingesting and storing it.

Across a distributed big data environment where billions of logs are generated daily, a comprehensive, telecom-class log management solution can mitigate risk, ensure service availability, and promote operational efficiency.

IntroductionAll businesses face two customer challenges: acquiring new customers and retaining the ones they have. That’s certainly true of telecommunications companies. To read various analysts and consultants, telcos are in dire straits. A 2012 publication by Accenture directed at communication service providers, a category to which telcos certainly belong, begins, “Ubiquitous technology innovations, a structural evolution in customer demand, and competition from a variety of new players are dramatically reshaping the communications and media industries. In the process, these trends are pressuring communications service providers (CSPs) to bring new digital products and services to market, and do so more quickly than ever.”1

Of course, no matter who’s talking, this generic “table setter” quickly leads to advice on how to fend off competitors, squeeze more revenue from existing customers, price new services in a volatile market, and so forth. Much of this advice comes under the heading,

“improving customer engagement.”

1 Pierron, Jean-Marie, et al. (2012) How Communications Service Providers Can Transform Telecom Billing Operations to Support a New Convergent, Digital Business. Accenture.com. Retrieved from: http://www.accenture.com/SiteCollectionDocuments/PDF/Accenture_Communications_Research_How_Service_Providers_Can_Transform_Telecom_Billing_Operations.pdf

The Foundation for Operational Success The value of log data technology for today’s telecommunications company

w h i t e p a p e r

4

But often ignored or rarely discussed in much depth is what underlies a telco’s ability to offer more services and provide better service delivery. And the “what” is the “plumbing,” the network infrastructure – and understanding the massive volumes of data that various components of that infrastructure churn out every second. Network infrastructure and its data are not nearly as flashy as new customer engagement strategies or even re-engineered billing systems, but that’s the reality.

In fact, data and what it says about infrastructure performance should be the operational foundation for today’s successful carrier-class telco. The remainder of this white paper will take a look at the extreme data environments telcos face, focusing on the most basic of data – log data, which is frequently underappreciated and underutilized. Then it will describe the benefit of using log data effectively and what a robust log data management solution should provide.

Coping with Data Extremes in the Network EnvironmentMake no mistake; log data is big data. Of the four telco network components—user devices, access points, backhaul, and core – three generate massive volumes of log data:

•  End user devices – mobile phones, desktops, laptops, tablets. One market alone may serve hundreds of thousands of devices, with 50M text-based device records per day.

•  Access points – cell sites, switches, and cable modems. These generate text-based logs similar to call detail records (CDR).

•  Core – routers and switches that connect everything to the outside world, the Internet or other provider networks. A pair of routers can generate 1M log messages per second.

According to Gartner, a medium-sized enterprise creates 20,000 messages per second of operational data in its activity logs. That’s more than 150 GB of operational data in a single, eight-hour day. But that volume is trivial compared to what a telco generates in merely an hour.

As much as it may strain an IT infrastructure to ingest and store data at the velocity and volume of millions of messages per second, that’s not really the problem. Mining it for useful information and real-time insight – that’s the problem.

“Typically, nothing in a telco’s IT arsenal can deal with the volume of log data it generates so they can look at it in any meaningful way. They’ve got the information. And it could be used to make intelligent decisions. But they have to have something to mine that data.”

— Clive McDonald, solution architect, TIBCO LogLogic

w h i t e p a p e r

5

Log Data: Ground Zero for Effective Network ManagementLog data is like a stream of non-stop tweets coming from nearly every IT asset in your infrastructure. It provides an immutable fingerprint of user and system activity. Log data can reveal something as simple as a failed logon. Or, where there are significant deviations from established baselines, it may indicate a runaway application or an actual security breach.

By mining this data and managing it proactively instead of ignoring it until something goes wrong, organizations can mitigate risk, ensure service availability, and promote operational efficiency. Most organizations could use their log data more effectively. But for telcos it is of vital importance. Because for telcos, their network is their business. Telcos can use log data to:

•  Perform root cause analysis

•  Effectively plan capacity

•  Manage security and compliance

Log data also contributes to or supports the key performance indicators (KPIs) that telcos monitor in every area of network operations, such as:

1 Connection establishment

•  Call setup success rate

•  Standalone dedicated control channel congestion

•  TDH (hub) congestion

2 Connection maintenance

•  Dropped-Call Rate (DCR)

•  Worst affected cells for call drop rate

•  Connection with good voice quality

3 Service performance

•  Round trip travel (RTT) delay

•  Application throughput

•  Call setup time

w h i t e p a p e r

6

The Capabilities of a Comprehensive Log Data SolutionIdeally, an effective telecom-class log management methodology should be able to collect, centralize, and consume log data in a distributed big data environment where billions of logs are generated daily.

CollectCollection entails gathering structured, unstructured, and semi-structured records of human and machine activity in a large variety of formats from network and security devices, servers, applications, point-of-sale systems, and any other source physical, virtual, on site, or in the cloud.

A log data collection solution should collect 100 percent of log data, 100 percent of the time – using agent or agentless methods – from any device, application, platform, or process. Collection tools must enable IT administrators to easily import data from all log sources. And they should handle LAN or WAN traffic with equal ease, smoothing network chatter to deliver a more predictable network profile. Collection technologies should be:

Flexible and Interoperable•  Convert custom protocols to industry standard syslog and forward to third-

party technologies

•  Adapt to time zones and scheduling

•  Compensate for unstable pipes

Secure•  Protect data from unauthorized viewers via encryption and authentication

Scalable•  Ingest, index, and store hundreds of thousands of messages per second using a

distributed architecture that scales across deployments

•  Scale to thousands of remote locations

•  Optimize bandwidth by filtering at the source and compressing payloads

CentralizeTo deliver insight into the operation of IT infrastructure, log data solutions should include a fast, scalable, centralized IT data warehouse, storing raw and normalized log data, protecting its chain of custody, and ensuring immutability. Replacing point-to-point connections between data sources with a centralized, virtual pool of data that can be reused across applications enables an historical, end-to-end view of infrastructure for in-depth analysis of past and present events and their root causes. Centralizing IT data also eliminates duplicate application silos and reduces network bandwidth demands. An IT data warehouse needs to:

Protect Data in Real Time•  Enable real-time alerts for events such as password brute force, denial of service,

account hijacking, and virus outbreaks

•  Support precise recommendations for threat removal by correlating internal and external patterns for known and unknown threats

According to researchers with the Institute for Applied Network Security (IANS), collecting log data is just the first step in creating value from it:

“Making sense of [logs from web applications, middleware, custom backend applications, and databases] requires an ability to normalize the collected data, correlate it, report on it, and send actionable alerts for issues related to security and malicious activity like APTs, but also for availability, work flow issues, and help desk efficiency.”

w h i t e p a p e r

7

Speed Resolution•  Resolve underlying security issues within seconds by tracking all user activity using

advanced root-cause analysis and forensics workflow

•  Integrate easily with third-party analytic solutions or service-desk management systems

See Everything•  Track all user activity including geography, access rights, configuration changes, asset

misuse, and proxy bypass using browser-based security information management

•  Provide configurable reports for access control, policy information, and network, database, IDS, email, Web, and legacy system activity

Scale as Needed•  Allow centralized data aggregation and search, and high-capacity storage

•  Deliver real-time access to external data stores (NAS and SAN) with fully automated data retention and deletion policies

Support Open Forwarding•  Simultaneously forward data to multiple analytics engines regardless of vendor

•  Protect investments in existing analytics applications

Simplify Management and Control•  Manage and administer log solution components as a peer group

•  Log all administrative actions including those made within the log management solution

ConsumeHistorical and real-time log data is extremely useful for creating reports and machine learning alerts. It supports forensic incident investigation, root-cause analysis, and external pattern correlation for threat recognition. To simplify compliance reporting, log data can be mapped to the control objectives of regulatory mandates such as the Payment Card Industry Data Security Standard (PCI-DSS). And, using web services APIs, log data can be made available to other systems and partner applications, such as customer service representative call screen interfaces and third-party reporting and analysis tools. To unlock the intelligence residing in log data, comprehensive solutions should feature:

•  Advanced analytics including browser-based dashboards, Google-like search, detailed reports, and real-time alerts

•  A real-time forensics engine that can quickly isolate the root cause of an incident within seconds, not days

•  Easy-to-produce audit quality compliance reporting with pre-packaged reports and alerts for major mandates (COBIT, FISMA, HIPAA, ISO, ITIL, NERC, PCI-DSS, SOX)

•  A configurable workflow manager that simplifies the review and audit process for adherence to internal and external policies

w h i t e p a p e r

8

exported02Jul2013

Next Stop: TIBCO LogLogicTIBCO LogLogic is the leader in telco-scale solutions for comprehensive, high-performance log data aggregation, analysis, and retention. It provides the industry’s most scalable log and security intelligence platform on premise and in the cloud, transforming data silos from physical and virtual devices into knowledge and insight. More than 1,250 customers worldwide entrust their most sensitive log and IT data to LogLogic award winning products for compliance, security, and network operations. To learn more about using TIBCO LogLogic technology to optimize your network infrastructure, please visit us online at www.tibco.com/loglogic, contact your value-added reseller, or contact us at www.tibco.com/company/contactus.jsp

w h i t e p a p e r

TIBCO Software Inc. (NASDAQ: TIBX) is a provider of infrastructure software for companies to use on-premise or as part of cloud computing environments. Whether it’s efficient claims or trade processing, cross-selling products based on real-time customer behavior, or averting a crisis before it happens, TIBCO provides companies the two-second advantage® – the ability to capture the right information, at the right time and act on it preemptively for a competitive advantage. More than 4,000 customers worldwide rely on TIBCO to manage information, decisions, processes and applications in real time. Learn more at www.tibco.com.

©2013, TIBCO Software Inc. All rights reserved. TIBCO, the TIBCO logo, and TIBCO LogLogic are trademarks or registered trademarks of TIBCO Software Inc. in the United States and/or other countries. All other product and company names and marks in this document are the property of their respective owners and mentioned for identification purposes only.

www.tibco.comGlobal Headquarters3307 Hillview AvenuePalo Alto, CA 94304

Tel: +1 650-846-1000 +1 800-420-8450Fax: +1 650-846-1005