network management megatrends 2016: hybrid cloud, network analytics and the internet of things

IT & DATA MANAGEMENT RESEARCH,

INDUSTRY ANALYSIS & CONSULTING

Network Management

Megatrends 2016Managing Networks in the Era of the Internet of Things,

Hybrid Clouds and Advanced Network Analytics

Shamus McGillicuddy

[email protected]

@shamusEMA



mailto:[email protected]


INDUSTRY ANALYSIS & CONSULTING© 2016 Enterprise Management Associates

Today’s Presenter

2

Shamus McGillicuddy, Senior Analyst, EMA

Shamus has more than ten years of experience in the IT industry,

primarily as a journalist covering the network infrastructure

market. At Enterprise Management Associates (EMA), he is the

senior analyst for the network management practice.

Prior to joining EMA, Shamus was the news director for

TechTarget's networking publications. He led the news team's

coverage of all networking topics, from the infrastructure layer to

the management layer. He has published hundreds of articles

about the technology and competitive positioning of networking

products and vendors. He was a founding editor of TechTarget's

website SearchSDN.com, a leading resource for technical

information and news on the software-defined networking industry.



Logistics for Today’s Webinar

3

An archived version of the event recording will be

available at www.enterprisemanagement.com

• Log questions in the Q&A panel located on the

lower right corner of your screen

• Questions will be addressed during the Q&A

session of the event

> QUESTIONS

> EVENT RECORDING

http://www.enterprisemanagement.com/



Network Management

Megatrends 2016Managing Networks in the Era of the Internet of Things,

Hybrid Clouds and Advanced Network Analytics

Shamus McGillicuddy

[email protected]

@shamusEMA



mailto:[email protected]



Sponsors

5


INDUSTRY ANALYSIS & CONSULTING© 2016 Enterprise Management Associates, Inc.

Network Management Megatrends 2016: Agenda

• Setting the Stage: The Managed Environment

• The Megatrends

– The Internet of Things

– Hybrid Cloud

– Advanced Network Analytics

– Overcrowded Network Management Toolsets

– Increased Network Management Outsourcing

• Emerging Network Management Requirements

and Challenges

• Wrap-Up and Q&A

6



– North American networking pros and chain

of command management/execs

– 150 total participants

– 100% involved in choosing, implementing

and/or using network management tools

– Three significant verticals

• Financial (17%)

• Software (11%)

• Retail (11%)

– Mixed organizational sizes (# of employees)

• 28% small (250-999)

• 38% medium (1,000-4,999)

• 34% large (5,000 and up)

Recognize and investigate major

shifts in the managed

environment, network

management technologies, and

network management

organizations within enterprise

and government settings

© 2016 Enterprise Management Associates

Network Management Megatrends 2016:

Research Focus and Demographics

> DEMOGRAPHICS

7

> RESEARCH FOCUS



Setting the Stage:

The Managed Environment



Number of physical network devices

aligns with size of company

9 © 2016 Enterprise Management Associates, Inc.

17%

19%

33%

19%

7%

5%

0%

0%

0%

0%

14%

26%

23%

21%

9%

4%

2%

2%

0%

10%

6%

22%

20%

16%

6%

12%

10%

LESS THAN 100

100 TO 250

251 TO 500

501 TO 1,000

1,001 TO 2,500

2,501 TO 5,000

5,001 TO 7,000

7,001 TO 10,000

MORE THAN 10,000

Small (250 to 999) Medium (1,000 to 4,999) Large (5,000+)

# of employees

# of physicalnetworkdevices



Small firms have fewer virtual network devices relative to

physical ones; Large firms have more virtual devices


26%

31%

24%

12%

0%

2%

0%

0%

5%

5%

14%

33%

9%

19%

14%

5%

0%

0%

6%

10%

8%

14%

22%

22%

12%

0%

8%

LESS THAN 100

100 TO 250

251 TO 500

501 TO 1,000

1,001 TO 2,500

2,501 TO 5,000

5,001 TO 7,000

7,001 TO 10,000

MORE THAN 10,000


# of employees

# of virtualnetworkdevices



Big Data and IoT data emerge

as significant sources of traffic


66%

63%

59%

58%

57%

57%

55%

46%

42%

41%

1%

SECURE WEB/HTTPS

EMAIL/COLLABORATION

STORAGE BACKUP/RESTORE

BIG DATA COLLECTION/BACKUPS/ANALYSIS

WEB/HTTP

VIRTUAL/HOSTED DESKTOP (VDI)

INTERNET OF THINGS APPLICATIONS AND DATA

VOIP

CUSTOM/PROPRIETARY APPS

STREAMING MEDIA

OTHER

In the course of an average day, which of the following traffic types are present in significant amounts on your network?

40.98%of all network traffic

originates from

external cloud

applications (SaaS,

IaaS, etc.)



Megatrends #1:

The Internet of Things



Most enterprises network teams are supporting

Internet of Things initiatives

• 87% of network teams are providing connectivity to Internet of Things Initiatives

– 47% support IoT on the corporate network

– 22% operate a separate IoT network

– 17% support IoT with a combination of the corporate network and a dedicated IoT network

• Organizations that are implementing IoT have more than one IoT initiative

– 47% have two

– 23%have three

– 15% have four or more

• 46% of networking teams take a leading role in planning and implementing ALL IoT initiatives

13



Top IoT applications pursued by enterprises


45%

35%

32%

32%

31%

29%

24%

18%

15%

15%

11%

0%

NETWORKS AND COMMUNICATIONS

SECURITY AND PUBLIC SAFETY

LOGISTICS AND TRANSPORTATION

RETAIL

BUILDINGS AND FACILITIES

CONSUMER GOODS

INDUSTRIAL CONTROLS (MANUFACTURING, OIL AND GAS, ETC.)

HEALTH AND BODY

MUNICIPAL AND UTILITIES (SMART CITIES, SMART GRID, ETC.)

ENVIRONMENTAL MONITORING (FOREST FIRE DETECTION, AIR POLLUTION, ETC.)

AGRICULTURE

OTHER



Networking pros deploying new security controls, IoT

device management & network upgrades to support IoT


55%

48%

47%

43%

40%

33%

32%

11%

0%

NEW SECURITY CONTROLS

NEW IOT DEVICE MANAGEMENT SYSTEMS

BANDWIDTH UPGRADES

NEW CONNECTIVITY

NEW IOT PERFORMANCE MONITORING AND MANAGEMENT SYSTEMS

NEW IOT NETWORK PROTOCOLS

APPLICATION OF ANALYTICS TO IOT PERFORMANCE AND AVAILABILITY

APPLICATION OF INVENTORY AND DISCOVERY FOR LIFECYCLE ASSET IOT SUPPORT

OTHER

Have you implemented any of the following in support of your organization's Internet of Things initiative?



Top IoT networking challenges: Lack of monitoring

and security controls for “things”


44%

38%

36%

35%

33%

28%

28%

17%

12%

2%

LIMITED MONITORING OF CONNECTED DEVICES

LACK OF SECURITY CONTROLS ON CONNECTED DEVICES

LIMITED REMOTE NETWORK CONNECTIVITY

LACK OF INDUSTRY STANDARD IOT PROTOCOLS

NETWORK TRAFFIC CONGESTION

INSUFFICIENT MANAGEMENT INTERFACES ON CONNECTED DEVICES

PROCESS AND POLITICAL ISSUES IN WORKING WITH OTHER DOMAINS

PROCESS AND POLITICAL ISSUES IN WORKING WITH BUSINESS/ENTERPRISE CONSTITUENTS

LACK OF EFFECTIVE MEASUREMENT FOR IOT SUCCESS

OTHER



Megatrends #2:

Hybrid Cloud



Hybrid cloud: A cloud computing

environment that combines

private, internal cloud

infrastructure with public,

external cloud infrastructure,

with connectivity and

orchestration across internal

and external cloud domains

Enterprises are busy with hybrid cloud

18 © 2016 Enterprise Management Associates, Inc.IT & DATA MANAGEMENT RESEARCH,


• 35% have completed a production

hybrid cloud deployment

• 35% are in the process of deployinga production hybrid cloud

• 20% plan to deploy a hybrid cloud

into production within 24 months

90% of enterprises have

specific hybrid cloud

deployment plans



Network teams deploying SDN overlays & performance

management software to support hybrid cloud


55%

51%

32%

31%

30%

28%

28%

15%

0%

NETWORK VIRTUALIZATION OVERLAYS

NEW NETWORK PERFORMANCE MONITORING AND MANAGEMENT SOFTWARE

COMMERCIAL OPENSTACK NETWORKING PLUG-INS/APIS

NEW NETWORK ORCHESTRATION SOFTWARE

BARE-METAL/WHITE-BOX SWITCHING HARDWARE AND SOFTWARE

OPENSTACK NEUTRON

NEW PRIVATE WAN LINKS

NEW INTERNET CONNECTIVITY

OTHER



Hybrid cloud networking challenges: Large and small

enterprises face different problems


21%

18%

24%

12%

35%

26%

32%

29%

6%

3%

21%

26%

23%

25%

21%

28%

34%

17%

4%

0%

34%

30%

30%

28%

26%

24%

24%

24%

2%

0%

LACK OF LOGICAL LAYER 2 CONNECTIVITY AND SHARED IP RANGES BETWEEN INTERNAL AND EXTERNAL CLOUD ENVIRONMENTS

LACK OF END-TO-END, MULTI-SITE NETWORK VISIBILITY AND TROUBLESHOOTING CAPABILITIES

NETWORK LATENCY BETWEEN INTERNAL AND EXTERNAL CLOUD RESOURCES

INCOMPATIBLE SECURITY DOMAINS BETWEEN CLOUD ENVIRONMENTS

LACK OF MONITORING AND TROUBLESHOOTING OF INTERCONNECTIONS BETWEEN PUBLIC AND PRIVATE CLOUD ENVIRONMENTS

COMPLEXITY OF MULTI-SITE NETWORK ORCHESTRATION

COMPLEXITY OF PROVISIONING INTERCONNECTIONS BETWEEN PUBLIC AND PRIVATE CLOUD ENVIRONMENTS

CLOUD APPLICATIONS NOT PROPERLY ARCHITECTED FOR HYBRID CLOUD NETWORKS

DON'T KNOW

OTHER


# of employees



Megatrends #3:

Advanced Network Analytics



Advanced network analytics

involve the use of various

analytical techniques on

multiple types of data extracted from network

infrastructure for the purposes of

improved network engineering &

operations and better alignment

with the business

• 49% of enterprises are doing this today– 40% plan to

implement this within 12 months

• 73% of these initiatives are led by the network infrastructure team– 17% are led by a

separate analytics team

Half of network teams have an active,

advanced network analytics initiative




Network flows and network security data are most

important sources for advanced analytics

• Top infrastructure data included in network analytics initiatives:

– Network flow data (NetFlow, IPFIX, etc.): 49%

– Network security data: 48%

– Interpreted packet flow (packet metadata) 39%

• Large companies (47%) especially use interpreted packet flows

– Log data (38%)

• Small companies (47%) especially use logs

– Time series monitoring data (SNMP, WMI, etc.): 37%

23



Network security monitoring is the top use case for

advanced network analytics


38%

30%

27%

25%

24%

22%

17%

17%

15%

14%

13%

11%

NETWORK SECURITY MONITORING

NETWORK OPTIMIZATION

BUSINESS PROCESS OPTIMIZATION

PREDICTIVE NETWORK ANALYSIS

PREDICTIVE SECURITY ANALYTICS

BUSINESS PROCESS MONITORING

TRANSACTION-AWARE APPLICATION PERFORMANCE/OPTIMIZATION

GRC (GOVERNANCE, RISK MANAGEMENT, AND COMPLIANCE) ANALYTICS

CROSS-DOMAIN/MULTI-LAYER NETWORK IMPACT ANALYSIS

CHANGE IMPACT ANALYTICS

NETWORK SIMULATION

IF/THEN CAPACITY PLANNING AND OPTIMIZATION



Challenges to success with

advanced network analytics

• The biggest hurdle: 30% struggle with process and political issues in sharing data effectively among relevant stakeholders– This is a bigger issue for large enterprises (37%)

• 25% say network capacity is strained by collection and backup of network data

• 24% struggle with normalizing heterogeneous network data types

• 23% say their network monitoring vendors do not allow them to export data for third-party analysis– A major problem for large enterprises (35%)

25



Megatrends #4:

The Impact of Overcrowded

Network Management Toolsets



The typical enterprise has between 4 and 15 network

monitoring and troubleshooting tools in active use


5%

1%

15%

19%

24%

17%

11%

1%

5%

1

2

3

4 TO 5

6 TO 10

11 TO 15

16 TO 20

21 TO 25

MORE THAN 25

Network monitoring tools in active use



Large network monitoring toolsets correlate with

ineffective network problem detection

• 40.07% of all network problems are experienced and

reported by end users before network operations is

aware of them

• Size of network monitoring toolkit influences this statistic

28

Number of Network Monitoring and Troubleshooting Tools in Use

Percentage of Network ProblemsFirst Reported by End Users

1 to 3 tools 28.97%

4 to 10 tools 35.72%

11 or more tools 52.23%


INDUSTRY ANALYSIS & CONSULTING29 © 2016 Enterprise Management Associates, Inc.

Organizations with smaller tool sets spend less time on

reactive troubleshooting, more time on other projects

33.21% 34.18%32.61%

36.27% 37.03%

26.70%

38.06%

32.42%

29.53%

REACTIVE TROUBLESHOOTING PROACTIVE PROBLEM PREVENTION ALL OTHER TASKS/PROJECTS

What percentage of total work time does your organization's network team spend per week on the following activities?

1 to 3 tools 4 to 10 tools 11 or more tools



Large toolsets lead to unstable networks;

Organizations are adopting policies to consolidate

• 34% of organizations with 11+ network monitoring tools

experience several network-related outages per day

– Just 6% of organizations with 1 to 3 tools experience multiple

network outages a day

• 18% of organizations with 1 to 3 network monitoring tools say

they “almost never” experience network-related outages

– Another 21% experience a network outage just “once or twice a

year”

• 60% of organizations with 11+ tools are actively trying to

consolidate network management tools wherever possible

• 52% of organizations with 1 to 3 tools and 45% with 4 to 10

tools try to maintain their current number of tools in use

30



Megatrend #5:

Network Management

Outsourcing on the Rise



The majority of enterprises are outsourcing aspects of

network monitoring and management to an MSP

• 51% of enterprises outsource some aspect of

network monitoring and management

– Large enterprises (65%) are the biggest outsourcers

– Small (40%) and medium (47%) are also outsourcing in

significant numbers

– In 2014, only 36% of enterprises outsourced network

management to an MSP

• 30% of outsourcers maintain network monitoring

tools in order to oversee MSP activity

32



Top network management functions

outsourced to MSPs

• The top five network management functions

outsourced:

– 24x7 network health monitoring: 40%

– Remote site monitoring: 38%

– Layer 1 support (help desk calls): 34%

– Off-hours network health monitoring: 32%

– WLAN networking and support: 32%

33



Effectiveness of network operations when

network management functions are outsourced

• Organizations that outsource

network management spend

more time troubleshooting

34

38.06%34.58%

27.35%

33.14%36.11%

30.75%

REACTIVE TROUBLESHOOTING PROACTIVE PROBLEM PREVENTION ALL OTHER TASKS AND PROJECTS

Outsourcers No outsourcing

• Network problems detected first by

end users

– Outsourced network management:

48.01%

– No outsourcing: 30.29%



Emerging Network Management

Requirements and Challenges



Network operations is integrated into unified,

cross-domain operations

• Just 26% of enterprises have converged, cross-domain operations centers

– The rest maintain independent NOCs (35%) or distributed operations teams (33%)

• Those who don’t have a cross-domain operations center are integrating network monitoring into cross-domain operations consoles

– 49% already have this

– 36% will integrate within 12 months

• Network management tools are most commonly integrated with:

– Virtual systems management (49%)

– Security monitoring (47%)

– Application performance management (37%)

36



The top challenges to network operations success


25%

24%

23%

20%

19%

19%

17%

17%

16%

15%

10%

5%

LACK OF END-TO-END NETWORK VISIBILITY

LACK OF RESOURCES (PEOPLE AND BUDGET)

LACK OF VISIBILITY/CONTROL INTO EXTERNAL CLOUD SERVICES

LACK OF DEFINED PROCESSES

FRAGMENTED MANAGEMENT TOOLS

LACK OF CROSS-DOMAIN VISIBILITY

LACK OF CHANGE MANAGEMENT CONTROLS

INEFFECTIVE IMPLEMENTATION OF INFRASTRUCTURE PROJECTS (NETWORK UPGRADES/EXPANSION, ETC.)

LACK OF EFFECTIVE CROSS-DOMAIN COOPERATION AND DECISION-MAKING

INEFFECTIVE NETWORK MANAGEMENT OUTSOURCING

LACK OF USAGE POLICY/ENFORCEMENT (UNAPPROVED DEVICES, APPLICATIONS, ETC.)

OTHER



Network management product

features that need the most work• Customizable reporting: 28%

• Compliance: 27%

• Support for multiple

users/operators: 25%

• Specific support for cloud

environments: 25%

• Log file analysis: 24%

Network management product

features that add the most value• Customizable reporting: 37%

• Log file analysis: 35%

• Specific support for virtual server

environments: 35%

• Specific support for cloud

environments: 33%

• APIs for integration with other

tools: 31%

© 2016 Enterprise Management Associates, Inc.

What network management tool vendors

get right and wrong

38



Key Takeaways –

Network Management Megatrends 2016

1. Most enterprises have at least one Internet of Things initiative, and the networking team usually take a leading role

2. Most enterprises are implementing hybrid clouds, and networking teams are adopting SDN overlays and new NPM tools to support them

3. Most enterprises are using advanced network analytics, which they use to enhance network security monitoring and optimize networks and business processes

4. Crowded network management toolkits degrade operational effectiveness

5. More than half of enterprises outsource some network management functions, but this outsourcing appears to degrade network operations

6. Network managers continue to need improved customizable reporting and log file analysis in their tools. Compliance has also emerged as an network management feature that needs improvement

39



Questions?

Get the Full Report:

http://bit.ly/1NsKZNA

© 2016 Enterprise Management Associates, Inc.40

http://bit.ly/1NsKZNA

network management megatrends 2016: hybrid cloud, network analytics and the internet of things

Technology