measure twice, cut once: 5 best practices for selecting your cloud service providers

Webinar: “Measure Twice, Cut Once: 5 Best Practices Webinar: “Measure Twice, Cut Once: 5 Best Practices For Selecting Your Cloud Service Providers”

Richard StoneCloud Solutions ManagerCompuware Inc

Dennis DrogsethVice President of ResearchEnterprise Management Associates

September 8, 2011

• Introduction & welcome

• “Five Steps to Managing Your Cloud Service Providers”

– Dennis Drogseth, Vice President of Research, EMA

Agenda

• “5 Best Practices For Selecting Your Cloud Service Providers”

– Richard Stone, Cloud Solutions Manager, Compuware

• Q&A session

– Use Webex ‘chat’ to ask your questions at any time

Five Steps to Managing Your Cloud Service Providers for Effective Application Delivery

Dennis Drogseth

Vice President

Enterprise Management Associates

September 8, 2011© 2011 Enterprise Management Associates, Inc.

Agenda

• Where and why cloud adoptions are changing

• Step One: Assess your application/infrastructure design requirements

• Step Two: Map your application/infrastructure design requirements to cloud/ hybrid options

• Step Three: Assess your monitoring requirements• Step Three: Assess your monitoring requirements

• Step Four: Select cloud service providers based on design needs, reputation and willingness to partner

• Step Five: Manage your cloud SPs as participants in a broader application/service ecosystem.

• Conclusion

Slide 4 © 2011 Enterprise Management Associates, Inc.

Do you believe, or did you discover, that once begun, Cloud initiatives required rethinking and/or redirection before the...

70%Yes

70% of Respondents have had to RETHINK or REDIRECT Initial Deployments

Sample Size = 155

8%

21%

0% 10%

Don’t know

No

Column %


Which changes has your org made/anticipate to rethi nk and/or redirect the original cloud initiative?

39%

40%

41%

47%

Changes in management technology adoption for service performance management

Changes in application development and design

Changes in management technology adoption for provisioning services & managing …

Changes in process and process flows

Of These, Process, Technology Adoption, and Application Development Dominate

Sample Size = 138

1%

26%

33%

36%

36%

0% 10% 20% 30% 40% 50%

Other (Please specify)

Changes in organizational ownership

Changes in how cloud and/or virtualization initiatives are measured for effectiveness

Changes in how cloud and/or virtualization deployments are initiated

Changes in how services are accounted for from a cost perspective

for service performance management

% Valid Cases (Mentions / Valid Cases)

Step One: Assess your Application/ Infrastructure Design RequirementsInfrastructure Design Requirements

© 2011 Enterprise Management Associates, Inc.Slide 7

Application/Infrastructure Design Requirements Cut Across Domains

• Database(s)

• Storage

• Application Server (s)

• Middleware Server(s)

• Mainframe

• Web 2.0 and other services – e.g. credit card


• Web 2.0 and other services – e.g. credit card validation

• Network interdependencies – local and WAN

• Devices – application acceleration, load balancing , etc.

• DNS, Active Directory, and other services

Note: SaaS can be private as well as public. And even public SaaS options, along with public IaaS and PaaS options, need to be understood in terms of overall application ecosystem design.

And Can Include Many Different Types of Component Interdependencies

50%50%

56%59%

61%61%

77%

MobileVirtual Application environments (Citrix, etc.)

WirelessVPN

Virtual Server environments (VMWare, Xen, etc.).NETJava

Which of the following environments pertain to your company’s Web-based business applications? Select all that apply.

3%16%

22%29%

31%33%34%

37%37%

40%50%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Other (Please specify)Ruby on Rails

MPLSContent Delivery Networks

AJAXCellular

Thin clientSaaS

Cloud ServicesVirtual Desktop Infrastructure

Mobile

% Valid Cases (Mentions / Valid Cases)Sample Size = 119, Valid Cases = 119


Who YOU are Matters

• Manufacturing: more focused on industry-specific apps and less focused on CRM/ERP than average, supports a wide range of technical environments – e.g. SaaS, virtualization CDNs.

• Retail: more likely to track Web abandonment rates and is asked to share information with suppliers Tracks competitors’ performance more than average.

• High Tech/Computer: favors Corporate Web sites and SOA, Tracks competitors’ performance, and more focused on sharing information with external customers, and favors social networking.

• Healthcare: strong on industry-specific applications and back office (ERP CRM), favors SaaS, is asked to share information with external customers, suppliers and partners more than average.

• Finance: more likely to view Web applications as critical, naturally focuses most on financial transactions, is least likely to go for Cloud and social networking.


Step Two: Map your Application/Infrastructure Design Requirements to Cloud/Hybrid Requirements to Cloud/Hybrid Options


What cloud computing service model(s) has your organization deployed, or is planning to deploy? Select all that apply. by...

Title76.92%

76.56%Software as a Service (SaaS) – a fully-functional pre-built application environment is delivered by

the (internal/external) cloud provider

IaaS and PaaS show more Organizational and Maturity Dependencies than SaaS

Sample Size = 155, Valid Cases = 155

45.05%

43.96%

29.69%

31.25%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Infrastructure as a Service (IaaS) – only basic system or machine-level resources are delivered

by the (internal/external) cloud provider

Platform as a Service (PaaS) – an environment for application development and deployment is

delivered by the (internal/external) cloud provider

Without X-Domain With X-Domain


Map your Application/Infrastructure Design to Cloud and Hybrid Options

• Internal Cloud versus External Cloud versus Virtualized Infrastructure only versus Physical Infrastructure

• SaaS, IaaS, Paas

• Plan A

• Plan B• Plan B

• Virtualized = ???

• Look before you leap


Indicate the top 2 technologies for building cloud services not including systems virtualization

49%

56%

Web Services

Virtual Desktop Infrastructure

Cloud is also Impacting Application Design Criteria

Sample Size = 155

19%

35%

41%

0% 10% 20% 30% 40% 50% 60%

Configuration libraries

New application APIs

Web 2.0 technologies



Step Three: Assess your Monitoring RequirementsRequirements


Some Overall Monitoring Objectives

• Pre-deployment and Dev/Ops

• Scalability

• Geographical variance and mobility

• SLM/ SLAs

• User Experience Management

• Business impact

• Multi-dimensional Transaction awareness

• External ratings and rankings

• And Understand WHO’s Doing the Monitoring


Web-based and other Application Monitoring Requirements Don’t Go Away

45%

57%

66%

67%

71%

Data on your competitors’ performance

Financial transactions

Performance of Web applications used internally

Web analytics (tracking how people use your Web sit e)

Load balancing/ traffic management

Which of the following is your company actively monitoring in support of Web-based applications performance? Select all.

2%

22%

23%

24%

35%

39%

41%

42%

44%

0% 20% 40% 60% 80%


CDN performance

Mashups

Ad-serving

Geographically specific content

Cloud infrastructure performance

SaaS (e-commerce packages integrated into your Web site, such as SalesForce.com,)

Integrations with third -party Web sites (e.g. travel reservation systems)

Suppliers (for manufacturing; healthcare; retail, e tc.)


Service Management Dashboards with Advanced Analytics Deliver Especially Strong Value

• Dashboard plus analytics best in ALL areas but especially:

• 1.8 times more likely to reduce management complexity

• 1.4 times more likely to reduce capital costs via cloud

• 1.7 times more likely to free up resources for strategic projects

• 1.9 times more likely to improve service resilience via cloud

• 1.4 times more likely to accelerate deployment of existing services • 1.4 times more likely to accelerate deployment of existing services via cloud

• 1.4 times more likely to increase infrastructure flexibility and agility via cloud

• 1.8 times more likely to expand revenue channels via cloud

• Twice as likely to deliver business model enhancements via cloud


What was the percentage increase or decrease of your organization’s annual IT budget from last year to this year? Grouped by Service level management

Title29.73%

With Service level Management Without Service level Management

IT Organizations Without SLM are more than 1.6 times more likely to show flat/declining Budgets

Sample Size = 142

70.27%

29.73%

Increased Flat/Decreased

51.47%48.53%

Increased Flat/Decreased


Step Four: Select Cloud SPs Based on Design Needs, Reputation and Design Needs, Reputation and Willingness to Partner


Selecting for Design, Reputation and Willingness to Partner

• Who are they?

• What do they offer?

• What SLAs do they offer

• What are their security guarantees?

• What is your effective access to them when things go wrong?

• Are they (at all) willing to partner with you or do they manage their clients as commodities?


What are your top decision factors when choosing a cloud computing service provider (external cloud)?

43%

43%

55%

63%

Low cost

Willingness to commit to SLAs

Proven performance

Security, risk, or compliance strengths

Security, Overall Performance, and SLA Commitments Lead Cloud SP Selection Criteria

Sample Size = 155

1%

20%

21%

28%

28%

43%

0% 10% 20% 30% 40% 50% 60% 70%


Willing to work with other cloud service providers

Willingness to provide data-level access

Diversity of services

Willingness to support your need to monitor

Low cost



Check All that Apply!

• Which are you using Cloud Service Providers for? (check all that apply)

• Mission critical customer-facing applications

• Production ERP applications

• Partner/ supply chain, or other community application environments

• Back office niche applications• Back office niche applications

• User productivity applications

• Messaging and collaboration applications

• Off-site Web hosting

• Platforms for application test and development

• Remote database

• Remote storage

• Full remote infrastructure support

• Trying out new apps to see if they’re useful


Step Five: Manage your Cloud SPs as Participants in a Broader Participants in a Broader Application/Service Ecosystem


Web-based Application Ecosystems are Increasingly on the Rise

51%They are an essential

part of our business or services

How would you describe the role of Web-based applications for your business/organization?


49%

0% 10% 20% 30% 40% 50% 60%

They are an important part of our business or

services

Column %Sample Size = 119

Information Sharing Across Web-based Application Ecosystems Provides Relevant Insights into Cloud

62%

66%

Internal on -line

Internal relevant IT groups

Internal between IT and the lines of business

Which of the following groups are actively asking you to share Web-based applications performance information? Select all.


35%

38%

39%

43%

0% 10% 20% 30% 40% 50% 60% 70%

External suppliers

External customers

External partners

Internal on -line operations


Sample Size = 119, Valid Cases = 119

Ecosystem Monitoring and Control Concerns Reflect Overall Cloud Concerns

47%

47%

49%

65%

Lack of unified SLAs

Overall lack of control

Lack of visibility

Security vulnerabilities

What are the obstacles that you foresee in monitoring cloud-related services? Select all that apply.


5%

33%

35%

40%

44%

47%

0% 20% 40% 60%


Support for virtualized environments

Unique API requirements

Managing the service provider politics beyond the technologies

Lack of accountability

Lack of unified SLAs

% Valid Cases (Mentions / Valid Cases)Sample Size = 119, Valid Cases = 43

How to Manage Your Cloud Providers as a Part of an Extended Application Ecosystem

• Cloud contracts can have shorter durations- - this is Cloud “Shopping Mall” and let them know it

• Use Cloud to try out new services – who’s and what’s right for you?

• Have multiple options – who’s performing best today?

• Revisit the term “stakeholders” – if you feel their stakeholder is their lawyer, maybe not the right choice.lawyer, maybe not the right choice.

• Formalize process expectations in the contract—they will if you don’t – be proactive .

• In parallel, publish what you’ve got– define your services to YOUR customers, so you can manage their expectations.

• Seek out and external resources for validating poor performance and document your concerns

• Manage them as RESOURCES for YOUR application services


Thank you for attending.


Thank you for attending.

“5 Best Practices For Selecting Your Cloud“5 Best Practices For Selecting Your CloudService Providers”

Richard Stone

Cloud Solutions Manager

Compuware Inc

5 Best Practices For Selecting Cloud Service Providers

Insure all stakeholders have a seat at the table,

including Dev & Test

Choose cloud model(s) based on both technical & business requirements

Implement a cloud Implement a cloud performance monitoring

strategy from Day 1

Evaluate CSPs based on your IT design goals and

business needs

Insure performance goals by continuously monitoring

the end user experience

Best Practice #1: Insure all stakeholders have a seat at the table, including Dev & Testseat at the table, including Dev & Test

Two important considerations ...

Your next strategic business initiative

• Mobility • In 2-3 years, mobility will become an integral

part of your cloud strategy

• This will create a new set of technology needs

• Make it part of your cloud thinking from Day 1

• Application performance is critical • Slow performance = Lost revenue

• Design for scale even if you don't need it (yet!)

• Only the developer really knows• How the application works

• What resources are required

Application design can impact ROI

• Customer example:

– An application enhancement that would have cost money in a traditional data center . In the cloud...

The Cloud Offers Unique Opportunities For Optimization

No loss of functionalityNo loss of functionality

No additional costNo additional cost

49 % Response Time Improvement49 % Response Time Improvement

Best Practice #2: Choose cloud model(s) based on both technical & business requirementson both technical & business requirements

SaaS, PaaS, or IaaS?

Key Considerations

Who are the subscribers?

What does a subscriber get? What does a subscriber get?

How are usage fees calculated?

How much control does a subscriber have?

How much control does the provider have?

http://csrc.nist.gov/publications/drafts/800-146/Draft-NIST-SP800-146.pdf

Another useful resource ...

http://www.programmableweb.com/

Best Practice #3: Implement a cloud performance monitoring strategy from Day 1performance monitoring strategy from Day 1

The Application Delivery Chain is more complex

Inside the firewall

Outside the firewall User’s browser

Flashanimation

Shoppingcart

Web analytics

Search

CMS content

Ads fromAdserver

CDNcontent

Video frommedia server

Users expectations are changing ...

Compuware Gomez Application Performance Monitor

Source: Compuware Gomez 2010 Study of 500 Million End-User Interactions Across 200+ Web Sites

• Announced today! (Available at www.compuware.com)

• "As page load time increases from one to five seconds,

• user abandonment rates increase by 40 percent"

Don't rely on SLAs to measure what’s important

Customers experience this SLAs measure this

Measure the only thing that really mattersThe End User Experience

Test/monitor your app the way your users access it:

• What they do (key transactions)

• Where they do it (geographic locations)

• How they do it (browsers and mobile devices)

Prioritize and resolve issues

Is it my

data

center?

Is it an

ISP or the

Internet?

Is it a

3rd party

provider?

Is it a

browser or

device?

Prioritize and resolve issues

• Measure the business impact (users)

• Isolate root causes

Best Practice #4: Evaluate CSPs based on your IT design goals and business needs design goals and business needs

Two major considerations

• Will we have all the

capacity needed when our

customers demand it?

• Are we in the right places

to provide a consistent

service our customers?

Will we have all the capacity needed when our customers demand it?

Evaluate vendors based on your goals…

Capacity

• Test vendors to 15-20% past estimated capacity goals

Elasticity

• Baseline end-user performance before & after testing

• Test during pre-deployment and in production

• Ramp elasticity testing to peak levels

Burstability

• Isolate the cloud elements from other infrastructure to test

• Test the “failover process”

Are we in the right places to provide a consistent service our customers?One Application

One Service Provider • The Cloud: It’s opaque, shared and elastic; it’s complex, multi-sourced and variable

• What your neighbor does affects you

• What’s needed is a Cloud Performance visualization tool … so we built one …

• And the results surprised us …

– The Earth isn’t flat

– The speed of light does matter

Gomez Network: Monitoring the Cloud

• Cloud education at cloudsleuth.net

• Cloud services evaluation

Community of cloud-based companies and experts providing:

• Hands-on tools

• Best practices

Global Provider View Cloud Performance Analyzer

•“Outside in” perspective of cloud service provider performance• Real-time data including impact of geography

• Performance & availability bottleneck identification

• Independent validation of providers’ SLA claims

Best Practice #5: Insure performance goals by continuously monitoring the end user experiencecontinuously monitoring the end user experience

CustomersCloud Cloud Cloud Cloud

Private Public Browsers

Find and fix problems quickly ...

1. What is the business impact?

2. Where is the root cause?

Employees

Mainframe

Storage

Data CenterData CenterData CenterData Center

Web Services

Mobile Components

Web Servers

App Servers

DB Servers

Load Balancers

WANOptimization

Virtual/Physical Environment

LocalISP

Mobile Carrier

Content Delivery Networks

3rd Party/Cloud Services

Employees

Private Public

Network

MajorISP

Browsers

Devices

Is it my Is it my Is it my Is it my datadatadatadatacenter center center center or WAN? or WAN? or WAN? or WAN?

Is it anIs it anIs it anIs it anISP or ISP or ISP or ISP or the the the the

Internet?Internet?Internet?Internet?

Is it a Is it a Is it a Is it a 3333rdrdrdrd partypartypartypartyproviderproviderproviderprovider

????

Is it a Is it a Is it a Is it a browser browser browser browser

or or or or device?device?device?device?

Test multiple Cloud Providers from multiple locations... including the Last Mile Network

Simulate hundreds of thousands of concurrent users, or millions of page views per hour by leveraging Gomez’

network of public Cloud data centers and Last Mile Peers

Simulate hundreds of thousands of concurrent users, or millions of page views per hour by leveraging Gomez’

network of public Cloud data centers and Last Mile Peers

Create load tests Create load tests

leveraging a global leveraging a global

network of cloud network of cloud

datacentersdatacenters

Run load tests with real IE and FF browsers or choose from over 500

mobile devices

Run load tests with real IE and FF browsers or choose from over 500

mobile devices

Include peers from Include peers from

dozens of countries to dozens of countries to

gain insight into end gain insight into end

user experience under user experience under

loadload

Compuware Application Performance Management

• Web, non-Web, mobile, streaming, cloud-based applications

• Across all customers, users, browsers, devices, infrastructure, and geographies

• And across the entire application life cycle ... development, test, and production

• Rapid issue notification with actionable diagnostics

• Insight into how these issues affect your business (revenue, brand, cost)

We help organizations optimize the performance of their business-critical applications

SaaS, Cloud-Based and

On-Premises Offerings

• Rapid startup and payback

4,000+ Customers Worldwide

• 2,500+ enterprise customers

• 1,500+ SMB customers

• 12 of top 20 US sites

Global Reach

• Over 80 offices in 29 countries worldwide

• Strategic service delivery

Recognized as Industry Leader

• Gartner:Leader in APM magic quadrant

• Forrester Research: “…a complete view of end user experience”*

• Ovum: “Game-changing”

*”Trends: The Diversification Of End User Experiencing Monitoring”, Forrester Research, Inc., July 5, 2011

Q & ASend questions via ‘chat’

Additional Resources Available To DownloadCOMPUWARE PAPER “THE WORLD IS NOT FLAT: HOW

APPLICATION LOCATION INFLUENCES USER EXPERIENCESUMMARY OF EMA “OPERATIONALIZING

CLOUD” REPORT

More information: www.compuware.com Cloudsleuth: www.cloudsleuth.net