lessons learned to keep cholesterol on proper level base on private cloud projects
DESCRIPTION
Complicated IT is like bad cholesterol. Lessons learned to keep cholesterol on proper level base on private cloud projects. My IBM Pulse2014 presentation.TRANSCRIPT
© 2013 IBM Corpora/on
RSM-‐1757: Leveraging IBM SmartCloud Virtual Storage Center to Reduce IT Complexity and Streamline Your Environment
Robert Bigos [email protected] +48 693 93 5191
!2
Please note
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. !Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. !The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
2
AgendaInfrastructure challenges in the enterprise environment
• It doesn’t maPer -‐ 2003 Harvard Business Review • 2014 – The World is 10 Xmes more interconnected , instrumented and complicated
Complicated and not virtualized infrastructure can be like “bad” cholesterol • Example of consolidaXon/virtualizaXon study -‐ road to private cloud
Complicated and highly virtualized infrastructure can be like “bad” cholesterol • Simple cholesterol check by visualizaXon “queueing” model in pracXce • “Capacity rolling disaster” challenge in your highly virtualized infrastructure • Best pracXce and matrix compaXbility check …
Lessons learned to keep cholesterol on proper level base on private cloud projects in CEE. • Private cloud approach “sXll” needs balance in: virtualizaXon, standardizaXon, automaXon and service/
placorm management. • “Private cloud” is not just technology or virtualizaXon -‐ it is “transformaXon” project and strong CxO
support is crucial !You will hear some interes/ng features about IBM products:
• IBM SmartCloud Virtual Storage Center • IBM Storage SAN Volume Controller • IBM PureSystems, PureFlex • IBM SmartCloud Analy=cs -‐ Predic=ve Insights
Abstract: According to Harvard Business Review "IT doesn't maHer" published in 2003 , enterprise IT in 2014 is above ten /mes more instrumented and interconnected than it was in 2003. Virtualiza/on helped us achieve beHer infrastructure u/liza/on, but now infrastructure is more complicated as Gartner says , “Virtualiza/on without service management is even more dangerous than not virtualizing in the first place.”Escala/ng IT complexity can lead to higher costs…or a "capacity rolling disaster". IBM SmartCloud Virtual Storage Center enables you to simplify management of an enterprise storage infrastructure. Come hear how to leverage SmartCloud VSC to get a picture of your environment (your infrastructure cholesterol check), see what kinds of findings can be gleaned to improve efficiency, and learn how to /e into private cloud infrastructure, including IBM PureSystem -‐ expert integrated systems. You’ll see how simple monitoring data from SmartCloud VSC and other sources can cut the level of your IT complexity significantly.?
4
New rules for IT Management…
”IT Doesn't MaBer” -‐ New Rules for IT Management from 2003 !… IT…” become commodity inputs. From a strategic standpoint, they become invisible; they no longer maMer. The staff of HBR voted "IT Doesn't MaHer" the best ar/cle to appear in the magazine during 2003…”
“With the opportuni/es for gaining strategic advantage from informa/on technology rapidly disappearing, many companies will want to take a hard look at how they invest in IT and manage their systems. As a star/ng point, here are three guidelines for the future: • Spend less. Studies show that the companies with the biggest IT investments rarely post the best financial results. As the commodi/za/on of IT con/nues,
the penal/es for wasteful spending will only grow larger. It is gecng much harder to achieve a compe//ve advantage through an IT investment, but it is gecng much easier to put your business at a cost disadvantage.
• Follow, don’t lead. Moore’s Law guarantees that the longer you wait to make an IT purchase, the more you’ll get for your money. And wai/ng will decrease your risk of buying something technologically flawed or doomed to rapid obsolescence. In some cases, being on the cucng edge makes sense. But those cases are becoming rarer and rarer as IT capabili/es become more homogenized.
• Focus on vulnerabili=es, not opportuni/es. It’s unusual for a company to gain a compe//ve advantage through the dis/nc/ve use of a mature infrastructural technology, but even a brief disrup/on in the availability of the technology can be devasta/ng. As corpora/ons con/nue to cede control over their IT applica/ons and networks to vendors and other third par/es, the threats they face will proliferate. They need to prepare themselves for technical glitches, outages, and security breaches, shifing their aHen/on from opportuni/es to vulnerabili/es.”
!hHp://www.nicholasgcarr.com/ar/cles/maHer.html Thank you Nick !
5
Regarding “IT doesn’t maPer”!IT infrastructure in the enterprise is like a “home made supercomputer” : very complicated and interconnected. Designed by business department, acquired by procurement department , implemented and managed by IT department.
!• Each year we have smarter sohware and devices but everything is
working as designed, configured and tested… New “smarter” features need Xme to be well tested and Xme to be adopted to producXon environment.
• From this perspecXve every enterprise “home made supercomputer” is similar to “others” but unique … if everything is working it’s ok , if it is not working we lose real money.
• “Spend less” we should probably change to “Spend smarter” and really “…focus on vulnerabiliXes, “ to proacXvely avoid ” … technical glitches, outages…” as result we will spend less … Spend less … can lead to high levels of bad cholesterol
6
The world is more interconnected and instrumented
2005 2006 2007 2008 2009 2010 20110
200
400
600
800
1,000
1,200
1,400
1,600
1,800
Exab
ytes
RFID,Digital TV,
MP3 players,
Digital cameras,
Camera phones, VoIP,Medical imaging, Laptops,
smart meters, mul/-‐player games,Satellite images, GPS, ATMs, Scanners,
Sensors, Digital radio, DLP theaters, Telema/cs ,Peer -‐to -‐peer, Email, Instant messaging, Videoconferencing,
CAD/CAM, Toys, Industrial machines, Security systems, Appliances
10x growth in five years
2012 2013 2014
Virtualiza/onCloud
BigData
Business Analy/cs
yearly accelera=on
Explosion of information driving 54% growth in storage shipments every year.
1.5x
7
A few IT related figures
200 Billion 2.2 Millioncyber aHacks every day
physical assets with IT intelligence
1 Billion consumers with Smartphones
20 MWh 200%DataCenter electricity
Requirements growth over last 5 years
electricity requirements of
medium DataCenter
51 %average y/y virtual server
growth
85% idleIn distributed computing environments, up to 85% of computing capacity sits idle.
70¢ per $170% on average is spent on maintaining current IT infrastructures versus adding new capabilities.
8
CEO declaring changes
Social
Cloud
Intelligent/Connected Systems
Mobile
Big Data
* IBM CEO study 2013
Source: “The Sofware Edge: How effec/ve sofware development and delivery drives compe//ve advantage,” IBM Ins/tute of
Business Value, March 2013 1. Technology factors 2. People skills 3. Market factors 4. Macro-‐economic factors
5. Regulatory concerns 6. GlobalizaXon
9
There is no “magic buPon”
There is no “MAGIC BUTTON” in your infrastructure which can resolve all your potenXal threats.
Source: hHp://make-‐everything-‐ok.com/
11
Example of virtualizaXon study
Source: IBM study in CEE for one of the top 100 companies in CEE.
Interac/ve mo/on chart shows the u/liza/on of about 1100 serves and related challenges from technical and business
perspec/ve
Source of picture:
hHp://engagedhealthsolu/ons.com/2011/09/cholesterol-‐explained/
==“Compa/bility matrix and migra/on challenges” as “bad” cholesterol example”.
14
Queues and buffers basis
▪ Response Xme depends on service Xme and queueing Xme. !
• Queue length depends on arrival rate and service /me • U/liza/on shows how busy the server is: work_/me/measured_/me • When u/liza/on reaches satura/on = 100% ,response /me going to infinity in some cases……
buffer
Queuing
Queuing
Queuing
Source: hHp://perfdynamics.blogspot.com/2010/03/bandwidth-‐vs-‐latency-‐world-‐is-‐curved.html Graph provided by Neil Gunther. Thanks !
QueuingQueuing
Queuing
Queuing Queuing Queuing Queuing
Of course there is powerful mathema/cs and very complicated mul/ queue models influencing each other…
working par/ally in parallel and sequence mode.
But this founda/on is enough to understand capacity planning and Cholesterol check.
To see more: hHp://en.wikipedia.org/wiki/Amdahl's_law hHp://en.wikipedia.org/wiki/LiHle's_law
hHp://www.perfdynamics.com/Manifesto/gcaprules.html ,
28
IBM SmartCloud Virtual Storage Center architecture
▪ My top 5 favorite features !
• Monitoring/alerXng data warehouse architecture (storage, SAN, hypervisor, VM) • Threshold violaXons observaXons, • Data Path Viewer • Storage AnalyXcs Engine • IBM System Storage SAN Volume Controller (SVC) online migraWon and Easy Tier
See more: !Chris Dotson works in IBM's CIO Office as a Senior IT Architect for Services TransformaXon. He is guiding IBM's own storage transformaXon. As a large enterprise, IBM manages over 100 petabytes of data, growing at 25% per year. Chris discusses block storage virtualizaXon, automated block storage Xering, file cloud storage, and automated block storage management at IBM. hPps://www.youtube.com/watch?v=SRpz1VS2mnU BJ Klingenberg is a DisXnguished Engineer and Enterprise Storage Management lead for IBM. BJ shares his experiences using IBM Tivoli Storage ProducXvity Center in IBM's Service Provider environment. hPp://www.youtube.com/watch?v=7XTBw7g4_OI Mike Griese explains how IBM SmartCloud Virtual Storage Center can help you get the most out of your SAN aPached storage devices by increasing efficiency, uXlizaXon, availability, disaster recovery capability and manageability. hPps://www.youtube.com/watch?v=QSADtwWyNtk
32
“Server-‐storage-‐centric” performance troubleshooXng
Data Path View •status •all relations
Agent less host monitoring •status, performance (SAN, Lun’s) •all relations
Threshold Violations
(A)
(B)
(C)
20
Do you know how complicated your “home made computer” is?
Cholesterol check visualiza/on prepared in CEE for one of top 100 enterprise in CEE. !Video shows small part of infrastructure: 10 storage arrays, about 1200 volumes, 1000 SAN ports , 10 hosts . 100 VM
Red color means … not good. Not showed host to vm mappings and SAN zones ( too big resolu/ons required for
presenta/ons ).
Source of picture:
hHp://engagedhealthsolu/ons.com/2011/09/cholesterol-‐explained/
==
“MulX-‐queing-‐centric” performance troubleshooXng in „share everything architecture"
Video: big or red means … not good 6 day view, 5 minutes sample.
Source data: IBM SmartCloud Virtual Stor age Center TPCREPORT,
Tools: DatasetR, igraph
Cholesterol case study findings/examples will be included
USL
20
UXlizaXon heatmap
!GREEN -‐ not used -‐ losing money RED -‐ business and customer wai/ng YELLOW -‐ perfect balance !About 700 volumens on enterprise Tier 1
and 2, 500 TB
Y = 24 h
X = 7 days observa=ons
21
Do you know the answers to simple quesXons about your infrastructure?
Are you sure all your nodes are fully compa/ble? !
• when did you check it out ?
!Are you sure all your nodes have more than 1 working path ? !
• when did you check it out ?
!How are they u/lized over /me? !
• is there a capacity for growth ? • is there a capacity for change ? • is there a capacity to restore ?
!Everything is…. working as designed, configured and tested… !IT industry appreciates experienced people…
"capacity rolling disaster”
�17
“…Reports that say that something hasn't happened are always interes/ng to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns -‐-‐ the ones we don't know we don't know…” !!!!!
Donald Rumsfeld, February 12th, 2004 DOD News Briefing Source: http://www.defense.gov/transcripts/transcript.aspx?transcriptid=2636
!!
Know unknowns and unknown unknowns …
22
I strongly agree with Gartner on this point
“VirtualizaWon without service management is even more dangerous than not virtualizing
in the first place” … Source: hBp://gartner.com/
Source: hHp://dilbert.com/
▪ Integrated placorm management and service management is not an addiXonal cost … • it is just an integrated part of solu/ons • spend smarter rather spend less
• cost reduc/on will come
24
Lessons learned to keep cholesterol on proper level▪ “Private cloud” is not just technology or virtualizaXon -‐ it is “transformaXon”
project and strong CxO support is crucial !
• regardless of what anyone says …. • Private cloud approach “s/ll” needs balance in: virtualiza/on, standardiza/on, automa/on and
service/pla{orm management • “private clouds” ready procurement behaviors and standards • project driven islands … may kill any private cloud project • management silos vs dynamic infrastructure management challenge • license agreements or SLA agreements … needs changes • others !
▪ Simplify your infrastructure by standardizaXon and virtualizaXon !
• one vendor policy if possible , • expert integrated systems or “cloud ready sofware” where applica/ons logic understand all poten/al
issues related to infrastructure…
25
Lessons learned to keep cholesterol on proper level▪ The best technology sXll needs backup and restore process.
!• RTO and RTO are s/ll important in cloud but not popular in RFP, • Highly virtualized environments are very complicated mul/ queueing network , backup process may
consume all of your resources if not well designed !
▪ Check your cholesterol regularly !
• Capacity planning process (uWlizaWon, response Wme curve check for all important queues) • if there is a lot of observaXons on the “bad” side invesXgate which interconnected queue is affected
• Automated “best pracXce / health check” (configuraXon best pracXce, sohware level, interoperability) • ask IBM local STG Lab services / GTS representaWve for infrastructure health check • ask for (internal) System Storage Interoperability Center Discovery UXlity from IBM Customer Center Montpellier !
!▪ “Sohware defined storage, network , infrastructure”, expert integrated systems,big
data, analyXcs, cloud and queueing are like gravity. !
• You can fight it, but it's beHer to understand and profit from it consciously.
20
See more ibm.com/cloud
!or contact
+48 693 93 51 91
© Copyright IBM Corporation 2012. All rights reserved.
– U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. !
IBM, the IBM logo, ibm.com,and IBM SmartCloud Virtual Storage Center, IBM Storage SAN Volume Controller, IBM PureSystems, PureFlex , IBM SmartCloud Analytics are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml !
Other company, product, or service names may be trademarks or service marks of others. !
Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are
provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.
37