japan aws user group osaka study session #0 lt: devops
DESCRIPTION
Japan aws user group osaka study session #0 Lightning talk about DevOps by nxhack 2010/11/17TRANSCRIPT
DevOpsIntroduction to DevOps movement,
and Some views of the Cloud Computing
Japan AWS User Group Osaka study session #0Lightning Talk @nxhack
2010/11/17
http://www.flickr.com/photos/pmtorrone/23362393/
@nxhack
Background: 25年以上 情報処理業界 17年間 イソターネット業界 Bias: 運用な人 *NIXな人 文系エソジニア
伝えたい事: DevOps 運動と何ぞや そこから見えるクラウドの姿
前説
IT Operations
http://www.flickr.com/photos/cote/54408562/
50年の歴史
The world's first-ever web server
1990年
Web
http://www.flickr.com/photos/cote/54408562/
TraditionalSystems OperationsX
http://www.flickr.com/photos/arthur-caranta/2926332140
Web Operation
CAUTION!WebOps BIAS運用さん視点
DevOpsMovement
http://velocityconf.com/
http://opscamp.org/
http://www.devopsdays.org/
Web AppsWeb Services
StartupScalableGrow fast
Continuous DevelopmentContinuous Deployment
ResearchDevelopment
DeploymentRelease
CustomerFeedback
ContinuallyPivot
(Iterate)
Hypotheses Testing
PDCA cycle
PDCA cycleX
http://www.flickr.com/photos/sdasmarchives/4727993474/
Colonel John (Richard) Boyd
(January 23, 1927 – March 9, 1997) was a United States Air Force fighter pilot and Pentagon consultant of the late 20th century, whose theories have been highly influential in the military, sports, and business.
OODA Loop
Web AppsWeb Services
Shrink WrapXhttp://www.flickr.com/photos/cogdog/116957071/
http://www.flickr.com/photos/arthur-caranta/2926332140
The infrastructure is the application.
The application is the infrastructure.
Devの役割Opsの役割
http://www.flickr.com/photos/aaronjacobs/64368770/
Developer
FeaturesEvolution
http://www.flickr.com/photos/troyholden/4036820768/
Operator
Services Stability
安定性安全性
http://www.flickr.com/photos/greg-palmer/3490903215/
Ops focuson failure
hardware failure
http://www.flickr.com/photos/purplemattfish/3760706750/
hardware failure
http://www.flickr.com/photos/purplemattfish/3760706750/
Avoid SPOF
hardware failure
http://www.flickr.com/photos/purplemattfish/3760706750/
Avoid SPF冗長設備の投資対効果を定量的に示しなさい
bursting
http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
bursting
http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
Capacity Planning
http://www.amazon.co.jp/dp/0596518579
bursting
http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
Capacity Planning
http://www.amazon.co.jp/dp/0596518579
性能設計の根拠と投資対効果を定量的に示しなさい
software failure
http://www.flickr.com/groups/webopsviz/pool/with/4819476138/
software failure
http://www.flickr.com/groups/webopsviz/pool/with/4819476138/
Testing
software failure
http://www.flickr.com/groups/webopsviz/pool/with/4819476138/
Testing????
software failure
http://www.flickr.com/groups/webopsviz/pool/with/4819476138/
Testing????Testing?
operation failure
http://www.flickr.com/photos/proimos/4199675334/
operation failure
http://www.flickr.com/photos/proimos/4199675334/
指示書・手順書
operation failure
指示書・手順書
http://www.amazon.co.jp/dp/4122018331http://www.amazon.co.jp/dp/0691004129
うがぁ失敗!
operation failure
http://www.amazon.co.jp/dp/4122018331http://www.amazon.co.jp/dp/0691004129
orz
Web AppsWeb Services
Continuous DevelopmentContinuous Deployment
http://www.flickr.com/photos/greg-palmer/3490903215/
CHANGE CAUSES OUTAGES
http://www.flickr.com/photos/adjourned/308053363/
http://www.flickr.com/photos/sis/124916325/
http://www.flickr.com/photos/carve/2856365431/
http://www.flickr.com/photos/rocketjim54/2955889085/
“It’s not my code, it’s your machines!”
“It’s not my machines, it’s your code!”
http://www.flickr.com/photos/rocketjim54/2955889085/
本番環境がおかしいじゃないの?俺のテスト環境じゃぁ問題なかったぜ!
サーバ環境はあなたの仕様書通りだぜ。問題はプログラムの方だろう!
Dev Ops
Dev Ops
Dev Ops
Wall o
f Con
fusion
Dev Ops
Wall o
f Con
fusion
Dev vs. Ops
http://www.flickr.com/photos/ammo_hauler/3504563587/
Dev vs. Ops
普通のDev
良いDev
悪いDev
普通のOps良いOps悪いOps
儲かりまへん ややウケ 障害多発Opsは地獄
ぼちぼちでんな バカウケ ドッチラケ
あきまへんリリース遅延Devは激怒 倒産
Dev・Ops 欽ドン! 的マトリックス
普通のDev
良いDev
悪いDev
普通のOps良いOps悪いOps
生ける屍 ◯ 倒産生ける屍 ◎ 倒産倒産 売却 倒産
Dev・Ops 欽ドン! 的マトリックス (辛口版)
ちょっと待てよ
Dev & Ops
Goal:To Enable
Our Business
We make Our Servicestogether
To EnableContinuous DevelopmentContinuous Deployment
じゃぁどうする
Developer
Manifesto for Agile Software Development
We are uncovering better ways of developingsoftware by doing it and helping others do it.Through this work we have come to value:
Individuals and interactions over processes and toolsWorking software over comprehensive documentation
Customer collaboration over contract negotiationResponding to change over following a plan
That is, while there is value in the items onthe right, we value the items on the left more.
http://agilemanifesto.org/
agile and iterative
ディーン・レフィングウェル (著), 玉川 憲 (監修, 翻訳)
http://www.amazon.co.jp/dp/4798120405/
DevelopersProduct Owners
Testers
DevelopersProduct Owners
Testersたいへんよくできました
OperatorsSystem AdminsDB Admins
Network EngineersDesigners
OperatorsSystem AdminsDB Admins
Network EngineersDesigners
がんばりましょう
どうすんのよ
Operator
Infrastructure:BootstrappingConfigurationMonitoring
agile operations
agile operations?
Infrastructure is HARD
http://www.flickr.com/photos/johnseb/3425464/
Infrastructure is HARD
http://www.flickr.com/photos/johnseb/3425464/
!
agile Infrastructure
IaaSInfrastructure as a Services
Server: EC2Storage: S3, EBSNetwork: ELB, CloudFrontMonitoring: CloudWatchDataBase: RDS, SimpleDB
Pooled IT Resources
On DemandSelf ServicePay as you goFair pricing
API
% ec2-run-instances ami-b232d0db -n 3 --availability-zone us-east-1a! ! ! !RESERVATION r-385c5950 012301230123 defaultINSTANCE i-5bca5a30 ami-b232d0db pending 0 m1.small 2010-04-07T12:25:47+0000 us-east-1a aki-94c527fd ari-96c527ff monitoring-disabled ebs paravirtualINSTANCE i-59ca5a32 ami-b232d0db pending 1 m1.small 2010-04-07T12:25:47+0000 us-east-1a aki-94c527fd ari-96c527ff monitoring-disabled ebs paravirtualINSTANCE i-5fca5a34 ami-b232d0db pending 2 m1.small 2010-04-07T12:25:47+0000 us-east-1a aki-94c527fd ari-96c527ff monitoring-disabled ebs paravirtual
サーバー 3台用意してね☆
Infrastructure as Code
API! API! API! API! API!Infrastructure as Code! Infrastructure as Code! Infrastructure as Code!
http://www.flickr.com/photos/programmerman/2315566040/
Give me an API, or give me DEATH!
http://www.flickr.com/photos/bitchcakes/3895840169/
- 2009/04/05 -「70/30から30/70へ変革」Amazon.comのEコマース・サイトにおいて、エンジニア・リソースの70%がネットワーク、ストレージ・クラスタなどのインフラ障害対応に費やされていて、本業であるEコマースサイトをどう効率よくするかというシステム開発に時間が割けない状態が続いていました。そこで、ハードウェアは壊れる前提で物事を考え、仮想化や分散処理、SOA技術を屈指した共有サービス・モデルに移行したとのこと。最終的には0%にはならないものの許容範囲であるリソースの30%程度に削減することができ、エンジニアはインフラ管理から解放されることとなりました。これらの経験から、拡張性、コスト効率化、信頼性、セキュリティに強い今の姿のAmazon Web Serviceが完成し、真のInfrastructure as a Service(IaaS)になったとのことでした。そして、自社のインフラ資産をCAPEX(資本的支出額)からOPEX(事業運営費用)に転換することができたとのこと。つまり、自社の持つIT資産を武器に利益を生み出すサービスとして事業転換に成功したということですね。
http://blogs.itmedia.co.jp/eno/2009/04/cloud-computing.html榎本 瑞樹さんのBLOG「ベンチャースピリット」 X 「セレンディピティ」より
agile operations を実現するために EC2
は生まれた
かなり本気な余談:IaaSを使う最大の利点は海外を含めた複数の拠点にデプロイできる事。アナタたち日本だけで商売するつもりですか?
Bootstrapping:Infrastructure as a ServicesInfrastructure as Code
agile Infrastructure
を使いこなす
http://www.flickr.com/photos/robenjoyce/4208638969/
Tools!
DevOpsTool Chainshttp://code.google.com/p/devops-toolchain/
Release
Generalized architecture
Control
Orchestration
Dispatcher
Provisioning
Deploy
Config management
OS boot/install
Artifact repository
Build
CI Server Issue tracker
SCM Repository
Model
Asset inventory
Host naming
Identity
CMDBMonitoring
Events
Trending Reporting
Trending
Workflows
Resources
Topology
Configuration
Code
Sources
Scheduler
Wednesday, June 23, 2010
http://code.google.com/p/devops-toolchain/
Release management
Artifact
Build
SCM
Trackerbugzillamantistrac
svncvsgithg
Repository
yum/yastartifactoryarchivaDFS/openEFSHTTTP
eggsgemsperlmod
tgz/ziprpm/deb/pkg/msijar/war/ear
makemavenant/ivy
sbtrakephing
Wednesday, June 23, 2010
http://code.google.com/p/devops-toolchain/
Dev
Monitoring and control
Measurement Instrumentation
Charting, History, SPC
Alarm Mgmt
Op Console, Control
Runbook Automation
nagioszenossopennms
rrdtoolcacti
ControlTierJobschedulerOpenScheduler
Wednesday, June 23, 2010
http://code.google.com/p/devops-toolchain/
Ops
Provisioning
Bcfg2cfengineChefPuppetSmart Frog
Application Service Orchestration
System Configuration
Cloud or VMImage
LaunchOS
Install
KickstartJumpstartCobblerOpenQRMxCAT
XenlxcopenVZEucalyptusKVM
CapistranoControlTierFabricFuncmCollective
Cloud: OS Install:
Command and Control
Configuration
Bootstrapping
Prov
isio
ning
Act
ivity
Wednesday, June 23, 2010
http://code.google.com/p/devops-toolchain/
Dev
Ops
自動化できうるところは全て自動化する
agile Infrastructure を活用しインフラ構築・構成を含めて
自動化する
Configuration:Configuration Management
agile Infrastructure を活用するよう System Architecture を見直す
System Architecture変えないといけない
従来の考え方から脱却新しい考え方が必要
障害対応
http://www.flickr.com/photos/greg-palmer/3490903215/
MTBFMean Time Between FailureX
Failure as a Service
Failure is a failurebut
embrace failure
Your sitewill fail!
http://www.flickr.com/photos/gaelx/2912634902/
MTTDMean Time To Detect
MTTRMean Time To Resolve
余談:今年の Velocity 2010 で、印象に残った事は、Failures/Outages が起こった時の対応と Postmortem (検死報告) の在り方についての議論でした。メキシコ湾原油流出事故の教訓にちなんだんでしょう。
http://www.slideshare.net/lennysan/the-upside-of-downtime-velocity-2010-4564992
Monitoring:Measurement
VisualizeShare
http://blog.b3k.us/ooda.html
WebOps OODA LoopDiagram
ActDecideOrientObserve
ExecutionPlanningResourcing
AnalysisVisualizationCorrelation
MetricsMonitoringAlertingAlarming
Benjamin Black, [email protected]
Visible DevVisible Ops
http://www.amazon.co.jp/dp/443407296X/
THE VISIBLE OPS HANDBOOK - 見える運用
Kevin Behr (著), Gene Kim (著), George Spafford (著)
官野 厚 訳出版社: ブイツーソリューション (2006/1/6)
彼らはもう...
http://twitter.com/allspaw/status/22742505774
◯
http://twitter.com/allspaw/status/29580411400
◯
Web OperationsKeeping the Data On Time
By
John Allspaw, Jesse Robbins
Publisher:O'Reilly MediaReleased:June 2010
http://www.amazon.co.jp/dp/1449377440/
Opsの役割
http://www.flickr.com/photos/troyholden/4036820768/
Operator
Services Stability
次のステップへ
http://www.flickr.com/photos/alexthomp/3545005410/
Operator
System AvailabilityEfficiency
Dev ♥ Ops
To EnableOur Businesstogether
Culture
http://www.flickr.com/photos/24443965@N08/3285794855/
PeopleProcessTools
http://www.flickr.com/photos/pmtorrone/23362393/
DevOpsCultural and
Professional Movement
Question
http://www.flickr.com/photos/statelibraryqueensland/4903167275/
Appendix:John Allspaw (twitter) (blog)・SlideShare・10+ Deploys Per Day: Dev and Ops Cooperation at Flickr (どのプレゼンも素晴らしいですが、このスライドは必ず見てください)
Andrew Clay Shafer (twitter) (blog)・SlideShare・DevOps ‒ You’re Doing IT Wrong « fate = will && choice || circumstance
Adam Jacob (twitter)・SlideShare・Velocity 2010 ‒ Adam Jacob on DevOps・Choose Your Own Adventure OSCON 2010 Edition (PDF)
Appendix:John M Wills (twitter)・SlideShare・What Devops Means to Me
Damon Edwards (twitter) (blog)・DevOps is not a technology problem. DevOps is a business problem.
PodCast:・DevOps Cafe・IT Management Podcast
Conference / UnConference:・DevOps DAYS・OpsCamp・O’REILLY Velocity
Appendix:日本語での情報:・すべての道は DevOps へと : DevOps ‒ mizzy.org ‒ Trac・DevOps: OpenSource Provisioning Toolchain : DevOps ‒ mizzy.org ‒ Trac・DevOpsって何?
・クラウドと「DevOps」を理解する ‒ builder by ZDNet Japan
My Blog:・すでにここにある雲 DevOps
このスライド:・http://www.slideshare.net/nxhack