Download - Virtual Server Self-Service Provisioning
VIRTUAL SERVER SELF-SERVICE PROVISIONING
Juraj SucikSystem Architect
CERN
CERN
What means « »?
European
Organization for
Nuclear
Research
European
Council for
Nuclear
Research
CERNCERN
Conseil
Européen pour la
Recherche
Nucléaire
1952
1954
Organisation
Européenne pour la
Recherche
Nucléaire
European Laboratoryfor Particle Physics
The largest particle physics lab in the world
Twenty Member StatesAustria, Belgium, Bulgaria, Czech Republic, Denmark, Finland, France, Germany, Greece, Italy, Hungary, Netherlands, Norway, Poland,Portugal, Slovakia, Spain, Sweden, Switzerland, United Kingdom
Annual budgetin 2007982 MCHF (610 MEUR)
External fundingfor experiments
Eight Observer StatesEuropean Commission, USA, Russian Federation,
India, Israel, Japan, Turkey, UNESCO
People2415 Staff730 Fellows and
associates200 Students9133 Users2000 External Firm
Do fundamental researchBy answering questions like the structure of matter…
4th - 5th
centuryBC
End of
19th
century
Beginning of
20th
century1960s
Checking existing theories: the standard modelLEPTONS QUARKS
ORDINARYMATTER
GLUONS
Strong Force
PHOTONS
Electro-Magnetic Force
BOSONS
Weak Force
GRAVITONS
Gravity
Images: www.particlezoo.net
ELECTRON ELECTRONNEUTRINO
MUON MUONNEUTRINO
TAU TAUNEUTRINO
UP DOWN
CHARM STRANGE
TOP BOTTOM
4forces
Answering fundamental questions…• How to explain particules have a mass?
Newton could not explain, neither can we…
• What is 96% of the Universe made of ?We can only see 4%of its estimated mass!
• Why isn’t there antimatterin the Universe?Nature should be symetric…
• What was the state of matter justafter the « Big Bang » ?Travelling back to the earliest instants of the Universe would help…
HiggsBoson
Bringing nations together and educate• Biggest international scientific
collaboration
• Various students programmes
• Over 100 countries
• Hundreds of physics institutes
• Half of the world’s particle physicists
By accelerating and colliding objects…
At incredible levels of energy!
E=mc2
The largest particle accelerators17 miles (27km)long tunnel
Thousands of superconducting magnets
Ultra vacuum:10x emptierthan on the Moon
Coldest placein the Universe: -271° C
In safe conditions!
The biggest and most sophisticated detectors
13
Cathedrals of science100m underground
600 million collisionsper second detectedby hundreds ofmillion sensors
Thousands of collaboratorsfor each detector
In safe conditions!
Practical applications: the World Wide WebWas developed in the frame of the LHC project in 1989!
Freely given to the World!
Practical applications: cancer treatmentFor both detection and cure of cancers
PET Scans
Hadron Therapy
Practical applications: detectorsScanning trucks in less than one hour without unloading them!
Practicalapplications:using the Grid
Ultra high-speed processingof satellite imagery in thecase of natural disasters
And of course… some Nobel prizes!
George Charpak
“for his invention and development of particle detectors, in particular the multiwire proportional chamber”
Carlo Rubbia(with Simon van der Meer)
“for their decisive contributions to the large project, which led to the discovery of the field particles W and Z, communicators of weak interaction”
GRID
IT Infrastructure at CERN
• General Purpose Computing Environment• Administrative Computing Services• Physics and engineering computing• Consolidation, coordination and standardization of computing
activities• Physics applications
(e.g., for data acquisition& offline analysis)
• Accelerator design and operations
LHC Data Every Year
• 40 million collisions per second• After filtering, 100 collisions of interest per second• > 1 Megabyte of data digitized per collision recording rate > 1 Gigabyte / sec• 1010 collisions recorded each year stored data > 15 Petabytes / year
• analysis requires a computing power equivalent to ~ 100,000 of today's fastest processors
Computing power available at CERN
• High-throughput computing based on reliable “commodity” technology
• More than 35’000 CPUs in about 6000 boxes (Linux)
• 14 Petabytes on 14’000 drives (NAS Disk storage)
• 34 Petabytes on 45’000 tape slots with 170 high speed drives
Nowhere near enough!
Computing for LHC
• Problem: even with Computer Centre upgrade, CERN can provide only a fraction of the necessary resources
• Solution: Data centers, which were isolated in the past, will be connected, uniting the computing resources of particle physicists worldwide
Users of CERN
Europe: 267 institutes4603 usersOut of Europe: 208 institutes1632 users
What is the Grid?
• The World Wide Web provides seamless access to information that is stored in many millions of different geographical locations
• In contrast, the Grid is an emerging infrastructure that provides seamless access to computing power and data storage capacity distributed over the globe
The most extensive scientific computing grid15 Petabytes(15 millions of GB)of data every year
100’000 processors
200 computer centres around the planet
Should run 100 millions jobs
Used by 5000 scientists in 500 institutes
Virtualization
Why virtual?• Steady flow of requests for dedicated servers
in the CERN computer centre– Excellent network connectivity– Reliable power supply, cooling– 24x365 monitoring with operator’s presence– Daily tape backup– Use the hardware without owning the responsibility (maintenance,
procurement)– Focus on application without sharing the resources– Improve the CPU utilization of grid nodes– Optimize TCO
Buy and maintain physical
hardware (and computer
centre)
Virtual infrastructure
(IaaS)
• Extra effort to procure and maintain HW
• Delivery time in several weeks
• Lack of flexibility• Not easy to adapt to
dynamic patterns
Infrastructure as a Service (IaaS)
• Ready in ~ minutes• Highly flexible• Efficient capacity
planning
Experience since 2006• Server Self Service Center (S3C)
o Choose your server from a set of predefined imageso Take resources from the pool of available HWo Available within minutes
Requirements have evolved• New requirements identified
o Flexibility of resource allocationo Higher performanceo High-availability model
adapted to customers• Larger scale
o Efficient management
Source: Gartner (August 2008)
Why Hyper-V?• A built-in component of the operating system• Create powerful VM• 64-bit support for guests• Linux support• High availability• Quick migration• Manageability• High performance, reliability, security• VHD compatibility
Why SCVMM?• One solution to centrally manage all virtual infrastructure• Windows Powershell API• V2V and P2V capabilities• Web portal• Intelligent placement• Library• Templates• Delegated management roles• Job history• Support for highly available VM• VM Migration
Hyper-V Infrastructure
System Architecture
Microsoft Virtual Machine Manager
Windows Powershell
SOAP Services
Virtual Machine Manager
AdminConsole
CERN Virtual InfrastructureWeb Interface
Backups
OS Maintenance
LAN DB
Application Management
Web Portal
Challenges• Console access from Linux• Missing .Net API for SCVMM• Time sync issues in guests
Experiences• Cost efficient customized cloud computing infrastructure• Maintenance with limited downtime • Disaster recovery of VMs within minutes• Improved performance compared to Virtual Server 2005
Source: http://blogs.msdn.com/modonovan
Experiences• 172 running virtual machines• 17 templates• Scalable to large number of VM• Expiration handling• Green computing
January
February
March
April May JuneJuly
August
Septem
ber
October
0102030405060708090
Monthly increment
January
February
March
April May JuneJuly
August
Septem
ber
October
0
50
100
150
200
250
300
350
Number of guests
Real life use cases• Video streaming for LHC First Beam Day
6 virtual machines needed for ~1 week• Terminal Servers for Engineering Apps
A terminal servers installed with older version of the apps• Oracle Application servers
Real life use cases• CERN Media Archive• CERN Alerter web server
Physical server with 2xCPU, 4GB RAM Upgrade necessary because of OS driver issue Virtual server set up “on demand” Resources limited to 1xCPU, 2 GB RAM
• Physics analysis running in VM• Etc, etc.
Future work• Upgrade to Hyper-V 2.0 & SCVMM 2008 R2• Use the new “Cluster Shared Volume” feature• Use the new “Rapid Provisioning” feature• VDI functionality
Conclusion• Innovative physics research laboratory• Pushing latest technology to its limits• Moving services to the cloud
Visit our websites:Informations: www.cern.chCERN TV: www.youtube.com/cernRecruitment: www.cern.ch/jobs