opennebulaconf 2013 - community clouds from scratch jordi guijarro
DESCRIPTION
Cloud Computing represents a radical change in the way we organize and use computing resources and storage. The scientific and academic communities face the challenge of not only adapting their procedures to this new paradigm, but also contributing Cloud Computing development and leading its evolution towards open, secure and interoperable computing infrastructures, which will playing a key role in the community clouds paradigm. The Spanish MEGHA initiative promotes and coordinates contributions to cloud computing R&D, education and management made by institutions affiliated with RedIRIS [7] in Spain. In the first phase (2010–2012), MEGHA validated federated cloud platforms using Opennebula and OCCI [10] to streamline the use of cloud technologies among R&E service centers. Representative infrastructure providers (CESCA, CESGA, PIC), middleware providers (OpenNebula, RedIRIS, OSAmI-Commons) and users (UAB, UOC, UM) together with intermediate/identity/brokers resources (RedIRIS) joined efforts to demonstrate the viability of this approach. The results stimulated the development of use cases including e-learning platforms on demand (Learning Apps project), a distributed HPC platform (e-Science), and Virtual Labs (VDI) in a hybrid scenario (Academic services). Next Steps? As next goal, the Spanish research and academic community is working to assess the possibilities of creating a productive Infrastructure Cloud Computing service within member institutions. With this new approach new challenges appear: Federated user authentication and authorization mechanisms. Brokering architecture scenario. Secure VM image distribution and validation. A federated cloud accounting system integrating the accounting records of multiple cloud managers and supporting federated cloud governance. Monitoring and notification of unpredictable changes in availability and readability status. Security Policies and Service Level Agreements (SLA’s). Bio: Jordi Guijarro, Operations and Security Manager at CESCA, obtained his Bachelor in Computer Engineering at Universitat Oberta de Catalunya (UOC) and his Bachelor in Systems Technical Engineering at Universitat Autònoma de Barcelona (UAB). Infrastructure and systems architect specializing in virtualization, cloud computing and cybersecurity, with more than 10 years of experience in areas related to Computer Systems starting his career as a Systems Technician in the private sector. From 2001 he has worked for UAB Computer Services as a Systems Engineer and subsequently at UOC as an Operations Analyst. He also combines his work at CESCA with the participation in several national projects, and collaborates with UOC as a Consultant on the Master of Free Software.TRANSCRIPT
Community Community Clouds from Scratch Clouds from Scratch A Real World Introduction
[email protected] – Berlin 2013 OpenNebula Conf 2013
Jordi Guijarro - [email protected]@cloudadms
About
What Cloud What Cloud Computing really Computing really means for R&E means for R&E A strategy, a tactical solution...
[email protected] – Berlin 2013 OpenNebula Conf 2013
A Research and Education Network is a non-profit Internet service provider dedicated to supporting the needs of the research and education communities.
It has a high-speed backbone network.
Research and Education Networks are usually the places where new Internet protocols and architectures are introduced before their public deployment:
R&E Networks
Protocols: IPv6 and multicast.
Architecture: Client/Server and Cloud Computing
[email protected] – Berlin 2013 OpenNebula Conf 2013
R&E Networks
[email protected] – Berlin 2013 OpenNebula Conf 2013
JUST A BUZZWORD?What Cloud Computing really means for R&E Community ?
[email protected] – Berlin 2013 OpenNebula Conf 2013
What Cloud Computing really means
SELFSERVICE(s)?
jguijarro@cescajguijarro@cesca,.cat – Reykjavik 2012,.cat – Reykjavik 2012 TNC 2012TNC [email protected] – Berlin 2013 OpenNebula Conf 2013
What Cloud Computing really mean for R&E Community?
RESPONSIBILITIES...
jguijarro@cescajguijarro@cesca,.cat – Reykjavik 2012,.cat – Reykjavik 2012 TNC 2012TNC [email protected] – Berlin 2013 OpenNebula Conf 2013
MEGHA Working MEGHA Working
GroupGroup Let's pull all together...
jguijarro@cescajguijarro@cesca,.cat – Reykjavik 2012,.cat – Reykjavik 2012 TNC 2012TNC [email protected] – Berlin 2013 OpenNebula Conf 2013
MISSIONMEGHA Working Group
the development of innovation in Cloud Computing technologies within the scope of the institutions affiliated to RedIRIS
for the application of open and interoperable Cloud technologies in other areas in which it can influence, such as business in government.
TO PROMOTE
TO ACT AS A CATALYST
INTEROPERABLE
OPEN
PROMOTION
USE CASES
TESTBED
IDENTITY
http://wiki.rediris.es/megha
[email protected] – Berlin 2013 OpenNebula Conf 2013
Deploy a federated Cloud infrastructure as a platform for innovation and evaluation.
Explore the possibilities for the application of Cloud computing has in research, academic and management environments.Facilitate the creation of an ecosystem of open-source middleware components to deploy interoperable Clouds and coordinate training and education activities for its deployment.
MAIN OBJECTIVESMEGHA Working Group
[email protected] – Berlin 2013 OpenNebula Conf 2013
MEGHA IntercloudMEGHA Intercloud
projectproject From myth to reality...
[email protected] – Berlin 2013 OpenNebula Conf 2013
INITIAL STAGEMEGHA Intercloud Project
[email protected] – Berlin 2013 OpenNebula Conf 2013
CESCA Cloud InfrastructureCESCA Cloud Infrastructure
CESGA Cloud InfrastructureCESGA Cloud Infrastructure
INTEROPERABILITYMEGHA Intercloud Project
A “Virtual Poka Yoke” ?A “Virtual Poka Yoke” ?
[email protected] – Berlin 2013 OpenNebula Conf 2013
OPEN PLEASE! MEGHA Intercloud Project
Director,...
[email protected] – Berlin 2013 OpenNebula Conf 2013
INTEROPERABLEMEGHA Intercloud Project
[email protected] – Berlin 2013 OpenNebula Conf 2013
FEDERATED MEGHA Intercloud Project
[email protected] – Berlin 2013 OpenNebula Conf 2013
FEDERATED MEGHA Intercloud Project
PKIRISGRIDX.509 Cert
UI
OpenN
ebula
3.8
r
[email protected] – Berlin 2013 OpenNebula Conf 2013
Public high speed connectivity.
Images, instances and networks management.
SSH access (root).User manual & Service Provider Howtos (ONE 3.x)“Terms of Service”.
MATURING TOGETHER
[email protected] – Berlin 2013 OpenNebula Conf 2013
MAIN PLAYERS MEGHA Intercloud Project
[email protected] – Berlin 2013 OpenNebula Conf 2013
Some Real World Some Real World
User CasesUser Cases Cloudified...
[email protected] – Berlin 2013 OpenNebula Conf 2013
UOC: Learning Apps
REAL WORLD USER CASES
A store on the internet inspired by Apple's App Store which focuses on educational applications
The provision of the educational environments is automated (OCCI 0.8) and the amount of infr. depends on parameters such as a preconfigured concurrency.
RESTFUL API
http://www.learningappsstore.org/
ON DEMAND
ACADEMICACADEMIC
EDUCATION
AUTOMATION
CONTEXTUALIZATION
[email protected] – Berlin 2013 OpenNebula Conf 2013
[email protected] – Berlin 2013 OpenNebula Conf 2013
PIC: Grid & CloudREAL WORLD USER CASES
DIRAC (Interware) allows homogeneous access to heterogeneous computing resources.
Providing transparent access and interoperability between Grids and Clouds OCCI 1.1 interface.
SCIENTIFICINTEGRATION
CONTEXTUALIZATION
DIRACINTERWARE
Víctor Méndez (www.pic.es)
[email protected] – Berlin 2013 OpenNebula Conf 2013
PIC: Grid & Cloud
REAL WORLD USER CASES
DIRAC WMS
TaskQueue
VM Scheduler
VM Director
MeghaCloudInfrastructure
Command-line
DIRAC API
Web Portal
VirtualMachine
UserAdmin
Image Repo
Cloud Manager
DIRAC CS
Cloud Driver
Integration Design: VM Job Running
Víctor Méndez (www.pic.es)[email protected] – Berlin 2013 OpenNebula Conf 2013
PIC: Grid & CloudREAL WORLD USER CASES
Víctor Méndez (www.pic.es)
DIRAC WMS
TaskQueue
VM Scheduler
OCCI Director(VM Director)
MeghaCloudInfrastructure
VirtualMachine
Image Repo
OCCI / OpenNebula
?
OCCI Image
OCCI RESTful Client
Integration Design: Occi VM Submission
[email protected] – Berlin 2013 OpenNebula Conf 2013
PIC: Grid & Cloud
REAL WORLD USER CASES
jguijarro@cescajguijarro@cesca,.cat – Reykjavik 2012,.cat – Reykjavik 2012 TNC 2012TNC 2012
Virtual Machine
VM Monitor
Job Agent
DIRAC WMS
TaskQueue
VM Manager
Job
Cloud Manager
VirtualMachine
MeghaCloud Infrastructure
VMVM
VM
Job Run
CPU statistics
Job State Update
Job Matcher
Víctor Méndez (www.pic.es)
Integration Design: VM Job Running
[email protected] – Berlin 2013 OpenNebula Conf 2013
Next Steps... Next Steps... This is only the tip of the iceberg
[email protected] – Berlin 2013 OpenNebula Conf 2013
DETECTED PROBLEMSNEXT STEPS...
[email protected] – Berlin 2013 OpenNebula Conf 2013
Small and medium Scientific Research centersdon ́t have the manpower or know-how tobuild a 24/365 data center
Server deployment takes a long time
Can t́ solve peak problems
Usually can ́t have more than one data center
TEST PLATFORMNEXT STEPS FOR...
VO's ? SUNSTONE FEDERATION ?
COMPATIBILITY AND FEATURES
TO BUILD A MARKETPLACE ?
[email protected] – Berlin 2013 OpenNebula Conf 2013
REAL SCENARIONEXT STEPS...
[email protected] – Berlin 2013 OpenNebula Conf 2013
Some Some
conclusions... conclusions... Options to cater to all tastes!
[email protected] – Berlin 2013 OpenNebula Conf 2013
NEW CHALLENGESNEXT STEPS...
[email protected] – Berlin 2013 OpenNebula Conf 2013
Federated user authentication and authorization mechanisms and user management between different cloud managers.
Secure VM image distribution and validation.among heterogeneous cloud managers.
Federated Cloud accounting system integrating the accounting records of multiple cloud managers and supporting the Federated Cloud governance.
Monitoring and notification of unpredictable change of availability.
Distributed scenario
FUTURE COMMUNITY FEDERATED CLOUD MODELFUTURE COMMUNITY FEDERATED CLOUD MODEL
Scalability (Commercial prov. included)
Resource sharingHigh Availability
Efficient use of infrastructure
[email protected] – Berlin 2013 OpenNebula Conf 2013
Strategy, strategy, strategy, ...
Dedicated to those who made MEGHA possible.
Gràcies! Gràcies! Gracias! Gracias! ThanksThanks! !
Jordi Guijarro [email protected]
@cloudadms