recent advances in grid use developed at cnes: the virtual private facilities (wgiss-20, kiev )
DESCRIPTION
Recent advances in GRID use developed at CNES: The Virtual Private Facilities (WGISS-20, Kiev ). Earth Observation as a tool for Services Virtual Private facilities & GRIDs The Emulation way (EPs versus IPs) Practical consequences. Jean Pierre Antikidis - PowerPoint PPT PresentationTRANSCRIPT
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Recent advances in GRID use developed at CNES: The Virtual Private Facilities
(WGISS-20, Kiev )
Earth Observation as a tool for Services Virtual Private facilities & GRIDs The Emulation way (EPs versus IPs) Practical consequences
Jean Pierre AntikidisCNES Programme & Strategy Directorate-ARP"Space Information Systems”[email protected]
JP Antikidis, WGISS20-Kiev, 12 sept 2005
To day EO large scale objectives
World-wide follow-on ozone
World-wide follow-on of sea state (MERCATOR)
World-wide follow-on of vegetation index
World-wide follow-on of crop and food resources
GMES programme
GEO endeavour
....
Can we afford on the long run to pile-up investments as we do today ?
JP Antikidis, WGISS20-Kiev, 12 sept 2005
What is a Software based service ?
It is a set of Hardware , Software and user interfaces connected in such a way a given end-user service is provided.
To make it available we need practically:
# To decide on the needed HW arrangement
# To decide on a suitable SW running on this HW
# To decide on machines interconnections and interfaces with final users
This can be made on a real fashion (Private Facilities) or rely on an exact virtual equivalent (Virtual Private facilities).
=> One possible way to implement a VPF is to “Emulate” the needed HW/SW/Files and Connections topology
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Some upcoming conceptsService is the Goal and is achieved by creating Hw/Sw facilities:
The Network story
PN: Private Network
VPN: Virtual PrivateNetwork=VP+distr.Networks
X. Jiang, D. Xu, “VIOLIN: Virtual Internetworking on OverLay INfrastructure”, Perdue Un.
Combined Services (Network+computers+data)
PF: Private Facilities
VPF: Virtual Private facilities=PF+GridsEmulation is one way to create a VPF =>EVPF
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Exemple of to-day computer systems definition for a given Service
JP Antikidis, WGISS20-Kiev, 12 sept 2005
A Software service created as a VPF
VPF=MACHINES+PROCESSES+FILES+CONNECTIONS
Machine 1
Machine 7
Machine 3
Machine 2
Machine 4
Machine 6
Machine 5
Sw3
Sw7
Sw6Sw5
Sw4
Sw3
Sw2Sw1
Real File 1
Real File 2 Real access to Web service Real File 3
Virtual File
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Experiment Setup(ref:violin project)
Physical Cluster(ITaP)
Two mutuallyisolated virtual clusters
VM
Physical Switch
VS VS
JP Antikidis, WGISS20-Kiev, 12 sept 2005
JP Antikidis, WGISS20-Kiev, 12 sept 2005
JP Antikidis, WGISS20-Kiev, 12 sept 2005
The HEAVEN paradigm
"The result produced by running an "emulator" is equivalent to the result produced by the real system."HEAVEN can "mimic" any required system
Instead of creating complex processing system, perhaps better describe them in a powerfull and fully flexible computing topology and make the resulting emulation running as a replacement for the real system.
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Simple emulation
Game machine Emulation on a PC
PC emulation in a PC (looks stupid but …)
SW1
SW2SW2
SW1
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Somewhat more interesting…
Emulation of a complex PC set of machines with data storage & multi-Hw/Sw layout ("Still stupid but not that much in a GRID based system") Key issue: Should power compute resources becomes cheap and flexible enough, perhaps better exchange flexibility and full Sw compatibility with raw CPUs power=>> GRID is bringing the needed characteristics for such an evolution
SW1
SW2
SW3SW1
SW2
SW3
JP Antikidis, WGISS20-Kiev, 12 sept 2005
A Software service created as a EVPF
EVPF= Emulation of (MACHINES+PROCESSES+FILES+CONNECTIONS)
Machine 1
Machine 7
Machine 3
Machine 2
Machine 4
Machine 6
Machine 5
Sw3
Sw7
Sw6Sw5
Sw4
Sw3
Sw2Sw1
Real File 1
Real File 2 Real access to Web service Real File 3
Virtual File
Exchange Virtual File
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Building elements for a given machine
Emulated machine Standard SW Exchange Files File services
Machine “n”
Software “n”
Virtual Real
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Multitasking environnement
Reproduces the full topology needed for a given VPF
Machine “n”
Software “n”
Machine “n”
Software “n”
Machine “n”
Software “n”
Files & data
JP Antikidis, WGISS20-Kiev, 12 sept 2005
JP Antikidis, WGISS20-Kiev, 12 sept 2005
JP Antikidis, WGISS20-Kiev, 12 sept 2005
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Multitasking environement implementation
1) Create as many machine Emulators as needed machines by either:
- Creating a machine HW emulator and plug a genuine O/S on it
- Emulate a full compound of machine+O/S emulator
2) Plug without change the applications softwares associated to any of the needed machines
3) Describe inter-machine data exchanges with respect to their content (can correspond to real or virtual files)
The result behave in an asynchronous fashion as the full equivalent to the real implementation
JP Antikidis, WGISS20-Kiev, 12 sept 2005
“With VMWare, you have the option to run your choice of 'primary' OS at all times. Then, when you need access to the other OS all you need to do is start up a vmware session. VMware includes file system shares to facilitate getting files betweeen one environment and the other, and it automatically supports a seamless NAT translation that allows your virtual machine to have full access to the Internet”
The Emulation Packet concept (VMWare based example)
Emulation Packet (1 EP per Machine to reproduce)
EP
JP Antikidis, WGISS20-Kiev, 12 sept 2005
HW EMULATOR
O/S IMPLEMENTATION
FILES DEFINITIONS (METADATA)
“Emulation Packet”
APPLICATION SW
Autonomous processing packet containing:The HW emulator (To be developed once for a given machine brand)The genuine O/S (in theory the one already available)The native user application SW (no change)The Files description associated to the given machine (as created by the GUI)
STANDALONE “Emulation Packet” (1 EP per Machine)EP
JP Antikidis, WGISS20-Kiev, 12 sept 2005
EPs versus IPs (Grid versus Internet)
Same stand with “EP” concept (these are much bigger packets of course !) in which Emulation packets once generated becomes fully autonomous with respect to their location or timing. This allows the full compatibility of millions of EPs at a time corresponding to perhaps several thousand of “Private facilities” that look from the user viewpoint as a pseudo HW/SW service implementation under his full control.
The efficiency of Internet principle rely on the IP concept by which a data packet once submitted to the system becomes autonomous (no time nor routes relations in between the packets)
This allow Internet to handle on an unvisible fashion Billions of packets at a time without need for strong centralised means.
JP Antikidis, WGISS20-Kiev, 12 sept 2005
EP
EPEPEP
EP
EPEPEP
Power Computing Resources
Graphical Interface for Service developer
Emulation Packet prep.XML descr.
End userReal interfaces
Information provider
Virtual Private Facilities created by Emulation
UpperWare
UpperWare
JP Antikidis, WGISS20-Kiev, 12 sept 2005
GRID environement implementation
1) Create as many Emulation Packets (“EP”s) machine Emulators as needed machines by either:
- Creating a machine HW emulator and plug a genuine O/S on it
- Emulate a full compound of machine+O/S emulator
2) Plug without change the applications softwares associated to any of the needed machines
3) Describe inter-machine data exchanges with respect to their file content (can correspond to real or virtual files)
The result behave in an asynchronous fashion as the full equivalent to the real implementation
JP Antikidis, WGISS20-Kiev, 12 sept 2005
HEAVEN Systems and services deployments onto a GRID based on autonomous “Emulations Packets”
EP EP
EP
EP
EPEP
EPEP
EPEP
EPEP
UPPERWARE
EPEPEPEP
GRID
GUI
Files Web Service
Interfaceto GRID
Ftp?
Urls ?
JP Antikidis, WGISS20-Kiev, 12 sept 2005
• A grid can generate significant value from idle resources but:– Users must be given absolute control of their environment.– Owners must retain absolute control of their machines.
• Virtualisation/Emulation enables this to happen:– Administration can be simple enough to become a fixed cost.– Security concerns can be minimised almost to the “anonymous” level.
• We still need new technology to make this happen, but we may be very close……
Quote CNES=>>> VFC, EVFC,WAG are such a Tool
CONCLUSION, Quote: "Overview of New technologies for using Grids by :Rhys NewmanManager of Interdisciplinary Grid Development, Oxford University
JP Antikidis, WGISS20-Kiev, 12 sept 2005
Virtualisation is the Key
• To provide these properties, reliably and uniformly on heterogeneous resources, the user’s environment must be “Virtual”.
• Simplifies the administration:– The “service end” of eScience can focus on the provision of
“virtual hardware”– The “user end” of eScience can focus on developing applications
in a standardised environment.
• Imagine M users and N owners, each wanting their concerns addressed:– Without Virtualisation: MxN negotiations.– With Virtualisation: M+N negotiations.