novel network computing paradigms (i)3 computer, computing and communication tech. evolution *...
TRANSCRIPT
1
Lecture 4
Novel Network Computing Paradigms (I)
Graduate Course, Hosei U., J. Ma
Internet/Web Service-oriented Computing
on Cyber Worlds
2
Outline
Web Computing
Web Service
Semantic Web
Web Intelligence
Grid Computing
P2P Computing
Internet Distributed Computing (IDC) / Global Computing (GC)
Service Computing, EaaS, Cloud Computing
3
Computer, Computing and Communication Tech. Evolution
* Sputnik
1960 1970 1975 1980 1985 1990 1995 2000
* ARPANET* Email
* Ethernet* TCP/IP * IETF
* Internet Era
* WWW
* Mosaic* XML
* PC Clusters
*Crays *MPPs
* Mainframes
* HTML* W3C
*P2P*Grid
* XEROX PARC wormCom
pute
rCom
munic
ation
*Semantic Web
* Minicomputers
* PCs* WS Clusters * PDAs
* Workstations* HTC
*Ubiquitous
*Pervasive
*Web Services
*Mobicomp
Com
puting
* Parallel Comp
*Webcomp
* Distributed Comp* M-Media Comp
*IDC/SC/CC
2005
*AmI
* Cell Phone
*CPS
4
Web Services - What
CGI, ASP/JSP, RMI, CORBA
Web services is industry initiative to standardize mechanisms
to describe, locate, and communicate with online applications
Applications become accessible Web service components, can
be published and used by other Web service components
Based on technologies standardized through W3C (WWW
Consortium), but not necessarily tied to ―the‖ Web
Service
providers
Service
requesters
Service
brokers
Create services,
publish by
registering with
brokers
Find services from
registry, bind their
application to service
provider
Maintain
service registry
5
The Web Today
Publishers
Clients
Today Web services
Servers
Create and publish
web page
Create and publish
a web service
Browsers display
an HTML web page
Clients understand
WSDL and can find
and bind to services
Host HTML web
pages
Provide a service
E-mail WWW XML Web Services
Connects
People
Connects
People to
Information
Connects
Applications
6
Web Services - Conceptual Stack
Service publicationUDDI
Service description
XML-based
messaging
Network
protocol
WSDL
SOAP
HTTP, SMTP,
FTP, BEEP
Secu
rity
Ma
na
gea
bility
Qu
ality
of s
erv
ice
Service discovery
Service flow
UDDI
WSFL (web service
flow language
7
Web Services - Core Technologies
Core Web-based technologies are important for
interoperability - a central problem in distributed computing
XML (extensible markup language): specifies structure of
document, and can define arbitrary cross-platform structures
Web SOAP (simple object access protocol): provides an
envelope that encapsulates XML data for tansfer via HTTP
between Web services and client applications
WSDL (web services description language): describes a
service in XML using XML schema, (ie, programmatic
interface of service) similar to an interface definition language
UDDI (universal description, discovery, and integration): for
distributed registries of web services, to publish/find/bind,
similar to yellow/white pages services
8
Service Oriented Architecture
3 roles (an entity may have multiple roles at same time):
Service provider, Service broker, Service requestor (client)
3 operations:
Publish: provider publishes a service description in a UDDI repository
Find: a requestor finds a service in UDDI repository (like yellow pages)
Bind: after finding a service, a client ―binds‖ to that service
Web service
registry
WS client WS provider
XML/SOAP/HTTP
UDDI/WSDLUDDI
1. Publish: provider updates
registry about available services
2. Find: Client sends a
request for a service
Registry tells client
about service
3. Bind: client interacts
with service provider to
invoke service
Service
description
Service
9
SOAP initially created by Microsoft 1999, later joined by IBM,
Lotus, UserLand, Developmentor
SOAP is simple, lightweight protocol for exchanging XML
SOAP message = a header and body, wrapped by an
―envelope‖ element
SOAP envelope = (optional) header and (mandatory) body
Header contains information for security (eg, related to
encryption), routing, recipient’s handling of message (eg, enable
detection of retransmitted duplicate messages; error handling and
notification)
Body = XML message<SOAP:Envelope xmlns:SOAP=
http://schemas.xmlsoap.org/soap/envelope/><SOAP:Header>
<!- content of header goes here -></SOAP:Header><SOAP:Body>
<!- content of body goes here -></SOAP:Body>
</SOAP:Envelope>
SOAP (www.w3.org/TR/SOAP)
10
Semantic Web - What
The Semantic Web provides a common framework that allows data
to be shared and reused across application, enterprise, and
community boundaries. It is a collaborative effort led by W3C with
participation from a large number of researchers and industrial
partners. It is based on the Resource Description Framework (RDF),
which integrates a variety of applications using XML for syntax and
URIs for naming.
―The Semantic Web is an extension of the current web in which
information is given well-defined meaning, better enabling computers
and people to work in cooperation." -- Tim Berners-Lee, James
Hendler, Ora Lassila, Scientific American, May 2001.
The Semantic Web is a vision: the idea of having data on the web
defined and linked in a way that it can be used by machines not just
for display purposes, but for automation, integration and reuse of data
across various applications.
11
Semantic Web - Goal
Original driver: Automation
• it would be nice if computers could do more (on the Web)
• solution: make information on the Web more ―machine-
friendly‖
• origins of the Semantic Web are in web metadata
Short term goal: Interoperability
• combining information from multiple sources
• Web Services: discovery, composition
• ―serendipitous‖ interoperability
Long term goal: “Departure from the Tool Paradigm”
• instead of using computers like tools, make them work on our
behalf
• removing humans from the loop to the extent possible
12
Semantic Web Architecture
Semantic Web is built in a layered manner
Web Ontology Language (OWL)
What is an ―ontology‖
1. a controlled vocabulary
2. a concept taxonomy
3. other relations between concepts
4. definition: A specification of conceptualization
13
Web Intelligence
Web Intelligence (WI) exploits the fundamental and practical impact that advanced Information Technology (IT) and innovative Artificial (AI) will have on the next generation Web WI = IT + AI
The goals of WI are too broad to meaningful !!
14
Four Levels of WI Support
Internet-level communication, infrastructure, and security protocols
Interface-level multi-media presentation standards
Knowledge-level information processing and management tools
Application-level ubiquitous computing and social intelligence utilities
Level-1
Level-2
Level-3
Level-4
support
15
Computational Grids
A single virtual computing system consisting of distributed
heterogeneous systems and resources, connected by high
speed networking
Key is large scale sharing of resources -->
More resources are accessible to each application
More efficient resource utilization
Computing can be distributed -->
Better match between applications’ needs and specialized
resources
Better application performance if computing can be parallelized
Resources are accessible any time to mobile users (independent
of location or device)
16
A Typical Grid Computing Environment
Grid Resource Broker
Resource Broker
Application
Grid Information Service
Grid Resource Broker
databaseR2R3
RN
R1
R4
R5
R6
Grid Information Service
17
Grid Application Drivers
Distributed HPC (Supercomputing):
Computational science.
High-Capacity/Throughput Computing:
Large scale simulation/chip design & parameter studies.
Content Sharing (free or paid)
Sharing digital contents among peers
Remote software access/renting services:
Application service provides (ASPs) & Web services.
Data-intensive computing:
Drug Design, Particle Physics, Stock Prediction...
On-demand, realtime computing:
Medical instrumentation & Mission Critical.
Collaborative Computing:
Collaborative design, Data exploration, education.
Service Oriented Computing (SOC):
Towards economic-based Utility Computing: New paradigm, new
applications, new industries, and new business.
18
Grid Challenges and Technologies
Security
Resource Allocation
& Scheduling
Data locality
Network Management
System Management
Resource Discovery
Uniform Access
Computational Economy
Application Construction
19
Networked Resources across Organizations
Computers Networks Data Sources Scientific InstrumentsStorage Systems
Local Resource Managers
Operating Systems Queuing Systems Internet ProtocolsLibraries & App Kernels
Distributed Resources Coupling Services
Information QoSProcess
Development Environments and Tools
Languages/Compilers Libraries Debuggers Web tools
Resource Management, Selection, and Aggregation (BROKERS)
Applications and Portals
Prob. Solving Env.Scientific…
CollaborationEngineering Web enabled Apps
Trading
…
…
…
…
APPLICATIONS
SECURITY LAYER
Security Data
CORE MIDDLEWARE
USER LEVEL
MIDDLEWARE
Monitors
Layered Grid Architecture
20
Some Grid Projects & Technologies Australia
Nimrod-G
Gridbus
GridSim
Virtual Lab
DISCWorld
GrangeNet
Europe
UNICORE
Cactus
UK eScience
EU Data Grid
EuroGrid
MetaMPI
XtremeWeb
and many more.
India
I-Grid
Japan
Ninf
DataFarm
Korea...
N*Grid
USA
Globus
Legion
OGSA
Sun Grid Engine
AppLeS
NASA IPG
Condor-G
Jxta
NetSolve
AccessGrid
and many more...
Cycle Stealing & .com Initiatives
Distributed.net
SETI@Home, ….
Entropia, UD, Parabon,….
Public Forums
Global Grid Forum
Australian Grid Forum
IEEE TFCC
CCGrid conference
P2P conference
http://www.gridcomputing.com
21
Category Project Organisation Remarks
Integrated Grid
Systems
NetSolve U. Tennessee
A programming and runtime system for accessing
high-performance libraries and resources
transparently.
Ninf U. Tokyo Functionality is similar to NetSolve.
ST-ORM UPC, Barcelona A scheduler for distributed batch systems.
SILVER PPNL and U. of Utah A scheduler for distributed batch systems.
Albatross Vrije U. Object oriented programming system.
PUNCH Purdue U.A portal computing environment and service for
applications.
Javelin UCSB Java-based programming and runtime system.
XtremWeb Paris-Sud U. A global computing environment
MILAN Arizona and NY
Aims to provide end-to-end services for
transparent utilization and management of
networked resources
DISCWorld U. of Adelaide A distributed information-processing environment.
Unicore GermanyJava-based environment for accessing remote
supercomputers.
Core Middleware
Cosm Mithral A toolkit building P2P applications.
Globus ANL and ISI
Globus provides uniform and secure environment
for accessing remote computational and storage
resources.
Gridbus Melbourne UniversityEconomic paradigm for service-oriented
computing
GridSim Monash U. A toolkit for Grid simulation.
JXTA Sun MicrosystemsA Java-based framework and infrastructure for
P2P computing.
Legion U. of VirginiaA Grid operating system providing transparent
access to distributed resources.
P2P Accelerator IntelA basic infrastructure for creating P2P applications
for .NET platform.
22
Access to distributed astronomical databases and
processing.Johns Hopkins U. & Caltech
National Virtual
Observatory
NeuroscienceUCSDVirtual Instruments
Climate ModelingLLNL, ANL, &NCAREarth System Grid
Spread sheet processing Monash, QUT, & DSTCActive Sheet
AerospaceNASAIPG
Magnetic fusionPrinceton/ANL/Fusion Grid
Aerospace Design OptimisationSouthampton U. Geodise
Earthquake EngineeringNCSANEESGrid
High Energy Physics apps.Melbourne UHEPGrid
Molecular modeling for drug design.Monash U and WEHIVirtual Laboratory
High Energy PhysicsCaltech and ANLPPDG
High Energy PhysicsUCF and ANLGriPhyN
High Energy Physics, Earth Observation, BiologyCERN European Data Grid
Applications and
application
driven Grid
efforts
Tools for creating computing portals.SDSCGridPort
Grid application development tools.Rice U.GrADS
A framework for writing parallel applications. It is
developed using the MPICH-G and Globus.
Max Planck Institute for
Gravitational PhysicsCactus
MPI programming and runtime environment.RWTH, AachenMetaMPICH
A declarative language parametric programming. Monash U.Nimrod parameter
programming tools
MPI implementation on Globus.Northern Illinois U. MPICH-G
User-level
Middleware:
Programming
Environments
Economic-based Grid resource broker for
parameter sweep/task farming applications. Monash U.Nimrod-G
A wide area job processing system.U. of WisconsinCondor-G
Application specific scheduler. UCSDAppLeS
User-level
Middleware:
Schedulers
23
Globus Grid Project and Layered Architecture
Applications
Grid ServicesGRAM
GSI HBM
Nexus
I/O
Grid Fabric
LSF
Condor MPI
NQEPBS
TCP
NTLinux
UDP
Application Toolkits
DUROC globusrunMPI Nimrod/GCondor-G HPC++
GlobusView Testbed Status
GASS
Solaris DiffServ
GSI-FTPMDS
Globus Project (www.globus.org) was rooted from the I-WAY (Info. Wide Area Year) project 1995, which aimed at integrating multiple existing high speed network testbeds to support HP applications of distributed supercomputing, remote visualization, virtual reality, video
24
Globus Evolution
Incre
ased functionalit
y,
sta
ndard
ization
Time
Globus Toolkit
Open Grid
Services Arch
GGF: OGSI, …
(+ OASIS, W3C)
Multiple implementations,
including Globus Toolkit
Defacto standards
GGF: GridFTP, GSICustom
solutions
X.509,
LDAP,
FTP, …
Web services
App-specific
Services
25
Web Service Resource (WS-Resource)
WS-Resource
Open Grid Services Architecture (OGSA)
26
WS-Resource Framework (WSRF) Model
27
Convergence of Grid and Web Service
IEEE International Conference on Services Computing (SCC 2004), Shanghai, Sept., 2004
Grid vs Semantic Web?, Grid vs P2P?
28
What is P2P
Compute model – or - a set of technologies – or - an infrastructure / middleware – or, a mind-set..; no matter what
– P2P --
Enables direct exchanges between peers
Allows resource sharing with other
Takes place at the edges of the Net
[Dynamic operability] P2P applications must keep operating transparently although hosts join and leave the network frequently.
[Scalability] P2P applications exhibit what economists call the ―network effect‖ in which a network’s value to an individual user scales with the total number of participants.
[Robust] Highly robust, resilient, and self-organizing.
[Anonymity] The application should protect the privacy of people seeking or providing sensitive information.
[Equality] All peers assume equal role.
[Non Centralized] No centralized server in the space.
29
P2P Topologies
P2P apps may apply any and all of these topologies
Server
Client Client
ClientClient
Peer Peer
PeerPeer
Peer
Peer
PeerPeer
“Pure”
Client-server
Mediated
Server
Client Client
ClientClient
Hybrid
30
Taxonomy of P2P Applications
Distributed Computing
Content Sharing
Collaboration
•Internet Distributed Computing
•Intranet Distributed Computing
•Grid Computing
•File delivery
•Content Distribution
•Distributed Storage
•Caching, Edge Services
•Information Mgmt – discover
aggregate, filter, organize,..
•Communication – chat, messaging
•Co-review/edit/author/create
•Gaming
•Discovery
Collaboration is based uponforms of content sharing
31
P2P Networks/Middleware
Centralized Database Napster
Query Flooding Gnutella
Intelligent Query Flooding KaZaA
Swarming BitTorrent
Unstructured Overlay Routing Freenet
Structured Overlay Routing Distributed Hash Tables (DHT)
Others JXTA
Groove
CAN
Chord
Pastry
Koorde
32
Napster
A killer application: Napster Free music over the Internet
Key idea: share the storage and bandwidth of individual (home) users
Hybrid
No security
AB
C
D
E
F
m1m2
m3
m4
m5
m6
m1 A
m2 B
m3 C
m4 D
m5 E
m6 F
E?
m5
E? E
33
Gnutella
Distribute file location and decentralize lookup.
Idea: multicast the request
How to find a file: Send request to all neighbors
Neighbors recursively multicast the request
Eventually a machine that has the file receives the request, and it sends back the answer
Assume: m1’s neighbors are m2 and m3; m3’s neighbors are
m4 and m5;…
AB
C
D
E
F
m1m2
m3
m4
m5
m6
E?
E?
E?E?
E
34
Security and ProtectionTrust
AnonymityReputation
Intelligent Agents/Web-based Services
MatchmakingService Description
Network Architectureand Design
Network TopologyRouting
Overlay Networks
Distributed DatabasesQuery Decomposition
Query DistributionMediation
P2P
SociometrySmall World Phenomena
Power-Law Networks
Business and Legal Issues
Business ModelsIntellectual Property Rights
Distributed Data Structures
Distributed Hash TablesScalable Distributed
Data Structures
P2P Issues
Network Architectureand Design
Network TopologyRouting
Overlay Networks
35
JXTA
JXTA is a set of open, generalized peer-to-peer (P2P)
protocols that allow any connected device on the network —
from cell phone to PDA, from PC to server — to
communicate and collaborate as peers.
It is designed to be independent of prog-languages,
transport protocols and physical networks.
The JXTA protocols standardize the manner in which peers:
■ Discover each other
■ Self-organize into peer groups
■ Advertise and discover network services
■ Communicate with each other
■ Monitor each other
36
JXTA Architecture
Project JXTA software architecture is divided into three layers
37
JXTA Virtual Network and IDs
Independent logical addressing model. Every network resource (peer,
pipe, data, peergroup, etc.) is assigned a unique JXTA ID, 128-bit random
UUIDs, allowing each peer to self-generate its own IDs.
JXTA Virtual Network
38
JXTA Super-Peers
Rendezvous and Relay
39
TOMSCOP
TOMSCOP is a P2P collaboration platform that eases developments
of shared applications and creates collaborative communities.
TOMSCOP Architecture
Synchronous Message Transportation (SMT)
PeerRoom Administration
Awareness Monitor
Presence
Control
Identity
Control
Security
Control
PeerCommunication Support
Medium Processor
Text Audio Video
ApplicationSpace Management
Space
Manager
Application
Manager
Collaborative Community
JXTA Services and Virtual Network
Ubiquitous System Shared Applications
40
P2P Technology is Rebalancing Distributed Computing
Serv
er
Cen
tric
Clie
nt C
en
tric
Data Center Mid-Tier Front-End Personal
Client
Server Centric Model
Smart Client Model
Client-Server Computing
- Centralized control
- Resource intensive deployments
- High cost of MIP’s / Mbytes
Innovation is limited by deployment
resources and capital ROI requirements
Peer to Peer Computing
- Local control
- Mass deployment of new capabilities
- Low cost of MIP’s / Mbytes
Innovation is accelerated by ease of deployment
and “free” use of resources
Driving a Rebalancing
of Computing Models
41
Web Services, P2P, and Grid Computing
Web Services
Standard protocols for service oriented architecture
Impetus in business process
Grid
Infrastructure services for sharing distributed resources
Origins in high performance computing community
P2P
Collaboration and computing at the edges of the net
Focused on client-side of network computing
42
Internet Distributed Computing (IDC)
Global Computing (GC)
Goal: Accelerate distributed computing on the Internet, to increase business and consumer end-user value
Scope: Provide a foundation for a variety of computings from a small-scale Personal Area Network (PAN) to virtual, planetary grids.
Intersection of Web Services, P2P, and Grid computing.
The Internet as a distributed computing platform unified by common aspects of Web platform unified by common aspects of Web Services, P2P, and Grid computing.
43
Converged Middleware: The Building Blocks
44
Computing on Cyber Space
Cyber/Web/Internet Computing, Service Computing (SC)- Cross/intersection of WbS, Semantic Web, P2P, Grid, SaaS, Cloud, etc.
Cloud
P2P Grid
Cyber SpaceInternet/Web
SC - Service Computing
EaaS/Cloud
WbS
SmW
45
Novel Computing Trends
Service Computing
EaaS/XaaS: SaaS, HaaS/IaaS, PaaS, CaaS
Cloud Computing
46
Cloud vs. Grid
Large-Scale Resource Sharing
CloudMassive-Scale Service Sharing
Grids
All W
ays to
Clo
ud
47
Cloud Computing System and Services
StorePhotoVideo Blog
ProcessStore
Send
Reorganize the IT Commercial Mode1. Sell Service instead of Software2. Charge Users instead of Advertisement
48
Cloud Services
Infrastructure as a Service
(IaaS)
CPU, Storage: Amazon.com,
Nirvanix, GoGrid….
Platform as a Service (PaaS)
Google App Engine, Microsoft
Azure, Manjrasoft Aneka..
Software as a Service (SaaS)
SalesForce.Com Infrastructure as a Service (IaaS)
Software as a Service (SaaS)
Platform as a Service (PaaS)
49
Some Commercial-Oriented Cloud
platforms/technologies System
PropertyAmazon
EC2 & S3
App Engine
Microsoft
Azure
Manjrasoft
Aneka
Focus IaaS IaaS/PaaS IaaS/PaaS PaaS
Service TypeCompute (EC2),
Storage (S3)Web apps
Web and non-web
appsCompute/Data
Virtualisation OS Level: Xen Apps container OS level/Hyper-VResource Manager
and Scheduler
Dynamic
Negotiation of
QoS
None None None
SLA-oriented/
Resource
Reservation
User Access
Interface
EC2 Command-line
Tools
Web-based
Administration
Console
Windows Azure
portalWorkbench, Tools
Web APIs Yes Yes Yes Yes
Value-added
Service Providers Yes No Yes No
Programming
Framework
Amazon Machine
Image (AMI)Python .NET framework
Multiple App models
in.NET languages
50
Market-oriented Cloud Architecture: QoS
negotiation and SLA-based Resource Allocation
Dispatcher
VM
Monitor
Service Request
Monitor
Pricing Accounting
Service Request Examiner and
Admission Control
- Customer-driven Service Management
- Computational Risk Management
- Autonomic Resource Management
Users/
Brokers
SLA
Resource
Allocator
Virtual
Machines
(VMs)
Physical
Machines