research on key technology in grid delivery network zhihui lv network & information engineer...
Post on 22-Dec-2015
213 views
TRANSCRIPT
Research on Key Research on Key Technology in Grid Technology in Grid Delivery Network Delivery Network
Zhihui LvZhihui LvNetwork & Information Engineer CenNetwork & Information Engineer Center of Fudan university, ShangHai, Cter of Fudan university, ShangHai, C
hinahina 2003.8.282003.8.28
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
Clients
1.Content Delivery Network (CDN)
Host Server
MindSpring
PSINetSprint
Gloobix
QWest
@Home
UUnet
Huge Requests
Server Crash
Slow Response
Clients
Clients
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
1.Content Delivery Problems
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
1.(1)Use Client Cache/Client Side Cache Server
Host Server
MindSpring
PSINetSprint
Gloobix
@Home
UUnet
Fewer Requests
Clients
Clients
Clients
ClientCache
ClientSideCacheServer
QWest
Fast Response
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
1.(2)Use Mirror Sites
Host Server
MindSpring
PSINetSprint
Gloobix
QWest
@Home
UUnet
Fewer Requests
Server
Fast Response
Clients
Clients
Clients
Mirror Site
Mirror Site
Need improvement by guiding the selection of mirror servers with server load/network bandwidth measurement
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
1.(3)Edge Network Cache Servers
Host Server
MindSpring
PSINetSprint
Gloobix
QWest
@Home
UUnet
Fewer Requests
Server
Fast Response
Clients
ClientsClients
ClientCache
Mirror Site
Mirror SiteEdgeNetworkCacheServer
CacheServer
CacheServer
CacheServer
CacheServer
ClientSideCacheServer
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
1.Akamai CDN Solution --Edge Delivery Service
Date # of Edge Servers
# of Networks
# of Countries
11/2000 6000 335 54
6/2001 9700 650 56
5/2003 12900 1000 66
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
2.Drawback of CDN
Cache Location Problem: Where to put cache servers?How many are needed?When/where/how to push/delivery the content?How about dynamic content?
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
2.Drawback of CDN
Although CDN is an effective means of multimedia information access and delivery, there are several barriers to making CDN a more common service: Big cost Bad scalability Complicated replication integrity.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
3. Emergence of Grid Delivery
The combination of the grid technology and CDN technology has produced a novel technology——grid delivery. Grid delivery is an emerging technology that enables the distribution of rich media throughout a bandwidth-constrained network.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
3. Emergence of Grid Delivery
GGG ( Great Global Grid )
HP IBM WebSphereMicrosoft.NetSun ONE
Computing and Data Grid
TeraGridIPGGIGASCI GridData Grid
Information and Knowledge Grid
Semantic WebKnowledgeManagementOntologyInformationPlatform
Business Grid
CDN
RTEC Real-Time Enterprise Computing
Web Service
Other Grid Models
P2PParasitic Computing
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
3. Emergence of Grid Delivery
Grid delivery promises to be a viable solution for distributing on-demand streaming content and other large files directly to end-user systems. Grid delivery has its origins in the world of Internet-based content delivery. The objective of grid delivery is to create a network of systems for enabling content delivery to a single recipient.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
4. Infrastructure of Grid Delivery
Grid Delivery
Server
Grid Delivery Server
Network ManagerNetworkProtector
NetworkPublisher
Grid Delivery Technology
XML Web Services Connector
NetworkAnalyzer
(name might change)
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
4. Infrastructure of Grid Delivery Grid Delivery Technology creates a network of shared resources where any PC or server can deliver content on demand.
• Bandwidth Harvesting
• XML Delivery Security
• Dynamic Optimization
• Grid Delivery Servers
• Delivery Managers
Grid Deliver
y Server
Grid Delivery Server
NetworkAnalyzer
(name might change)
NetworkProtector
NetworkPublisher
Grid Delivery Technology
XML Web Services Connector
Network Manager
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.1Time Shifted Delivery5.1Time Shifted Delivery
Time Shifted DeliveryTime Shifted Delivery
0%0%
100%100%
0%0%
100%100%
Smoothing via Scheduling and Bandwidth Rate LimitingSmoothing via Scheduling and Bandwidth Rate Limiting
Time Shifted DeliveryTime Shifted Delivery
PeakPeak
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology
5.2 Adaptive Rate Multiserving (ARM)5.2 Adaptive Rate Multiserving (ARM)
Adaptive Rate Multiserving (ARM)Adaptive Rate Multiserving (ARM)
OriginServer
Typical Serving Model
User
Adaptive Rate Multiserving Model
OriginServer
1
OriginServer
2
OriginServer
3
User
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.3 Outer Edge Caching5.3 Outer Edge Caching
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.3 Outer Edge Caching 5.3 Outer Edge Caching
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.3 Outer Edge Caching5.3 Outer Edge Caching
LANInternetOrigin Server
Origin Server
USER1
USER2
USER3
USER1Grid
Server
Grid Server
GridServer
Grid Delivery --Outer Edge Relay CachingGrid Delivery-- Outer Edge On-Demand Caching
Grid Server
USER2
2(3) 2(4)
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security
A secure, managed delivery system must satisfy the following security requirements :A secure, managed delivery system must satisfy the following security requirements :
(1)Access Control - The delivery system must provide an access control mechanism that controls access to the content in the network and also provides controls over who is authorized to publish content to the network. This system must work with an enterprise's existing security infrastructure and be integrated with a central directory system to facilitate administration and maintenance of users and groups.
(2)Network Security - The delivery system must ensure that files are not delivered to unauthorized users, and that attackers cannot infiltrate the system itself.
(1)Access Control - The delivery system must provide an access control mechanism that controls access to the content in the network and also provides controls over who is authorized to publish content to the network. This system must work with an enterprise's existing security infrastructure and be integrated with a central directory system to facilitate administration and maintenance of users and groups.
(2)Network Security - The delivery system must ensure that files are not delivered to unauthorized users, and that attackers cannot infiltrate the system itself.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security
A secure, managed delivery system must satisfy the following security requirements :A secure, managed delivery system must satisfy the following security requirements :
(3)Desktop Security - The delivery system must provide a level of control over content use and redistribution that is appropriate to the nature of the content and the identity of the intended recipient.
(4)Enterprise-Ready Standards and Support - The delivery system must support all enterprise standards and be easily deployable within an enterprise. The system must also offer a high level of reliability and scalability to handle large numbers of users across geographic boundaries and network topologies.
(3)Desktop Security - The delivery system must provide a level of control over content use and redistribution that is appropriate to the nature of the content and the identity of the intended recipient.
(4)Enterprise-Ready Standards and Support - The delivery system must support all enterprise standards and be easily deployable within an enterprise. The system must also offer a high level of reliability and scalability to handle large numbers of users across geographic boundaries and network topologies.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security
XML Delivery Security InfrastructureXML Delivery Security Infrastructure
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security
The security measures in this infrastructure are as following:
( 1 ) Access Control:
User Authentication and Authorization;
Secure Publishing;
Content Access Rules;
Enterprise Security Infrastructure Integration;
Centralized Deployment and Administrator.
The security measures in this infrastructure are as following:
( 1 ) Access Control:
User Authentication and Authorization;
Secure Publishing;
Content Access Rules;
Enterprise Security Infrastructure Integration;
Centralized Deployment and Administrator.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security
The security measures in this infrastructure are as following:
( 2 ) Network Security:
Strong Encryption;
XML-based Secure Distributed Delivery Protocol;
Point-to-Point Security;
Unique Private Keys;
Secured Node;
Content Integrity.
The security measures in this infrastructure are as following:
( 2 ) Network Security:
Strong Encryption;
XML-based Secure Distributed Delivery Protocol;
Point-to-Point Security;
Unique Private Keys;
Secured Node;
Content Integrity.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
5.Grid Delivery Technology5.Grid Delivery Technology5.4 XML Delivery Security5.4 XML Delivery Security
The security measures in this infrastructure are as following:
( 3 ) Desktop Security and Usage Control:
Copy and Re-Distribution Prevention;
Encrypted File Controls; Point-to-Point Security;
The 3rd Party Digital Rights Management Support.
The security measures in this infrastructure are as following:
( 3 ) Desktop Security and Usage Control:
Copy and Re-Distribution Prevention;
Encrypted File Controls; Point-to-Point Security;
The 3rd Party Digital Rights Management Support.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
6.Delivery Grid dramatically increases performance while efficiently utilizing centralized bandwidth
7.30
0.47
0 2 4 6 8
KontikiGrid
CentralServer
Avg. Delivery Speed (Mbps)
Testing Notes:
-Central Server bandwidth speed is 0.47 Mbps or approx. 1/3 use of a dedicated T1
-Delivery speed calculated by ratio of file size to time of delivery for 20MB file
-Load % is ratio of average bytes delivered by Central Server to average bytes delivered by Grid Server.
15x times faster than central server
Central Server Load
Central Server 100%
Grid Server 30%
Central server bandwidth load reduced by 30%
Delivery Performance Use of Internet or WAN Bandwidth
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
6.Average Time to Deliver 20MB file Grid over 15x times faster than central server
- Central Server bandwidth speed is 0.47 Mbps or 1/3 of T1 speed (1.4Mbps)
21.91
342.86
0 100 200 300 400
Kontiki Grid
Central Server
Delivery Time (sec)
Typical Delivery
Grid Delivery
0 100 200 300 400
342
22
Avg.Delivery Time(sec)
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
7. The Challenges and Future Prospect
(1)Rich media grid delivery is a novel model for content delivery initially focused on the needs of the enterprise.
It is a flexible system that utilizes the network and computing resources already present in the enterprise much more efficiently, extremely important in this age of tight IT budgets and staffing.
But the challenge is how to apply Grid Delivery to Internet scale.
(1)Rich media grid delivery is a novel model for content delivery initially focused on the needs of the enterprise.
It is a flexible system that utilizes the network and computing resources already present in the enterprise much more efficiently, extremely important in this age of tight IT budgets and staffing.
But the challenge is how to apply Grid Delivery to Internet scale.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
7. The Challenges and Future Prospect
(2) As content becomes more digital, locating and managing the most atomic elements of that content for potentially millions of simultaneous users will be an exponentially difficult challenge to address.
Structured content such as XML-defined contexts combined with a strict adherence to the practices associated with many commercial content management systems will help alleviate some of this burden.
But the sheer volume and diversity of digital content types—text, image, audio, and video, each with their own variety of file formats—will demand a more flexible solution with massive computational capacity. This is one of the problems that Grid Delivery had to solve in the recent future.
(2) As content becomes more digital, locating and managing the most atomic elements of that content for potentially millions of simultaneous users will be an exponentially difficult challenge to address.
Structured content such as XML-defined contexts combined with a strict adherence to the practices associated with many commercial content management systems will help alleviate some of this burden.
But the sheer volume and diversity of digital content types—text, image, audio, and video, each with their own variety of file formats—will demand a more flexible solution with massive computational capacity. This is one of the problems that Grid Delivery had to solve in the recent future.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
7. The Challenges and Future Prospect
(3) Grid Delivery promises to be a viable solution for distributing on-demand streaming content and other large files directly to end-user systems.
Broadcasting live, concurrent, real-time streams using a grid delivery network is possible, but has not yet proven to be scalable. Therefore, researching and developing real-time grid streaming delivery technology is a great challenge.
(4) Grid Delivery technologies offer an advantage for storing and retrieving large datasets efficiently, but that it remains a question whether the Grid is beneficial for storing and managing large numbers of files that vary greatly in size.
(3) Grid Delivery promises to be a viable solution for distributing on-demand streaming content and other large files directly to end-user systems.
Broadcasting live, concurrent, real-time streams using a grid delivery network is possible, but has not yet proven to be scalable. Therefore, researching and developing real-time grid streaming delivery technology is a great challenge.
(4) Grid Delivery technologies offer an advantage for storing and retrieving large datasets efficiently, but that it remains a question whether the Grid is beneficial for storing and managing large numbers of files that vary greatly in size.
Niec, Fudan University, ShNiec, Fudan University, ShangHai, ChinaangHai, China
7. The Challenges and Future Prospect
(5) the next Globus Toolkit upgrade (3.0)—to be rolled out over the course of 2003—will incorporate Open Grid Services Architecture (OGSA), providing standardized discovery, management, and monitoring facilities for coordinating multiple Web services and provisioning their associated resources.
While Globus Toolkit 3 marks a milestone, there is still a long way to go before the ability to access distributed computation and resource is as straightforward as access to the Web.
(5) the next Globus Toolkit upgrade (3.0)—to be rolled out over the course of 2003—will incorporate Open Grid Services Architecture (OGSA), providing standardized discovery, management, and monitoring facilities for coordinating multiple Web services and provisioning their associated resources.
While Globus Toolkit 3 marks a milestone, there is still a long way to go before the ability to access distributed computation and resource is as straightforward as access to the Web.