1 ae@sg media grid initiative by a/prof. bu-sung lee, francis nanyang technological university
TRANSCRIPT
1
AE@SG Media Grid Initiative
By
A/Prof. Bu-Sung Lee, Francis
Nanyang Technological University
2
Content
BackgroundArchitectureApplications
Maya(with plug-in)PovRayMental Ray
Resource Matching
3
Objectives
Illustration of an ICT application encompassing diverse components in applications, portal & middleware
Application has potential to benefit digital media industry
Complements promotion of Grid Computing in digital media industry
4
Background
Project duration: Nov’2004 till Nov 2006
Initiative of Infocomm Development Authority(iDA) Singapore Hewlett Packard(HP)
Participants School of Information Systems, Singapore
Management University School of Computer Engineering, Nanyang
Technological University Institute of High Performance, A*STAR
5
Background(2)
6
Portal
Media Grid Architecture
iHPC Meta-scheduler
ESCSubmit job
SOAP ESC
ESC
Platform LSF Meta-scheduler
NGOLSF Grid
Submit job
WWW Browser
Maya
submit
iMac
ssh
http
ssh
animator
Toolkits- Data collection- Job monitoring- Results preview- Fault-redundancy (Missing job resubmission)- Input data preprocessing/uncompressing
Applications:- Pixie- Povray- Modeler- MentalRay
Resource Matchmaker
Mental Ray/others
ESC
7
Use scenarioData
Media Portal
Charging model
QM
MAYA/3DS,etc.
Infoservice
Jobservice
othersvcs
Virtualization/Middleware
Layer
data
data
data
data
data
8
Work Packages
WP1 - Point of access for users. Provide real-time collaborative environment & Media Workbench
WP2 - Gridification of animation & image rendering applications
WP3 - Generic middleware for job execution & management
WP3 - Expanding the scope of HP – Quartermaster tool from UDC to Grid for resource virtualization & allocation
WP4 - Globus standards. No development undertaken WP5 - Aims to model usage patterns using NGO/GOG
data & work on market based resource allocations models/biz models
9
Multiple Interfaces for Job Submission
10
Step 1 – Create Model in Maya
11
Step 2 – Job Submission from Maya to the Grid
Each frame is associated with one Grid job.
12
Step 3 – Output Data Collection from Portal
The package contains all resultant TIF files.
13
Speedup Sample
The time to render a RIB file is about 40 seconds.
351 RIB files need about 4 hours for rendering in a single computer.
Clusters hpc-pdpm has 20 CPUs in 10 machines. Cluster melon has 20 CPUs in 10 machines.
By using LSF to submit the jobs to hpc-pdpm and melon, now rending 351 RIB files only need about 14 minutes including the time for data stage in/out. The speedup is about 17.
14
Job Submission from Portal to LSF
The uploaded package includes all rib files, textures and shaders.
15
Check Status for the Most Recently Submitted Job
Progress
16
Portlet of Checking Status for the Rendering Jobs Submitted from Maya and from Portal
17
2. POV-Ray Interactive Rendering
GoalsUse Grid to render complex photo-realistic
objects when a single computer is not sufficient.
Deliver the application through the rich media portal directly to the users, without installing additional software
Perform on-the fly visualization of the rendering process to give the user more control.
18
Architecture overview
Web interface
GRID
Cluster A
Nodes
Cluster B
Nodes
Portal
Proxies
Rendering is performed in parallel for every web user
19
POV-Ray Interactive Rendering
20
POV-Ray Interactive Rendering
21
POV-Ray Interactive Rendering
Rendering in progress
22
POV-Ray Interactive Rendering
23
3. Interactive Shape Modeling App.
Goals Use Grid to work with 3D shapes when a single
computer computational power is not sufficient. Deliver the application through the rich media portal
directly to the users, without installing additional software
Methods Java3D web-based interface to receive user
commands and visualize the model Real-time communication link between computing
nodes and the interface
24
Architecture overview
Web interface
GRID
Cluster A
Nodes
Cluster B
Nodes
PortalWeb clients
Proxies
Rendering is performed in parallel for every web user
25
Task distribution
1. Object is split into several parts
2. The parts are distributed among the processing nodes
3. The results are sent back and combined
4. Users are able to visualize the modelIn their browsers interactively
26
Interactive Modeling App.
27
Interactive Modeling App
28
Interactive Modeling App
29
4. Mental Ray
30
Mental Ray Arch.
Melon020Mentalray client
Melon001Mentalray server
Melon002Mentalray server
Melon003Mentalray server
Melon004Mentalray server
31
Web-Service Enabled Resource Management
Understanding User Requirements
Formal Representation of Grid Resources
page 3August 31, 2003 Grid definition
32
Web-Service Enabled Resource Management
Discovery and Matching of Grid Resources
•Problems Countered in Integration •Information Service
33
Conclusion
Maya Application plug-in and portal submission
AchievementsMental Ray submission through portal.
Integration from Application to
Information management.Integration with LSF
34
The challenge
Thanks!