msf and mage: e-science middleware for bt applications
DESCRIPTION
MSF and MAGE: e-Science Middleware for BT Applications. Sep 21, 2006 Jaeyoung Choi Soongsil University, Seoul Korea [email protected]. Application SW. Human Genome to Chemicals (HG2C). Portal (Web). Service Component Framework. Service Component Framework (MAGE). - PowerPoint PPT PresentationTRANSCRIPT
MSF and MAGE: e-Science Middleware
for BT Applications
Sep 21, 2006
Jaeyoung ChoiSoongsil University, Seoul Korea
2 Sep 21, 2006
HG2C System Architecture
Infrastructure (Cluster, Network, …)
Globus Toolkit (GT3, GT4)
Portal (Web)
Application Research
Grid ServicesGlobal scheduler, Monitoring, Adaptive API, etc.
Visualization
Steering
Workflow
Portal toolkit
Application SW
Service Component Framework
Human Genome to Chemicals (HG2C)
Infrastructure
Low level : ResourceManagement
Mid level :
Fundamental Services
ApplicationSW
VisualizationHigh level:
Steering Workflow
PSE toolkit
MSF 2.0
StreamingGlobal
schedulerMonitoringAdaptive
APIText
SearchData Mining
High level :Application Support
Service Component Framework (MAGE)
3 Sep 21, 2006
Meta Services
Define a part of a workflow as a new service Workflow instance can be declared as a workflow unit
in the service description By overriding some attributes of a workflow unit,
Pass parameters of a service to the workflow’s attributes Setup service specific information
The new service can be wrapped to a Web service or a Grid service,
therefore it can be easily reused
Manage service specific information Restrict resources to allocate a specific service
(user’s preference and/or organization’s policy) Scheduling jobs with priority
4 Sep 21, 2006
Meta Services
5 Sep 21, 2006
Meta Services Framework (1)
Meta Services Framework MSF is a workflow system for Bio Grid portal Users can compose a DAG-based workflow using legacy applications such as a BLAST Schedule User’s workflows on Grid environments Provide reusable workflows using Meta Services Users can compose services, flows, and tasks using XML Can be easily installed and configured
6 Sep 21, 2006
Meta Services Framework (2)
MSF Workflow model Workflow divided into three layers - service layer, flow layer, and task layer Increase reusability of workflow Service layer is specially designed using Meta services
concept
MSF Middleware Architecture Consist of five modules (SM, RM, EM, AM, OM) Three agent modules process each layer of the workflow
model (SM - service, RM - flow, EM - task)
AM manages authentication and access control OM maintains XML description
7 Sep 21, 2006
Five Agents in the Architecture
SM (Service Manager) Manage meta services and converts a meta service to a workflow
RM (Resource Manager) Collect resource information, and allocating resources
EM (Execution Manager) Launch & monitor (workflow) jobs, detect faults,
and collect results and performance data
AM (Access Manager) User authentication, environment setup, and a job submission
service
OM (Ontology Manager) Manage ontology of service, flow, and task
8 Sep 21, 2006
Middleware Architecture
9 Sep 21, 2006
Operations defined in MSF
10
Sep 21, 2006
MAGE
MAGE FeaturesProvide API for easy development of Grid applicationProvide transparency to end-users and developers
Protocol transparencyRunning location transparency
Provide layered architecture for easy replacementMobility for each agent
MAGEModular & Adaptive Grid Environments
11
Sep 21, 2006
MAGE architecture
Interpreter Layer
12
Sep 21, 2006
Communication Layer
To provide easy replacement of the communication protocol without affecting other layers
Administrator can select an appropriate protocol component
before running an applicationTasks and message interpretation does not affected by changing of communication protocol
13
Sep 21, 2006
Interpreter Layer
Interpret received messages & deliver to the appropriate task agents
Two basic query components implementedMonitor Query: use SQL’s SELECT-like statementTable Query: use name=value pair table
14
Sep 21, 2006
Task Agent Management Layer
Control the life cycle of task agents
Provide functions for installation from remoteBasic functions for mobile agentsAdvanced functions for suspending a job & resuming it at another node
15
Sep 21, 2006
MAGE & MSF
Interpreter Layer
16
Sep 21, 2006
Advantages of MAGE & MSF
MSF contains essential functionalities for BT Grid portals workflow, service interface, job distribution,
and parameter scheduling
Implementing MSF modules as MAGE agents increases reconfigurability and adaptability
From MSF modules to MSF agents MAGE environments provides various services to MSF modules Resource information, System environments, ..
17
Sep 21, 2006
HG2C Portal
18
Sep 21, 2006
Meta Services Framework Demo (1)
19
Sep 21, 2006
Meta Services Framework Demo (2)
20
Sep 21, 2006
HG2C Pubmap
21
Sep 21, 2006
HG2C Homepage http://www.hg2c.org
Open sources of MSF & MAGE http:// www.sourceforge.net/projects/mage4ubi http:// www.sourceforge.net/projects/msf
HG2C References
22
Sep 21, 2006
Q & A