collaborative environment and workflow decomposition for remote instrumentation

Collaborative environment and workflow

decomposition for remote instrumentation

Roberto PuglieseELETTRA - Sincrotrone Trieste SCpA

On Behalf of the GRIDCC Collaboration

BoF - Remote Instrumentation Services in Grid Environment (RISGE)OGF20 - Manchester 7-9 May 2007

[email protected] BoF RISGE - OGF20

Outline The GRIDCC project

project introduction pilot applications

GRIDCC technologies IE VCR Workflows

Final considerations


The GRIDCC Project The GRIDCC - Grid Enabled Remote Instrumentation with

Distributed Control and Computation (www.gridcc.org) It is a 3-years EU FP6 project started in September 2004 Define and implement the “Instrument Element” to allow a

standard remote access to any type of sensors and instruments Tight integration between instruments and the eInfrastructure Complex workflows integrating instruments, computational and

storage resources Human Computer interaction via Virtual Control Room groupware

tools that support distributed and heterogeneous teams of people in the collaborative access to the extended eInfrastructure

http://www.gridcc.org/




The GRIDCC partnersParticipant name Country

Istituto Nazionale di Fisica Nucleare Italy

Institute Of Accelerating Systems and Applications Greece

Brunel University UK

Consorzio Interuniversitario per Telecomunicazioni Italy

Sincrotrone Trieste S.C.P.A Italy

IBM (Haifa Research Lab) Israel

Imperial College of Science, Technology & Medicine UK

Istituto di Metodologie per l’Analisi ambientale – Consiglio Nazionale delle Ricerche

Italy

Universita degli Studi di Udine Italy

Greek Research and Technology Network S.A. Greece


GRIDCC Architecture

Web ServiceInterface

Execution Services

WfMS

WMS

AS

Instrument elements

(IE)Instrument elements

(IE)Instrument

Element(IE)

Compute element

(CE)Compute element

(CE)

Computing Element

(CE)

StorageElement

(SE)

StorageElement

(SE)

StorageElement

(SE)

ProblemSolver

Information and Monitoring

Services(IMS)

Information System(BDII)

Security Services

Virtual Control

Room

Virtual Control

Room

CollaborativeServices

(CS)

WMS Workload Management System

WfMS WorkFlow Management System

AS Agreement Service

[email protected] BoF RISGE - OGF20 Execution ServicesCollaborative Environment

GRIDCC ArchitectureInstruments Grid Computational Grid

IE

IE

CE

CE

SE

Instrument Element

Problem Solver

InstrumentManager Information &

Monitoring System

DATA

ES

VCR

VCR

VCR

VCR

Workflow


GRIDCC main target areas(Large-scale) scientific

experimentsHigh energy particle physics

(Radio-) Telescopes

Remote process control Accelerator control (Tele-) Biomedicine

Robotics Automotive

Electronic microscopes

Widely Sparse Instrumentation Power Grids

Monitoring of the territory Monitoring of the seaGeo-hazard prediction

Distributed laboratories Transportation monitoring

Sensor network

GRIDCCMiddleware


Remote Operations of ELETTRA: a feedback via Grid

Remote Experts

Operator

I E

SE

CE

IMAccelerator

IMPositionMonitors

IMCorrectors

See DEMO at

the e

xibitio

n booth 2


Instrument Element Requirements Interactive access to allow control and monitoring Standard interface to the physical devices Fast publishing of the data acquired by instruments Fast information (logs, errors, etc.) publishing to

track the behaviour of instruments and possibly solve problems

Quality of Service and Advanced Reservation Computing Grid integration

Move data from instruments to the Grid storage resources (Storage Element) and viceversa

Process data produced by instruments using Grid computing resources (Computing Element)

I


The Instrument Element (IE)

Inst

rum

ent C

ontro

l

Grid

Acc

ess

Access to Instruments

Data/Info Pubblishing

InstrumentElement

Web

Ser

vice

SR

M/G

RID

FTP

Custom (JMS, ..... )

Custom + Plugins

I’m here

Discovery

QoSAdvance Reservation


SOA accessible operations to control and monitor the instruments (via VIGS) execute a command get / set attributes / parameters

Multi channel data production

Instrument Element: multichannel I/O

VIG

SCommands

Status

Data Subscribers

Grid Interaction

SR

M

VCR

StorageElement

(SE)

StorageElement

(SE)

StorageElement

(SE)

Logs, Errors,

States, Monitors

Instruments

IMS

ParametersES

VIGS Virtual Instrument Grid ServiceIMS Information and Monitor ServiceSRM Storage Resource Management

IE


VIG

S

IMSProxy

ControlManager

DataCollector

Real Instruments

Data Flow

Control Flow

State FlowError FlowMonitor Flow

The term Instrument Element describes a set of services that enable the remote control and monitoring of instruments

InputManager

EventProcessor

FSMEngine

ResourceProxy

Control Manager

IMSInstrument Element Architecture

ResourceService

IMS

ProblemSolver

InstrumentManager

Instrument Element

Data Publisher

Acc

ess

Con

trol M

anag

er

SRM/SEGridFTP S

EIM

SInstrument Protocols


Instrument Discovery The discovery of the instruments or of the IEs is an issue when the number

of elements is high We can have two use cases:

Quasi static case The number of IEs is well defined In this case a register based discovery mechanism can be used. GRIDCC tesbed is using the LCG BDII (Berkely Database

Information Index ) based on LDAP The information collected in the BDII follow a GLUE schema

Dynamic case The number of IEs can change very quickly, they are very simple

devices, often with poor hardware support The discovery is just use to know which are the online IEs A new approach has been developed based on Peer to Peer (P2P)

protocols

See DEMO at

the e

xibitio

n booth 2


Instrument Element: Scalable on embedded systems

FPGAPPC405

Xilink Virtex IVCustom Board

Cus

tom

Ele

ctro

nics

1 Gbps Ethernet

Web

Ser

vice

Grid

CC

IMJa

vaV

MLi

nux

USER INTERFACE

Custom Logic

Montavista www.montavista.com

JamVM http://jamvm.sourceforge.net

Standalone Axis

IEInstrument Manager


The Virtual Control Room (VCR) The VCR provides a collaborative environment where

users: Meet and collaborate by means of groupware tools (e.g.

chat, shared calendar, logbook, video conference tools) Search, discover and browse resources such as people,

instruments and other eInfrastructure resources (e.g. computing and storage)

Transparently operate with remote instruments using the GRIDCC middleware Infrastructure


Architectureof the VCR

User interface Layerproviding General and

Application Specific tools

Services Layerproviding low-level

functionalities needed by the VCR

Web-based portalproviding some basic

functionalities and support for aggregating components


VCR in action


Grid in the palm of your hand


WfMS Architecture


Remote Operations Workflows: OrbitFB


Final considerations The GRIDCC project is integrating instruments into the

“classic” computational/storage Grids. The Instrument Element allows:

virtualisation of the real instruments support of a variety of instruments from high end to

embedded devices The Virtual Control Room is a groupware tools that support

distributed and heterogeneous teams of people in the collaborative access to the extended eInfrastructure

The Complex workflows integrating instruments, computational and storage resources can be executed

Sustainability Production installations Support new user communities

collaborative environment and workflow decomposition for remote instrumentation

Documents

elementibof risge ogf20

distributed control

behaviour of instruments

grid computing resources

instruments tight integration

standard remote access

viceversaprocess data

collaborative access