python/c fase prototype l. paioro, b. garilli et al. opticon network 9.2 mima collaboration pandora...
TRANSCRIPT
Python/C FASE prototype
L. Paioro, B. Garilli et al.
OPTICON Network 9.2MiMa CollaborationPANDORA @ INAF-IASF Milano
L. Paioro - Python/C FASE prototype - Marseille, June 2010
Architectural Concepts
Applications Top level GUI/Web/CLI applications exploiting external
computational components Computational components
Task stateless component single operation
driven by a parameter set Tool
stateful component multiple operations
each operation driven by a parameter set Packages
Bundles of components
L. Paioro - Python/C FASE prototype - Marseille, June 2010
Example Of Use 1
GUI Application
FASECompute
wavelength calibration
mypipeline package
lamp_calib task
mypipeline.lamp_calib
Application API
Comp-Cont API
L. Paioro - Python/C FASE prototype - Marseille, June 2010
Example Of Use 2
Compute wavelength calibration
FASE
mypipeline package
lamp_calib task
Comp-Cont API
mypipeline.lamp_calib
Application API
Command Line Application
L. Paioro - Python/C FASE prototype - Marseille, June 2010
How Applications/Components Communicate Within FASE
INLINE PROFILE Direct in-process communication
Python applications with Python components Python applications with ANSI-C components through SWIG
bindings
SAMP PROFILE IVOA SAMP messaging communication
From any application to any component Embedded SAMP Hub and Package-Manager daemon
L. Paioro - Python/C FASE prototype - Marseille, June 2010
Component-Container Model
Components are wrapped by containers implementing a standard set of interfaces which connects them to any framework implementation compliant with the CCM standards
Component-containers are packaged together in a standard format which enables them to be deployed within any framework implementation compliant with the packaging standards
L. Paioro - Python/C FASE prototype - Marseille, June 2010
The Application Programming Interfaces
System API Shared by applications and computational
components to perform generic access to the framework resources and services
Component-Container API Allows to implement a task or tool container and
make it executable through the framework Application API
Provides proxies to the computational components in order to access and execute them through the framework
L. Paioro - Python/C FASE prototype - Marseille, June 2010
System API (UML)
Connect with the framework instance and return references to the main services
Load the packages, query their capabilities and create new requests, responses and related psets
Service for synchronous and asynchronous invocations
Service for events notification and receiving
L. Paioro - Python/C FASE prototype - Marseille, June 2010
Component-Container API (UML)
Introspection capabilities
Single atomic operation
Task (IS one operation) and Tool (IS COMPOSED BY several operations)
Single atomic operation I/O:-Parameter set-Job ID-Errors
L. Paioro - Python/C FASE prototype - Marseille, June 2010
Component-Container API (UML)
Collection of parameters
Single parameter
L. Paioro - Python/C FASE prototype - Marseille, June 2010
Application API (UML)
Proxy to the package after loading, allows to get new tasks and tools instances
ToolProxy allows to access tool operations and execute them (synch/asynch)
TaskProxy to launch a task execution (synch/async)
L. Paioro - Python/C FASE prototype - Marseille, June 2010
INLINE Component Container Example: Python
Python Task Component
L. Paioro - Python/C FASE prototype - Marseille, June 2010
INLINE Component Container Example: Python
Python Task INLINE Container
Python Task Component
L. Paioro - Python/C FASE prototype - Marseille, June 2010
INLINE Component Container Example: Python
L. Paioro - Python/C FASE prototype - Marseille, June 2010
OR
MAGICAL DECORATION
INLINE Component Container Example: ANSI-C
ANSI-C Task Component
L. Paioro - Python/C FASE prototype - Marseille, June 2010
INLINE Component Container Example: ANSI-C
ANSI-C Task Component
ANSI-C Task INLINE Container
L. Paioro - Python/C FASE prototype - Marseille, June 2010
The Package Description File
Given a package description file, component-containers skeletons can be generated by appropriate tools for any language!
L. Paioro - Python/C FASE prototype - Marseille, June 2010
Python Application Example: synchronous and asynchronous
L. Paioro - Python/C FASE prototype - Marseille, June 2010
Issues To Be Faced
Completion of ANSI-C interfaces A better FASE errors handling (suitable for multi-
threading cases) Tools handling Application API
Batch execution (OpenPBS) Parallel execution
MPI (multi-process) OpenMP (multi-threaded) OpenCL/CUDA (GPU parallel)
L. Paioro - Python/C FASE prototype - Marseille, June 2010