jas – distributed data analysis grid enabled analysis workshop caltech - june 23-25, 2003
Post on 21-Dec-2015
220 views
TRANSCRIPT
Contents
JAS2Historyclient-server modeJAS2 and the Grid
JAS3What’s newJAS3 and AIDAPlans for Gridification
JAS Client-Server Mode
Java Compiler+
Debugger
GUI
Experiment Extensions
(Event Display)
User’sJava Code
Padded Cell
Data AnalysisEngine
DATA
Distributed Analysis System: Goals
Prototype for GRID enabled JAS analysisRun analysis on a farm of machines
Use multiple CPU’s in parallel for CPU-intensive analysisAccess multiple I/O channels for data-intensive analysis
Use standard JAS (Client) as if we are running a local JobGet interactive feedback
Create analysis modules (code)Control job executionView results (Plots/Histograms)
Access distributed datasets as if they were local datasets
Distributed Analysis System: Architecture
JASClient
JAS DataServer
JASClient
JASClient
…
Network
Users
CatalogServer
ControlServer
ControlServer
Network
JAS DataServer
JAS DataServer
…
JAS3 Overview
A completely new version of JASDesign based on Application Shell, into which many (optional)
modules can be pluggedHighly customizable for different application domains
– HEP/Astrophysics/Other– DST analysis/Online Monitoring/GRID analysis– Experiment/User specific modules
Modules can be updated independently of shell– Possible to release bug fixes fast
Includes support for programming in many languagesScripting: Python, Pnuts, Dynamic Java, ….
– Command promptJava (compiled)
Analysis (histograms, tuples, fitting) based on AIDA standard
Not technically backwards compatible with JAS2But migration is straightforward.
AIDA Overview
AIDA = Abstract Interfaces for Data AnalysisCovers key areas for data analysis
Histograms, Tuples, Fitting, Data Points, Plotting, Management
Developed collaboratively at series of workshops by groups at CERN, LAL, SLAC.Next workshop June 30-July 4 -- CERN
Interfaces developed for C++ and Java ( and maybe Python?)Several implementations/tools available
Anaphe/Lizard/LCG PI – CERNOpen Scientist – LALJAIDA/JAS/AIDAJNI – SLAC
JAS3 and AIDA
JAS3 has adopted AIDA for analysis AIDA allows us to leverage experience and skill of other
developersAIDA is functionally more complete than JAS2 analysis packageAIDA allows JAS to exchange data with other AIDA toolsAIDA provides bridge to C++ programs (e.g. Geant4)AIDA encourages creativity and innovation
JAS3 HEP Analysis tools based on JAIDAJAIDA = Java implementation of AIDA
JAIDA is part of FreeHEP libraryUsable as standalone library for any Java Application
AIDAJNI = Interface between C++ and Java AIDAAllows C++ programs to use JAIDA, JAS3
JAS3, AIDA and C++
C++ program
AID
A
C++ AIDA Implementation
AIDA- JNI
AID
AJAIDA
JAS3
.aida file(XML)
Java programA
IDA
JAS3 and AIDA
JAS3 supports all AIDA functionality, includingHistograms (includes arithmetic, projections, etc.)Clouds (unbinned histograms, scatterplots)PlotterTuplesFitting – AIDA interfaces allow for multiple fitters
Uncmin -- pure java minimizerMinuit -- Fortran called by Java Native Interface (JNI)
IOAIDA XML, PAW, Root
JAS3 supports user interaction with AIDA in three waysScripting (Pnuts, Python etc)Compiled (Java) codeGUI – Plotting, Fitting, Cuts etc.
JAS3 Scripting
JAS3 has multi-language OO scripting supportCommand line, Console, EditorMajor components (e.g. AIDA) have scripting interfaces
Currently have plugins to supportPnutsPnuts – syntax almost identical to Java, fast, well documented
and feature completePythonPython (using Jython)More scripting languages can be added
not restricted to Java implementations (e.g. could use C-Python, JPE)
JAS3 Lightning Tour
Tour designed to
give you an
overview of the
capabilities of JAS3,
you can try them
out for yourself this
afternoon.
Welcome Page, gives initial info and links to example
scripts and programs
Memory monitor
Graphical Interface to AIDA
Histograms, Clouds, Tuples all presented in
AIDA tree
.aida files, .hbook
files, .root files all presented
as AIDA objects
Drag items onto page, or use
(popup) menus
Printing
Can send individual
plots or full page
direct to printer
Or save as PS, EPS, PDF, SWF, SVG, PNG, GIF…
Or copy/paste into Word, PowerPoint
etc.
Java Editor, Compiler and Loader
Built-in Java compiler
Tree shows loaded
programsBuilt-in
editor for writing
analysis code
Unlike JAS2 which only supported “event analyzers” JAS3 allows any Java program
to be loaded. This example “main routine” is taken directly from the AIDA manual
Pnuts Language
Currently support Pnuts scripting languageComplete and well documented
http://javacenter.sun.co.jp/pnuts/doc/guide.html
Fast (although not as fast as compiled Java)Syntax very similar to JavaCan easily call compiled Java classes from scripts – best of
both worlds
Plan to support other languages in futureIn particular Python
Record Sources
Opening record (or event) based files
causes the run control toolbar
to appear
Works similarly to JAS2 Job control, but now also supports random access and “tagged” data sets
(mainly for event displays)
Histogram Profile
ScatterPlot
XY Data(More appropriate forsmaller data sets)
Tuple Explorer - Plots
Works with any tuple, read from file or
dynamically created
JAS3 Spreadsheet
Simple spreadsheet pluginfor
Displaying resultsCalculationsSimple Plots
Supports reading/writing.csv filesExcel files
Cut/Paste with Excel etcComing Soon…
Scripting interfaceGUI for building plotsUser defined functions
– Java, scripting
Status
Currently released JAS3 version 0.7.1AIDA functionality is quite solidCompiler, Loader, Record Loop all quite recently added,
Certainly still some rough edges
Documentation limited but availableBuilt-in example scripts and programsTutorial on web
If you are used to JAS2 you will find some functionality
not yet ported to JAS3Remote (client/server) access to data.3D Lego/Surface plots
JAS3 and the GRID
We plan to add client-server/distributed capabilities to
JAS3 similar to those in JAS2Will be based on (distributed) AIDA
Next AIDA workshop (at CERN next week) will discuss this
Want to use Grid standards where they existWork with others (PPDG-CS11,???) to define standards where they
do not exist
Want to be compatible with C++ serversTech-X have submitted phase II SBIR and if approved will work
closely if approved
JAS3 Links, More Info
JAS – Java Analysis Studio - http://jas.freehep.org
JAS3 – http://jas.freehep.org/jas3
JAIDA – http://java.freehep.org/jaida/
AIDA – http://aida.freehep.org
FreeHEP - http://www.freehep.org
FreeHEP Java Libraries - http://java.freehep.org
WIRED – http://wired.freehep.org