victor serbo, chep071 jaida, jas3, wired4 and the aida tag library - experience and new developments...
TRANSCRIPT
Victor Serbo, CHEP07 1
JAIDA, JAS3, WIRED4 and the AIDA tag library - experience
and new developments
DONSZELMANN, Mark (SLAC) JOHNSON, Tony (SLAC) SERBO, Victor (SLAC) TURRI, Max (SLAC)
Victor Serbo, CHEP07 2
FreeHEP Java Library Items discussed in this talk are all part of FreeHEP Java Library
Open-Source utilities for HEAP and beyond Mostly Java (now also open-source) http://java.freehep.org/
FreeHEP philosophy Avoid common HEP pitfall of lock-in to monolithic frameworks
Small modular projects which can be used together or separately Define interfaces to allow open communication between modules
Easy to extend or replace components Use open-standards where possible
Example: AIDA - Abstract Interfaces for Data Analysis Data can be in different formats and representations Adapt data to “look like” standard AIDA object and then use set of
standard interfaces to work with it Tools built using AIDA can work with analysis objects from Root,
PAW, Fits, SQL transparently. New in FreeHEP organization
Re-factor dependences and move code from CVS to SVN Use Maven 2 to build/test/deploy
see poster by Mark Donszelmann: #52 in Poster Session 1
Victor Serbo, CHEP07 3
AIDA AIDA is a standard set of interfaces for creating and
manipulating histograms, n-tuples and related data analysis objects, like Plotter, Fitter, etc.
Documentation and examples: http://aida.freehep.org Current release 3.3.0
There are several AIDA implementations: Java: http://java.freehep.org/jaida C++
Open Scientist: http://www.lal.in2p3.fr/OpenScientist PI: http://cern.ch/pi
Python: http://paida.sourceforge.net Also AIDAJNI layer that allows using Java
implementation from C++: http://java.freehep.org/aidajni
Victor Serbo, CHEP07 4
JAIDA – Java Imlementation of AIDA
Features: Data analysis and fitting (JMinuit and FMinuit) High quality plots in variety of formats (pdf, ps, gif,
png, swf, etc.) Read/Write support for AIDA XML format (with zip and
gzip options) Read support for Hbook, Root and plain Text files.
Download and Setup: Current version 3.3.0-5, implements AIDA 3.3.0 Documentation and setup instructions at http://
java.freehep.org/jaida Needs Java 1.4 or higher
Victor Serbo, CHEP07 5
New in JAIDA
Full implementation of the AIDA 3.3 interfaces Add Zip option for AIDA XML files
Provides directories that map to AIDA ITree structure Now can read and create AIDA XML objects on-demand
Opening XML files with many objects much faster Add Binary XML option for AIDA XML files
Faster reading – don’t have to convert numbers from String to double
Saves space Will support incremental storage of large objects (e.g. n-tuples)
New module for reading and writing Fits files Add support for plotting 1D and 2D histograms with variable bins Add utility class to fill IHistogram, ICloud, or ITuple with data
distributed according to a given IFunction Add utility class to convert individual AIDA objects to XML and
back Support for native i386-MacOSX (Hbook, FMinuit)
Victor Serbo, CHEP07 6
Using JAIDA
GEANT4 (C++)
GEANT4 (C++)
AIDAJNI Layer
AIDAJNI Layer
J
A
I
D
A
J
A
I
D
A
Oracle, MySQL, ...
Oracle, MySQL, ...
Java Application
Java Application
AIDATLD
(Tag Library)
AIDATLD
(Tag Library)
JAS3 GUI JAS3 GUI
XMLXML
HbookHbook
RootRoot
FitsFits
Victor Serbo, CHEP07 7
JAS3
Pulls together many FreeHEP modules, event display (WIRED4), etc. into single GUI application
Design based on Application Shell, into which many (optional) modules can be plugged Highly customizable for different application domains
HEP/Astrophysics/Other Data analysis/Online Monitoring/GRID analysis Experiment/User specific modules
Modules can be updated independently of the shell Possible to release bug fixes fast
Includes support for programming in several languages Scripting: currently Pnuts and Python Java (compiled)
Analysis (histograms, tuples, fitting, etc.) based on JAIDA JAS3 web page: http://jas.freehep.org/jas3 Current 0.8.4rc1
ftp://ftp.slac.stanford.edu/software/jas/JAS3/v0.8.4rc1
Victor Serbo, CHEP07 8
JAS3 Example
Can write and run scripts, Java code
Console allows direct interaction
with scripting language
Victor Serbo, CHEP07 9
JAS3 in BaBar
Combine on one HTML page Live monitoring plots Static reference plots Instructions and useful information
Fast Monitoring Live plots received from the shared memory via CORBA Reference plots from Hbook and Root files Featured on YouTube (during the last minute of
http://www.youtube.com/watch?v=dj7gCZTEoq0 ) Ambient or “Slow Monitoring” (time histories of
voltages, currents, temperatures, etc.) Data received from the DB or Live Archivers via CORBA
Victor Serbo, CHEP07 12
Data on the webAIDA Tag Library
Using the AIDA Tag Library (AIDATLD) you can easily insert live plots into the web pages (using Java Server Pages technology).
Users don’t need any special software – just a web browser Can be used to make data/plots accessible to distributed
collaboration quickly HTML-like tags: <aida:tree>, <aida:plotter>, <aida:style> The AIDA Tag Library is designed to work in a container
supporting JSP 2.0 or greater (such as e.g.Tomcat 5.5.*) Examples/Demo and Documentation:
http://aidatld.freehep.org
Web Application Server (e.g. Apache Tomcat)
AIDATLD
Web Browser
Data(Root,
SQL etc)
Plots sent as images (png, swf, pdf etc.)
Victor Serbo, CHEP07 13
AIDATLD Demo – plot from ROOT file
<%@page contentType="text/html"%><%@page pageEncoding="UTF-8"%>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<%@taglib prefix="aida" uri="http://aida.freehep.org/jsp20" %>
<%@page isELIgnored="false" %>
<head> <title>SimpleAIDA Plot</title></head><body> <h1>Plot Histogram from Root File</h1> <p/> <c:set var="treeName"
value="C:/work/Data/rootdemo.root"/> <c:set var="treePath" value="/h100"/>
<aida:tree storeName="${treeName}" storeType="root"/>
<aida:objects storeName="${treeName}" path="${treePath}" var="aidaPlotObject">
</aida:objects>
<aida:plotter> <aida:region> <aida:plot var="${aidaPlotObject[0]}"/> </aida:region> </aida:plotter> </body></html>
Victor Serbo, CHEP07 14
JAIDA/AIDATLD in GLAST The Gamma-ray Large Area Space Telescope
http://glast.gsfc.nasa.gov http://www-glast.slac.stanford.edu
Many web applications developed for GLAST make use of JAIDA/AIDATLD Software releases validation
http://glast-ground.slac.stanford.edu/SystemTests Telemetry trending Data Quality Monitoring Calibrations trending Pipeline processing overview
See also talk by Dan Flath (Id:361): “The GLAST Data Handling Pipeline”
Sky source and Gamma Ray Burst Monitoring
Victor Serbo, CHEP07 15
GLAST Data Origin
Data is stored in various formats Databases
Oracle, MySQL Files
Xml, Fits, Root Adapters are used to convert it to AIDA
objects These objects are then handled and
displayed using JAIDA and AIDATLD Users don’t care where data is coming from
and in what format they were stored. Can mix-and-match different data on one
web page
Victor Serbo, CHEP07 17
Telemetry Trending
~12K telemetries are stored in Oracle db Raw values on change Accumulated statistics every 5 minutes
This application allows users to View any telemetry over any time period Group telemetries together in a navigation tree
Stack corresponding plots Or produce a single overlay plot
Access raw/statistical data Choose the maximum number of data points per plot
If necessary data is accumulated on the fly Tabular view of the selected data
Victor Serbo, CHEP07 19
SQL Backend for AIDA
Next 3 slides provided by Julius Hrivnac [email protected]
SQLTuple SQL backend for AIDA ntuples Uses direct SQL commands to perform ntuple
operations (like creation of projections) => allows work with very big ntuples (unlike other SQL-AIDA couplings)
Exists also as a JAS plugin and a Web Service More information at
http://cern.ch/hrivnac/Activities/Packages/SQLTuple/
Victor Serbo, CHEP07 20
AIDATLD at Atlas
AIDATLD used together with SQLTuple to look at SQL databases deployed as J2EE Web Service at http://
cern.ch/SQLTuple Atlas Tag Database Characteristic
Histograms available in http://cern.ch/SQLTuple/presenter.jsp
or as a Google Gadget: http://www.google.com/ig/adde?moduleurl=http%3A//hrivnac.web.cern.ch/hrivnac/Activities/Packages/GoogleGadgets/TagsCharacteristicsHistograms.xml
Victor Serbo, CHEP07 22
Status and Plans Continue support and development of JAIDA and JAS3
Need new, more functional plotter Need ability to work with big data volumes
Support and develop web applications based on AIDATLD
AIDA meeting Summer 2007 to discuss improvements and future releases
WIRED4: experiment-independent interactive event display http://wired.freehep.org
User Support: Bug reports: http://bugs.freehep.org Forum: http://forum.freehep.org