jas3 – current status and prospects

JAS3 – Current Status and Prospects

by Victor Serbo, SLAC

ACAT03 1-5 December 2003 JAS3 – Current Status and Prospects, V. Serbo

Contents

Introduction to JAS3 (Java Analysis Studio)Basic: Opening Files, Plots, Printing, etc.Web Browser, Editor, Java Compiler, ScriptingMiscellaneous: Spreadsheet, Configuration Save/Restore,

Preferences, etc.

AnalysisAIDA – Abstract Interfaces for Data AnalysisJAIDA – Java Implementation of AIDAAIDA JNI – using JAIDA from C++Analysis with JAIDA and JAS3

JAS3 Plugins What is Plugin, examplesDemo: AmbientPlugin and Geant4 PluginDemo: JAIDA Web Interface


JAS3 Overview

A completely new version of JASDesign based on Application Shell, into which many (optional)

modules can be pluggedHighly customizable for different application domains

– HEP/Astrophysics/Other– Data analysis/Online Monitoring/GRID analysis– Experiment/User specific modules

Modules can be updated independently of shell– Possible to release bug fixes fast

Includes support for programming in many languagesScripting: Pnuts, Python, ….

– Command promptJava (compiled)

Analysis (histograms, tuples, fitting) based on AIDA standard

Not technically backwards compatible with JAS2But migration is straightforward.


JAS3 Lightning Tour

Tour designed to

give you an

overview of the

capabilities of JAS3,

you can try them

out for yourself:

http://jas.freehep.org/jas3

Welcome Page, gives initial info and links to example

scripts and programs

Memory monitor


Opening Files

Use file menu

Drag from explorer


Plot Manipulation and Printing

Can send individual

plots or full page

direct to printer

Or save as PS, EPS, PDF, SWF, SVG, PNG, GIF…

Or copy/paste into Word, PowerPoint

etc.


Java Editor, Compiler and Loader

Built-in Java compiler

Tree shows loaded

programsBuilt-in

editor for writing

analysis code

JAS3 allows any Java program to be loaded. This example “main routine” is taken directly from the AIDA manual


Miscellaneous Features

Save/Restore configuration

User Preferences

Plugin Manager


JAS3 Spreadsheet

Simple spreadsheet pluginfor

Displaying resultsCalculationsSimple Plots

Supports reading/writing.csv filesExcel files

Cut/Paste with Excel etcComing Soon…

Scripting interfaceGUI for building plotsUser defined functions

– Java, scripting


JAS3 Scripting

JAS3 has multi-language OO scripting supportCommand line, Console, EditorMajor components (e.g. AIDA) have scripting interfaces

Currently have plugins to supportPnutsPnuts – syntax almost identical to Java, fast, well documented

and feature completePythonPython

Use Jython (the Java implementation of Python)Limitation: can not call Python libraries written in C

More scripting languages can be addednot restricted to Java implementations (e.g. could use C-Python,

JPE)


Pnuts Language

Currently support Pnuts scripting languageComplete and well documented

http://javacenter.sun.co.jp/pnuts/doc/guide.html

Fast (although not as fast as compiled Java)Syntax very similar to Java; can compile script into Java

bytecode or convert it to Java codeCan easily call compiled Java classes from scripts – best of

both worlds









JAS3 Scripting II

Can also write and run

scripts

Console allows direct interaction

with scripting language


AIDA - Abstract Interfaces for Data Analysis

AIDA is a standard set of interfaces for creating and manipulating

histograms, n-tuples and related data analysis objects, like Plotter,

Fitter, etc.

AIDA defines storage XML format.

It has been created cooperatively by a group of developers working

on high-energy physics data analysis tools. The goal of the AIDA

project is to provide the user with a powerful set of interfaces which

can be used regardless of which analysis tool they are using.

Only define “protocol” for analysis objects, no internal detailsAIDA defines behavior, analysis tool provides implementationC++ and Java versions of interfaces are generated from the AID

files.


AIDA Examples

IHistogram1D- IAxis axis()

- void fill(double x)

- void fill(double x, double w)

- double mean()

- double rms()

- double binEntries(int index)

- double binError(int index)

- ….

For more details on AIDA,

see: http://aida.freehep.org

IAxis- int bins()

- double lowerEdge()

- double upperEdge()

- double binLowerEdge(int index)

- double binUpperEdge(int index)

- double binWidth(int index)

- int coordinateToIndex(double x)

- ….

http://aida.freehep.org/






JAIDA

JAIDA = Java implementation of AIDAComplete implementation of AIDA interfaces

With a few exceptions– Support for plot styles is somewhat limited– Some missing features in ITree, like cp, mv

Uses Java “service provider mechanism” for dynamic loading of services at RuntimeCan be used to add support for new fitters, stores etc.Have defined “developer level” interfaces for IOptimizer, IStore etc

Uses Java Expression Library (JEL) for runtime compilation of expressions (for tuple evaluators and functions)


JAIDA – Features continued… Fitting

support for multiple fitting engines (optimizers) UNCMIN (pure Java) Minuit (needs minuitAdapter native library)

Support for multiple fitting methods LeastSquares, Chi2, BinnedMaximumLikelihood and UnbinnedMaximumLikelihood

Stores Read/Write support for AIDA XML format (with built-in gzip) Read support for HBOOK/PAW files (needs hbookAdapter native library) Read support for Root files (write support hopefully coming soon) Future plans to add:

SQL interface, binary store format, text file support …

Plotting Two implementations of IAnalysisFactory

BatchAnalysisFactory – dummy plotter, ignore calls to IPlotter AnalysisFactory – Full plotter functionality

Multi-Threaded – plots update in real-time as they are filled No need to call interact() or refresh()

Support for embedding plotter In other Java GUI’s In Java servlets for web based applications.

Graphics export in many formats including: PDF, EPS, SVG, SWF, PNG, GIF, JPG


AIDA-JNI

C++ interface uses AIDA-JNI

C++ program

AID

A AIDA- JNI

AID

A JAIDA (Java)

AIDA-JNI partially generated by AID, partly by handDoes support full AIDA functionality


Using JAIDA with Java


Using JAIDA with Python


Using JAIDA with C++


Future Plans for JAIDA

Complete implementation of missing featuresContinue to track improvements to AIDA standard

Add support for function arithmetic

Web Interface

Support for “Remote AIDA”

Support for more storage formatsNeed good binary IO format .. either AIDA standard or roll our own

Future AIDA developers’ meetings


JAS3 and AIDA

JAS3 uses JAIDA - supports all AIDA functionality, includingHistograms (includes arithmetic, projections, etc.)Clouds (unbinned histograms, scatterplots)PlotterTuplesFitting – AIDA interfaces allow for multiple fitters

Uncmin -- pure java minimizerMinuit -- Fortran called by Java Native Interface (JNI)

IOAIDA XML, PAW, Root

JAS3 supports user interaction with AIDA in three waysScripting (Pnuts, Python etc)Compiled (Java) codeGUI – Plotting, Fitting, Cuts etc.


Graphical Interface to AIDA

Histograms, Clouds, Tuples all presented in

AIDA tree

.aida files, .hbook

files, .root files all presented

as AIDA objects

Drag items onto page, or use

(popup) menus


Histogram Profile

ScatterPlot

XY Data(More appropriate forsmaller data sets)

Tuple Explorer - Plots

Works with any tuple, read from file or

dynamically created


Tuple Explorer – Define Columns


Tuple Explorer - Cuts


Tuple Explorer - Tabulate


To be used with record loop

Tuple Explorer – Record Source


Record Sources

Opening record (or event) based files

causes the run control toolbar

to appear

Supports random access and “tagged” data sets

(mainly for event displays)


JAS3 Plugins JAS3 is based on the Freehep Application Framework, which provides a

basic application framework into which extension modules, or "plugins" can be loaded.

The application framework provides general purpose functionality, such as top level GUI (windows, menus etc.), help system, storing and retrieving user preferences etc.

All of the data analysis specific functionality of JAS3 is provided by a set of extension modules.

Modules communicate with each other via service interfaces and notification events. JAS3 comes with a standard set of service interfaces and notification events, but new plugins can also define their own interfaces and events.


JAS3 Plugin II Plugin functionality:

Do some plugin-specific taskUse services provided by other pluginsProvide services to other pluginsModify JAS3 menus and pop-up menus

“Core” Plugins – provide basic servicesSimple: File Opener, Memory Status, Status Bar, …More Complex: Web Browser, Plotter, Console, XMLIO, …

“Extensions” – separate packagesIncluded with JAS3 distribution

AIDA, TupleExplorer, JAVA Compiler and Editor, Spreadsheet, …Specific to particular Experiment

BaBar Ambient DB Browser: AmbientPlugin, Geant4 Plugin, etc.

Partial Tutorial on writing a plugin: http://jas.freehep.org/jas3/developers/tutorial.jsp


JAS3

(Java)

JAS3

(Java)

JAS3 Plugin Demo I

AmbientPlugin: Access BaBar Ambient DB via CORBAPresent Time History as an AIDA object (DataPointSet)

BaBar

Ambient DB

BaBar

Ambient DB

CORBA Server (C++)

CORBA Server (C++)

AmbientPlugin

(Java CORBA Client)

AmbientPlugin

(Java CORBA Client)


JAS3

(Java)

JAS3

(Java)

JAS3 Plugin Demo II

Gean4 Plugin Gean4 Plugin

Geant4 Application (C++)

Geant4 Application (C++)

JNI LayerJNI Layer Java RMI Server

Java RMI Server

Geant4 Plugin

(Java RMI Client)

Geant4 Plugin

(Java RMI Client)

AIDA Web Interface


Goals

Provide access to Histogram plots without requiring any

software installationPutting results on to the web for collaborators etc.

For example for viewing results of “quality control” plots

Viewing “live” histograms on the web forOnline monitoringMonitoring progress of batch jobs

Add as much functionality as possible in a “generic” way

so that it can be used by others (with minimal work)Complete “out-of-the-box” web application

Packaged as a .war file for use with any J2EE app server – (e.g. Tomcat)

Configurable via a simple .ini file (or better via a web interface)


Architecture

.root

.paw

.aida

Application Server (Tomcat)

Web Application

JAIDAAIDA

Servlet

SQL

…

RemoteAIDA

C++ App

Java App


Features

Current Feature setDisplay of “AIDA Tree” for selection of histogramsDisplays plots as .png or .gif images in browser

Allows (limited) control of image propertiesAllows (limited) control over plot propertiesAllows comparison to reference plots (including Kolmagorov test)Allows export of plots to PDF, EPS, SVG, SWF, GIF, PNG, etc.

Planned featuresExtend to support n-tuples as well as histograms

Dynamic definition of new columns and cuts (evaluators, filters)Web based fitting (including function definition)Support multiple plots per page

Thumbnail browserExtend which plot options can be configured through webAllow users to retain settings via cookies


Current Interface

Can try from the web:http://glast02.slac.stanford.edu:8080/aida/welcome.html


Status

Currently released JAS3 version 0.7.3 (0.7.4 next week)AIDA functionality is quite solidCompiler, Loader, Record Loop all quite recently added,

Certainly still some rough edges

Documentation limited but availableBuilt-in example scripts and programsTutorial on web

If you used JAS2 you will find some functionality not yet

ported to JAS3Remote (client/server) access to data.3D Lego/Surface plots


JAS3 Conclusion

JAS3 is a powerful and flexible Analysis Tool

JAS3 uses Plugin ArchitectureEasy to add new functionalityEasy to fix problems with existing modules – do not have to wait

for new release of all libraries

JAS3 release 0.7.3 is out – try it!It is still a beta version, so bugs happenWe need your input to make JAS3 better!

Report bugs, problems, suggestions at http://bugs.freehep.orgDo not have to be on JAS team to be JAS developer – ideas/code for

general-purpose plugins are always welcome

Try writing JAS3 plugins for your own needs: for Experiment and Home

http://bugs.freehep.org/




JAS3 Links, More Info

Report Issues – http://bugs.freehep.org/

JAS – Java Analysis Studio - http://jas.freehep.org

JAS3 – http://jas.freehep.org/jas3

JAIDA – http://java.freehep.org/jaida/

AIDA – http://aida.freehep.org

FreeHEP - http://www.freehep.org

FreeHEP Java Libraries - http://java.freehep.org

WIRED – http://wired.freehep.org




http://jas.freehep.org/










http://java.freehep.org/jaida/










http://www.freehep.org/



http://java.freehep.org/



http://wired.freehep.org/



jas3 – current status and prospects

Documents

java editor

java compilertree

java codecan

java bytecode

java program

java implementations

jas3 java analysis studiobasic

compiled java classes