icetray a software framework for icecube

Tyce DeYoungUniversity of Maryland

September 27, 2004

IceTray A Software Framework for

IceCube

Computing in High Energy PhysicsInterlaken, Switzerland

September 27, 2004CHEP 2004

The IceCube Collaboration

University of AlabamaUniversity of California, BerkeleyClark – Atlanta UniversityUniversity of DelawareInstitute for Advanced Study,

PrincetonUniversity of KansasLawrence Berkeley National

LaboratoryUniversity of MarylandPennsylvania State UniversitySouthern University and A&MUniversity of Wisconsin, MadisonUniversity of Wisconsin, River Falls

DESY, ZeuthenJohannes Gutenberg-Universität MainzBUGH Wuppertal

Vrije Universiteit BrusselUniversité Libre de BruxellesUniversité de Mons – Hainaut

Stockholms UniversitetUppsala Universitet

Imperial College, LondonUniversity of Oxford

Chiba University

University of Canterbury, Christchurch

Universiteit Utrecht

Universidad Simón Bolívar, Caracas

Amundsen-Scott Station, South Pole


Atmospheric Muons & Neutrinos

• “Atmospheric muons” from cosmic ray showers, penetrating to the detector from above

• “Atmospheric neutrinos” from the same air showers, forming a diffuse background and calibration beam

• Astrophysical neutrinos: interesting signal

Atm.

Atm.

Astrophysical

cosmic ray

cosmic ray

• A neutrino interacts with a nucleus in the ice

• A relativistic lepton is produced in the collision

• Optical sensors map the lepton’s Cherenkov light

• Timing, photon counting give direction & energy

Cherenkovlight cone

Muon

Opticalsensor


The IceCube Detector

1 km3 instrumented volume: 1 Gton of ice

4800 digital optical modules (DOMs) in the ice on 80 strings

AMANDA will be enclosed within the array

IceTop air shower array on the surface above the detector (80 stations, 320 DOMs)

Geometry optimized for TeV—EeV neutrinos

IceTop

SouthPole

AMANDAIceCube

1400 m

2400 m


Muon Tracks


Electron and Tau Events

Electron: All energy deposited in a short cascade – Spherical diffusion

Tau: “Double Bang”– N interaction– lepton decay– Separated at very

high energies


Software Framework

• Want advantages of a modern framework– Modularity (better maintainability!)

– Easy to reuse code, extend applications

– Allows user/developers to focus on specific task

– Reliability (core functions in common, well-exercised code)

– Common style, shorter learning curve

• What framework? – Some specific requirements similar to

colliders, some different– Detector Operations begin in February, 2005!


Reconstruction Requirements

• Very sparse event information– Five sensors per megaton of detector



• Reconstruction requires complicated techniques

– Algorithms evolving rapidly, development distributed throughout collaboration

– Software must be deployed at remote location, limited intervention possible

– Desktop-to-online conversion must be easy, very robust



Many hypotheses, hierarchical filteringVariable event flow, determined at run time


Monte Carlo Requirements

• Events occur at random times– No ‘heartbeat’ analogous to beam crossing– Event duration ~5 µs, rate 1700 Hz

→ Frequently overlapping



• Need to overlay events from different generators, preprocessing chains→ Multiple input streams, irregular heartbeat



• Difficult Monte Carlo problem

– Can’t track every photon from a kilometer-long PeV muon track through a km3 active volume

– Run special-purpose photon tracker before Monte Carlo production, use tabulated results

– Full photon tables hard to fit in memory (would use ~200 GB tables if possible)

→ Need to hold events in a buffer, sort for more efficient batchwise access


How to Drive Execution?

• Standard “push” (declarative) execution model not well suited– Individual modules need some control over the

flow of the overall process– But want to avoid coupling independent code or

requiring complex signalling mechanisms

• “Pull” (on demand) model also problematic– Places responsibility on framework, modules to

match requests, resources: developers, not users!

– Enumeration of types of information that can be passed: less flexibility for developers

• Either can work, but want something simple


The IceTray Approach

• Pushmipullyu Rube Goldberg Queue-based– Each module has one or more input, output

queues (‘Inboxes’ and ‘Outboxes’)– Module receives events in an Inbox, processes

them and places them in an Outbox – One Inbox is ‘active’ → drives execution– Events can be requested from the ‘passive’

Inboxes– User sets event flow topologically at configure

time, arbitrary† complexity based on module logic– Order of execution determined by data flow

†Required to be a Directed Acyclic Graph


Inboxes and Outboxes

• Context of inboxes, outboxes is created by user at configure time, not by developer


Module Lifecycle

• Module is a relatively flexible state machine

• Default implementations are provided transparently in an inherited base class


Analysis Containers

• Each IceTray module isolated from larger environment, in own “container”

• Interactions limited to requests for services, events, configuration, logging and errors

• All services, etc. provided by Framework through a context object – switchable

• Module appears to have own flow of control

• Lends itself to threads, distributed processing


Division of Responsibilities

• Framework handles only low-level tasks– Logging, configuration, event flow,

execution flow, connection of services, etc.– Does not need to understand modules,

data

• Modules focus exclusively on their task– No need to understand their role in the

larger scheme of things

• User supplies context at configure time


IceTray Status

• Basic implementation complete (linear functionality)

• In use within IceCube for simulation, online reconstruction/data monitoring (starting this winter), offline analysis

• Implementation of more advanced features in progress

• Code available by request– Uses IceCube distribution, build system


Old and New South Pole Stations

http://glacier.lbl.gov/offline/ ICETRAY-APP/index.html

icetray a software framework for icecube

Documents

monte carlo requirementsevents

monte carlo requirementsneed

air showers

collision optical sensors

muon trackschep

amuniversity of wisconsin

madisonuniversity of

aboveatmospheric neutrinos