easygrid: the job submission system that works! james cunha werner gridpp18 meeting – university...
TRANSCRIPT
EasyGrid: the job submission system that works!
James Cunha Werner
GridPP18 Meeting – University of Glasgow
User software
User computer
Datasets
Grid enabled software
User software + Gridification algorithms
Workload management
Data Management
Performance analysis
Grid resources
Gridification Process: from conventional to grid computing.
Data Gridification Functional Gridification
EasyGrid Job Submission system
•Submit jobs
•Manage datasets
•Recover results
•Recover reports
> BetaMiniApp Tau11-Run3.tcl
> Easygrid BetaMiniApp Tau11-Run3
See http://www.hep.man.ac.uk/u/jamwer/ for more information
Many files in sequence
Many files in parallel -> Fast
File name
Dataset name
Retrospective• Available since GridPP11 - September/2004:
http://www.gridpp.ac.uk/gridpp11/babar_main.ppt
• Several benchmarks with BaBar experiment data:Data Gridification:– Particle identification: http://www.hep.man.ac.uk/u/jamwer/index.html#06 – Neutral pion decays: http://www.hep.man.ac.uk/u/jamwer/index.html#07 – Search for anti deuteron: http://www.hep.man.ac.uk/u/jamwer/index.html#08 Functional gridification:– Evolutionary neutral pion discriminate function:
http://www.hep.man.ac.uk/u/jamwer/index.html#13
• Documentation (main web page):http://www.hep.man.ac.uk/u/jamwer/ 109 html files and 327 complementary files
• 60 CPUs production and 10 CPUs development farms running independently without any problem between November/2005 and September /2006.
EasyGrid in production: results with real data
Results data gridification
Source Dr Mitchell Naisbit
Date: Thu, 22 Dec 2005 15:51:08 +0000
From: Roger Barlow <[email protected]>To: [email protected]: [BABAR-USERS] Manchester babar
Dear Manchester BaBarians,
2 bits of good news.
1) easyroot works. I have carefully idiot-proofed it, and if I can make it work then anyone can. Today it gives access to a small farm, meaning you can run several jobs in parallel and speed up your tauuser analysis by an order ofmagnitude. Soon we will enable the rest of the existing BaBar farm. And beforelong we have the 1000 node Dell farm.
For brief instructions see http://www.hep.man.ac.uk/u/roger/easyroot.htmlFor full instructions see http://www.hep.man.ac.uk/u/jamwer/rootsrc.html
2) we have a new big disk, thanks to Sabah. 1.6 TB. We need to decide what toput on it (and what to call it.)
Father Christmas has been busy...
Roger
Tau to ηπν
Source Dr Marta Tavera
Functional Gridification:
NPDF Final results for several energy cuts-α: Sensitivity or efficiency.
-β: specificity or purity.
-γ: accuracy.
Hadronic tau decays results:
Dissemination
• 20 international events:
http://www.hep.man.ac.uk/u/jamwer/index.html#10
• 5 refereed papers Int. Conferences.
• GridPP stand at IoP2006 and IoP2007.
• Contributions at GridPP web pages.
http://www.gridpp.ac.uk/posters/
• Users support at France, Switzerland, New Zealand, etc: LCG and BaBar software, grid and cluster computer, etc.
Research beyond HEP
• Effect of complexity in NP hard optimization modeled with
genetic algorithms always feasible chromosome.
•Visual C++ framework for EasyGrid results visualization