na61/na49 virtualisation: status and plans dag toppe larsen budapest 14.05.2012
DESCRIPTION
NA61/NA49 meeting, Budapest3 CERNVM CERNVM is a Linux-distribution Designed specifically for virtual machines (VMs) Based on SLC (currently SLC5) Compressed image size ~300MB Both 32-bit and 64-bit versions Addition software “Standard” software via Conary package manager Experiment software via CVMFS Contextualisation: images adapted to experiment requirements during boot Data preservation: all images are permanently preservedTRANSCRIPT
NA61/NA49 virtualisation:status and plans
Dag Toppe LarsenBudapest 14.05.2012
14.05.2012 NA61/NA49 meeting, Budapest 2
Outline Quick reminder of CERNVM Tasks
Each task in detail Roadmap CernVM installation Input needed
14.05.2012 NA61/NA49 meeting, Budapest 3
CERNVM CERNVM is a Linux-distribution
Designed specifically for virtual machines (VMs) Based on SLC (currently SLC5) Compressed image size ~300MB Both 32-bit and 64-bit versions
Addition software “Standard” software via Conary package manager Experiment software via CVMFS
Contextualisation: images adapted to experiment requirements during boot
Data preservation: all images are permanently preserved
14.05.2012 NA61/NA49 meeting, Budapest 4
CVMFS Distributed read-only file system for CERNVM (i.e. the
same as AFS for LXPLUS) Can also be used by “real” machines (e.g. LXPLUS, grid) Files compressed and distributed via HTTP
Global availability Central server, site replication via standard HTTP proxies Files decompressed and cached on (CERNVM) computer
Can run without Internet access if all needed files are cached Mainly for experimental software, but also other “static”
data (e.g. calibration data) Each experiment has a repository to store all versions of
software Common software (e.g. ROOT) available from SFT
repository
14.05.2012 NA61/NA49 meeting, Budapest 5
Data preservation As technology evolves, no longer possible to
run legacy software on modern platforms Must be preserved and accessible:
Experiment data Experiment software Operating environment (operating system, libraries,
compilers, hardware) Just preserving data and software is not
enough Virtualisation may preserve operating environment
14.05.2012 NA61/NA49 meeting, Budapest 6
CERNVM data preservation “Solution”:
Experiment data stored on Castor Experiment software versions stored on CVMFS
HTTP “lasting” technology Operation environments stored as CERNVM image
versions Thus, a legacy version of CERNVM can be
started as a VM, running a legacy version of experiment software
Forward-looking approach (we start preserving now)
14.05.2012 NA61/NA49 meeting, Budapest 7
Tasks Make experiment software available Facilitate batch processing Validate outputs On-demand virtual clusters Production reconstruction Reference cloud cluster Data bookkeeping web interface
14.05.2012 NA61/NA49 meeting, Budapest 8
Make experiment software available NA61/NA49 software must be available on CVMFS
for CernVM to process data NA61
Legacy software chain installed Changes to be fed back to SVN
SHINE Experts are installing production version on CVMFS CernVM environment has to be made Automatic install of necessary packages via Conary SVN checkout should compile “out of the box” May be better to use 32-bit CernVM image
NA49 Software has been installed (validation needed)
14.05.2012 NA61/NA49 meeting, Budapest 9
Facilitate batch processing LXPLUS uses PBS batch system CernVM uses Condor batch system “Philosophical” differences
PBS has one job script per job Condor has common job description file with
parameters for each job Existing PBS scripts have been ported to
Condor
14.05.2012 NA61/NA49 meeting, Budapest 10
Output validation – status Run 8688 has been processed on both
CernVM/CVMFS and LXPLUS/AFS, using software version v2r7g According to analysis by Grzegorz, there are
relatively large discrepancies (larger than SLC4->5) Surprising, since software is the same, and CernVM
IS Scientific Linux 5 (just repacked) Can there be issues with calibration files? Or some
of the changes done to make software work on CVMFS?
14.05.2012 NA61/NA49 meeting, Budapest 11
Output validation – plan Have requested new reconstruction on
LXPLUS/CVMFS Will make it possible to separate effect of
CernVM/LXPLUS from CVMFS/AFS (three-way comparison)
Shine is production-ready now Should we “forget” legacy chain, and focus on Shine? On the other hand: NA49 reconstruction may have
same discrepancy as NA61 legacy chain; good reason to still investigate source
14.05.2012 NA61/NA49 meeting, Budapest 12
On-demand virtual clusters A cluster may need VMs of different
configurations, depending on type of jobs Memory, CernVM version, experiment SW, etc.
Thus, need for dynamic creation/destruction of virtual cluster
Created command-line script for creating virtual clusters Later to be controlled by data bookkeeping web
interface
14.05.2012 NA61/NA49 meeting, Budapest 13
Production reconstruction Production reconstruction next step Cluster of “decent” size needed
Need to submit ~50 VMs to process a large data set To run on LXCLOUD (experimental CERN service)
Awaiting conclusion from software validation step
14.05.2012 NA61/NA49 meeting, Budapest 14
Reference cloud cluster The virtual machines require a cluster of
physical hosts A reference cloud cluster has been created
Detailed documentation will simplify the process of replicating it at other sites
Based on OpenNebula (popular cloud framework) KVM hypervisor Provides Amazon EC2 interface (de facto standard
for cloud management)
14.05.2012 NA61/NA49 meeting, Budapest 15
Data bookkeeping web interface A web interface for bookkeeping of the data to be
created List all existing data with status (e.g. software versions
used for processing) Easy selection of data for (re)processing with selected
OS and software version A virtual on-demand cluster is created After processing, data written back to Castor
Either based on existing frameworks, or on new development
Using EC2 interface for the cloud management Allows for great flexibility of processing site
14.05.2012 NA61/NA49 meeting, Budapest 16
Bookkeeping outlook Most important/urgent task Founding ends end of October All dependencies (software, cloud, dynamic
clusters) in place Only bookkeeping system itself missing Optimistic about outlook for completion by end
of October But should be wary of sources (validation?) of
delays...
14.05.2012 NA61/NA49 meeting, Budapest 17
RoadmapTask Status/done Remaining ExpectedNA61 software installation
Legacy framework Shine framework End of May
NA49 software installation
Software installed Data validation To be determined
Facilitate batch system
OK OK OK
Validate outputs Created reference data sets for validation
Understand source of discrepancies
End of May
On-demand virtual cluster
OK OK OK
Production reconstruction
Set-up ready Awaiting conclusion of validation discrepancies
End of May
Reference cloud cluster
Cluster working Documentation End of July
Data bookkeeping web interface
Initial planing Evaluate frameworks“First” version“Final” version
End of MayEnd of JulyEnd of October
14.05.2012 NA61/NA49 meeting, Budapest 18
Next steps Parallel task 1
Understand validation discrepancies
Run large-scale processing on CernVM
Parallel task 2(critical path) Data bookkeeping
web interface for CernVM processing
Run large-scale processing using CernVM/web interface
Transfer to NA61
14.05.2012 NA61/NA49 meeting, Budapest 19
CernVM for development CernVM makes it possible to run production
version of legacy software/shine on laptop without local install
Also possible to compile Shine from SVN on CernVM “out of the box” when the proper NA61 environment is set up
Is also possible to mount NA61 software from CVMFS directly on laptop (but software dependencies may have to be resolved by user)
14.05.2012 NA61/NA49 meeting, Budapest 20
CernVM installation on laptop Install a hypervisor of your choice, e.g. Virtualbox:
https://www.virtualbox.org/ Download a matching CernVM desktop image:
http://cernvm.cern.ch/portal/downloads Open http://<ipaddress>:8004 in your web browser (user=admin,
password=password) Select NA61 and PH-SFT software repositories Reboot You are now ready to use NA61 software in CernVM on your
laptop! More information: http://cernvm.cern.ch/portal/cvmconfiguration
14.05.2012 NA61/NA49 meeting, Budapest 21
My schedule I will be around for assistance with installing
CernVM on laptops However, I plan to leave Wednesday morning,
since I have a CHEP-poster to print/make sure is brought to CHEP (I will not go to CHEP, only my poster will)
This is “negotiable”, e.g. if there is big demand for CernVM install, Shine installation work or important discussions
14.05.2012 NA61/NA49 meeting, Budapest 22
Input needed NA49 validation SHINE installation NA61 legacy validation discrepancies How to practically arrange for large scale
reconstruction Issues related to data bookkeeping Please keep virtualisation (CERNVM/CVMFS)
in mind when making plans ...