data size and io performance report
DESCRIPTION
Data size and IO performance report. ATLAS Software & Computing Workshop. Data size MC http://athena-infoioperformance.web.cern.ch/athena-infoIOperformance/MCsizes.html Streams http://athena-infoioperformance.web.cern.ch/athena-infoIOperformance/StreamsSizes.html - PowerPoint PPT PresentationTRANSCRIPT
Ilija Vukotic [email protected]
Data size and Data size and IO performance reportIO performance report
ATLAS Software & Computing Workshop
• Data size
– MC http://athena-infoioperformance.web.cern.ch/athena-infoIOperformance/MCsizes.html
– Streams http://athena-infoioperformance.web.cern.ch/athena-infoIOperformance/StreamsSizes.html
– Categories http://athena-infoioperformance.web.cern.ch/athena-infoIOperformance/T0streams.html
• IO performance
• Possibilities for improvement
20/04/23 Ilija Vukotic 2
MC event size
20/04/23 Ilija Vukotic 3
MC event size
20/04/23 Ilija Vukotic 4
Note: trigger size is not realistic as used trigger menu was not realistic. Problem rectified with addition of new aliases:Physics_default -> Physics_pp_v1MC_loose_default -> MC_pp_v1_loose_mc_prescaleMC_tight_default -> MC_pp_v1_tight_Mc_prescale (see bug report #74712).
Size AOD Size ESD
RecoTrf 572 +27.17%* 1880 +33.09%
stdSim 381 +11.18% 1620 +24.03%
fullSim 381 +11.18% 1620 +24.03%
physSim 320 +11.96% 1520 +24.79%
*Compared to 22 Aug. 2010
Streams sizes
20/04/23 Ilija Vukotic 5
Only runs with 5k+ eventsand good LBs
Shows:•AODs,ESDs and all DESDs•Events/total size/ev. size
Streams sizes
20/04/23 Ilija Vukotic 6
Stream Size[GB]
ESD 1023834
AOD 92319
DESDM_EGAMMA 55288
DESD_SGLEL 44191
DESD_SGLMU 39375
DESDM_TRACK 26548
DESD_MBIAS 16841
DESD_CALJET 12370
DESDM_MET 5108
DESD_PHOJET 4281
DESD_SGLMUDESD_SGLEL
DESDM_EGAMMAAODESD
20/04/23 Ilija Vukotic 7
Categories
Categories
20/04/23 Ilija Vukotic 8
EgammaL1CaloMuonsJetTauETMiss
https://spreadsheets.google.com/ccc?key=0AiPvgbRljNCodG5IY1BseHRHa2R5WjZoMGRnLXp0dHc&hl=en
Details and list of collections in categories can be found at:
20/04/23 Ilija Vukotic 9
IO performance
Slow
Bad CF
KEEP IN MINDIn real data TrackCollections are even larger !
20/04/23 Ilija Vukotic 10
IO performance
• Do we really need all of these?
• No way to say if object is ever used.
• What is so terrible about having a name attached to each object we store?
Possibilities for improvement
• TrackCollection/TrackParticle container
– Maybe 50kb in size and factor 2 in time
– Few MB in memory
• CaloShowerContainer
– Probably can go down in size
• Calo(Topo)TowerContainer
– Gets recreated at read time
– Must be possible to optimize further
• TRT_Drift_Circle_Container
– Must be faster and smaller
20/04/23 Ilija Vukotic 11
My estimate:
In total between 50 and 200kB/ev can be shaved in converters
Should be possible >100ms/ev (currently at 480 ms/ev)
Other options:
Object removal.
Cuts tuning