update of computing & software models
DESCRIPTION
Ian Bird Overview Board; CERN, 8 th March 2013. Update of Computing & Software Models. Background. Requested by the LHCC in December: need to see updated computing models before Run 2 starts A single document to: Describe changes since the original TDRs (2005) in - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/1.jpg)
Update of Computing & Software Models
Ian Bird
Overview Board; CERN, 8th March 2013
March 6, 2013
![Page 2: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/2.jpg)
Background• Requested by the LHCC in December: need to see
updated computing models before Run 2 starts• A single document to:
- Describe changes since the original TDRs (2005) in• Assumptions, models, technology, etc.
- Emphasise what is being done to adapt to new technologies, to improve efficiency, to be able to adapt to new architectures, etc.
- Describe work that still needs to be done- Use common formats, tables, assumptions, etc
• 1 document rather than 5
March 6, 2013
![Page 3: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/3.jpg)
Timescales• Document should describe the period from LS1 –
LS2- Estimates of evolving resource needs
• In order to prepare for 2015, a good draft needs to be available in time for the Autumn 2013 RRB, so needs to be discussed at the LHCC in September: Solid draft by end of summer 2013 (!)
• Work has started- Informed by all of the existing work from the last 2
years (Technical Evolution groups, Concurrency forum, Technology review of 2012)
March 6, 2013
![Page 4: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/4.jpg)
Opportunities• This document gives a framework to:
- Describe significant changes and improvements already made- Stress commonalities between experiments – and drive
strongly in that direction• Significant willingness to do this• Describe the models in a common way – calling out differences
- Make a statement about the needs of WLCG in the next 5 years (technical, infrastructure, resources)
- Potentially review the organisational structure of the collaboration
- Review the implementation: scale, quality of service of sites/Tiers; archiving vs processing vs analysis activities
- Raise concerns:• E.g. staffing issues; missing skills;
March 6, 2013
![Page 5: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/5.jpg)
Draft ToC• Preamble/introduction • Experiment computing models• Technology review and outlook• Challenges – the problem being addressed• Distributed computing• Computing services• Software activities and strategies• Resource needs and expected evolution• Collaboration organisation and management
March 6, 2013
![Page 6: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/6.jpg)
Experiment computing models• Data models – types of data, event sizes,
relationships, etc• Anticipated event rates and data streams• Data flows• Differences for Pb-Pb or p-Pb• Non-event data
March 6, 2013
![Page 7: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/7.jpg)
Technology review• Use (update) report from 2012• What are likely technologies in next 5 years?
- CPU: e.g. Intel vs ARM vs GPU vs ??- Storage- Clouds/virtualisation- Likely evolution of networks
• Etc.
March 6, 2013
![Page 8: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/8.jpg)
Challenges• What problem are we addressing?
- Need to make the best use possible of available resources
- Major investment in software needed • But missing skills, people, tools, infrastructure?
- Need for flexibility in adapting the models to changing technologies
- …
March 6, 2013
![Page 9: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/9.jpg)
Distributed computing• Use Cases
- Calibration, reconstruction, re-processing, stripping, analysis use cases & strategies for prompt vs delayed analysis, simulation
• Functions implemented at Tier 0,1,2, HLT- Include how we would use opportunistic resources- Review functions: perhaps distinguish between
archive needs and data distribution needs, QoS
• Networking- What will our needs be? - What topologies to interconnect tiers?
March 6, 2013
![Page 10: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/10.jpg)
Computing services• Emphasise commonalities between experiments (and
justify differences)- Workflow management – use of pilots, implementations, needs
at sites- Data management – strategies for use of tapes, disks, other
storage; services such as FTS, data popularity, data federation• Use of HEP vs standard protocols
- Distributed computing services (aka “grid mw”)• Describe the services required, central vs distributed deployments;
where do experiment needs diverge?• Federated identity management?• Infrastructure services: operations, monitoring, accounting, security,
etc• Other new technologies
March 6, 2013
![Page 11: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/11.jpg)
Application Software• Current and recent activities – gains already achieved• Strategies for the future• How to address the use of new architectures etc• Common tools and libraries (ROOT, GEANT, etc)• Possibility for common frameworks between
experiments, prerequisite for:• Setting up consultancy/optimisation team (inc. testing
infrastructure) to guide efforts in optimisation (memory use, I/O, parallel code, etc)
• What is the coordination and management needed for this?
March 6, 2013
![Page 12: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/12.jpg)
Resource Needs & Evolution• Assumptions on running conditions• Event parameters• Summary tables of resource requirements
- Tape, disk, cpu, bandwidths- 2015-2018/19
March 6, 2013
![Page 13: Update of Computing & Software Models](https://reader035.vdocuments.us/reader035/viewer/2022062408/56813dcf550346895da79773/html5/thumbnails/13.jpg)
Collaboration Organisation & Management
• Opportunity to review the WLCG organisational structure and associated bodies
• Stress the need for building collaborative activities – in order to find the effort for needed developments, operations, etc.
• Describe the anticipated interactions with e-infrastructures (EGI/Horizon 2020, OSG, etc)
• Interaction with other HEP experiments – i.e. should the scope of WLCG broaden to support HEP more widely?
March 6, 2013