post emi wlcg middleware lifecycle management
DESCRIPTION
Post EMI WLCG Middleware Lifecycle Management. s hort s ummary of the longer text: https:// indico.cern.ch / getFile.py / access?contribId =12&sessionId=1&resId=2&materialId= paper&confId =155073 Markus Schulz. Situation after EMI. Most middleware will be quite stable - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/1.jpg)
1
Post EMI WLCG Middleware Lifecycle Management
short summary of the longer text:https://indico.cern.ch/getFile.py/access?contribId=12&sessionId=1&resId=2&materialId=paper&confId=155073
Markus Schulz
![Page 2: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/2.jpg)
2
Situation after EMI• Most middleware will be quite stable
– no major functional changes expected– most software in EPEL (or EPEL “like” packaging)
• No (very little) funding for central roles • Products critical for WLCG supported by Product Teams located
at HEP institutes
• minimize central control• as close as possible to current approach• reuse existing structures• move responsibilities close to needs
– batch system support, tarballs, etc. • you need it, you do it
– or at least contribute to it
![Page 3: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/3.jpg)
3
Independent Product Teams
Info
VOMS
DPM
ARGUS
dCache
![Page 4: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/4.jpg)
4
Integration
• EPEL as integration point– Test and Stable Repository
• WLGC/UMD repository for non EPEL packages– Packages that can’t live in EPEL
• Commercial • Versioned Meta-Packages
• Tools– PTs’ choice during development– Fedora tools for release
• needed for EPEL
![Page 5: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/5.jpg)
5
Versioning
• Versioned Meta Packages – dependencies are not managed individually– version++ indicates changes
• Versions can be discovered and tracked – Resource info provider
• encoded by public key
– Local detailed logs by info provider for site admins• Improved WLCGBaselineVersions page – recommended and lowest accepted versions – time line for required upgrade
![Page 6: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/6.jpg)
6
Testing and Verification• PT independent tests– functional and regression tests – testing against production – site <-> PT coordination for resources • SEs, early pilots, test nodes,....
• Pilot services – coordinated by WLCG-Ops Team • needs PT-Site-Experiment coordination
![Page 7: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/7.jpg)
7
Staged Rollout• Staged Rollout– coordinated by EGI and WLCG-Ops Team• via web page with fields for sites and experiments
– this is the critical step– participating sites cover SEs and experiments• Active experiment sign off ✔
✔✔✔✔Test
![Page 8: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/8.jpg)
8
Repositories
• EPEL Stable• EPEL Test• UMD/WLCG non EPEL material– used also for rollback– highest priority and protected
• UMD/WLCG History– anything ever released
STABLE
TEST
![Page 9: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/9.jpg)
9
Rollback
• Staged Rollout catches most problems– but not all.......
• Using UMD/WLCG Repository and RPM tools– UMD/WLCG History
• This should happen only rarely • Coordination:
Operations Team
![Page 10: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/10.jpg)
10
Configuration Management
• Keep it simple• Offer what you use • Initially YAIM• Long term: – Generic documentation (must)– PTs’ preferences, like Puppet (may)– Sites should collaborate ( Chef, Quattor, Puppet..)
![Page 11: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/11.jpg)
11
Problem and Change Request Tracking
![Page 12: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/12.jpg)
12
Change Request Prioritization
WLCGGDB
WLCG-MB
2-3 Volunteers
nominates
Maintain ListPrepare new requests Remove duplicatesPrepare Pre-GDB 3 times a yearInvite PTs
List-1
Discuss relative prioritiesRemove Items
Info
VOMS
PTs concerned
Possible time linesArchitectural concerns
List-2
The prioritized list, including PT input, is discussed and endorsed.Progress of this list is tracked
DPM
![Page 13: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/13.jpg)
13
Expectation Management
• PTs are independent and not WLCG financed• WLCG can express needs and priorities • PTs’ have their priorities – and less resources
Info
VOMS
DPM
ARGUS
dCache
![Page 14: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/14.jpg)
14
Open Questions
• Where can users, site managers and PTs exchange information?– planned changes, progress– documentation – FAQs, ops tools (exchange)
• Between library and supermarket – “one stop shop” preferred– simple navigation– some level of uniform presentation
![Page 15: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/15.jpg)
15
Is ScienceSoft the Answer???
?
Fresh SoftwareVOMSDP
M
dCache
![Page 16: Post EMI WLCG Middleware Lifecycle Management](https://reader036.vdocuments.us/reader036/viewer/2022062423/56814e3a550346895dbba2c5/html5/thumbnails/16.jpg)
16
Open Question: WN/UI
• Who?– integration, configuration
• How?– RPMs, tarball, CernVMfs, Application Area.....
• First discussions on tarballs started– GridPP, ATLAS, CERN– Commitment needed
• Who will be the WN/UI PT ???