triumf site report corrie kost head scientific computing
DESCRIPTION
TRIUMF SITE REPORT Corrie Kost Head Scientific Computing. TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost. TRIUMF MANPOWER. TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost. WestGrid – UBC/TRIUMF Site. - PowerPoint PPT PresentationTRANSCRIPT
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
TRIUMF SITE REPORT
Corrie KostHead Scientific Computing
TRIUMF MANPOWER
Public Computing Services 2.0 FTE
Network Infrastructure 1.5 FTE
Software Support & Security 3.0 FTE
PC Support Group 2.0 FTE
Data Acquisition Systems 2.5 FTE
Private CPU farms and Westgrid 0.5 FTE
Conferencing facilities 0.3 FTE
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
WestGrid – UBC/TRIUMF Site
• 504 dual 3.06 GHz Xeon blades• Red Hat Linux 7.3 compute nodes• PBS Scheduling with Maui• 10 TB disk storage• 24 TB tape storage• Direct Gigabit connection between sites sites• Possible 10GB in future• Subatomic physics allocation: ~1/3 of compute
resources• TWIST will be a Beta user this month.
TRIUMF/UBC first HEP site to install large (1008 CPU) blade cluster?
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
WestGrid – UBC/TRIUMF Site(www.westgrid.ca)
• Like the summer rains – two weeks away since August!• Blade power consumption exceeded expectations
– Use 12 of 14– Replace hot swap power supplies
• GPFS has some teething problems• Reduced cabling welcome – wireless would be better!• Very little infant mortality (no blades failed!)
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
14 dual-cpu blades in 7U Chassis
Side view of a blade
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
Rear View of Blades Chassis
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
WestGrid –UBC/TRIUMF Site
Storage
1 Rack Blades
6 Racks
UPSBatteries Inverters Switch Gear
SATA IDE Drives system from HardData
64 bit 1.6GHz Opteronsapprox. equal to
32 bit 2.8GHz P4 (Xeons)
Performance details: Konstantin Olchanski ([email protected])
First look at Dual 1.6GHz Opterons
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
Misc. Developments
• 1 TB Raid 5 disks attached to Linux (TRSHARE) for Linux & Windows users (via Samba)
• Mosix & PBS Clustering – Steve McDonald 11:30 Wed
• Bandwidth Management – Steve McDonald – 11:40 Wed
• Colubris Radius Server – Steve McDonald – 11:50 Wed
• SPAM fighting at TRIUMF – Andrew Daviel – 13:50 Tue
• Document Management System: Docushare
• Windows 2003 Terminal Service (TNT2K3)
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
Misc. Developments
TRIUMF Site Report for HEPiX/HEPNT, Vancouver, Oct20-24/2003 – Corrie Kost
•Videoconferencing
–One older room with limited capability
–A new room with Westgrid funded equipment
–Several protocols supported: ISDN, H323, VRVS, AccessGrid
–Participation in the Westgrid Collaborative/Visualization effort to improve videoconferencing quality and tools
First Transatlantic Native 10 Gigabit Ethernet
ConnectionCatalin Meirosu
CERN and “Politehnica” University of Bucuresti
On behalf of the
International GRID Testbed
International
Testbed
The First Transatlantic 10 GE
WAN PHY over an OC-192c circuit using lightpaths provided by SURFnet and CANARIE
CiscoONS 15454Force10
E 600
Force10E 600
HPItanium-2
HPItanium-2
CiscoONS 15454
CiscoONS 15454
CiscoONS 15454
CiscoONS 15454
Ixia400T
IntelItanium-2
IntelXeon
Ixia400T
10GE WAN PHY 10GE LAN PHY OC192c
Ottawa Toronto Chicago Amsterdam Geneva
9.24 Gbps using traffic generators
5.5 Gbps using TCP on PCs
6 Gbps using UDP on PCs
ITU Telecom World 2003 12
International
Testbed
The First Transatlantic 10 GE
CiscoONS 15454Force10
E 600
Force10E 600
HPItanium-2
HPItanium-2
CiscoONS 15454
CiscoONS 15454
CiscoONS 15454
CiscoONS 15454
Ixia400T
IntelItanium-2
IntelXeon
Ixia400T
10GE WAN PHY 10GE LAN PHY OC192c
Ottawa Toronto Chicago Amsterdam Geneva
ITU Telecom World 2003 13
International
Testbed
ESTA, IST-2001-33182