ide disk servers at cern
DESCRIPTION
IDE disk servers at CERN. Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003. Introduction. HEP computing in the past: mostly reading from, processing, and writing to (tape) files sequentially Mainframe era (until ~ 1995): single machine, CPUs, tape drives, little disk space - PowerPoint PPT PresentationTRANSCRIPT
IDE disk servers at CERN
Helge Meinhard / CERN-IT
CERN OpenLab workshop
17 March 2003
Introduction
HEP computing in the past: mostly reading from, processing, and writing to (tape) files sequentially
Mainframe era (until ~ 1995): single machine, CPUs, tape drives, little disk space
In response to scaling problem, development of SHIFT architecture (early 1990s) Scalable farm out of ‘commodity’ components
RISC CPUs (PowerPC, MIPS, Alpha, PA-RISC, Sparc) SCSI disks
SHIFT architecture
Network- FDDI- Hippi- Myrinet- Ethernet
Tapeserver
Diskserver
Batchserver
Interactiveserver
BatchanddiskSMPNetwork
- Ethernet
PC batch nodes
1995: First studies at CERN of PCs as batch nodes (Windows NT)
1997 onwards: Rapidly growing interest in Linux (on IA32 only)
1998/99: First production farms for interactive and batch services running Linux on PC hardware at CERN
PC disk servers
1997/98: Prototypes with SCSI disks 1998/99: Prototypes with EIDE disks
Different IDE adapters Not RAIDed
1999/2000: First Jumbo servers (20 x 75 GB) put into production
2001: First rack-mounted systems 2002: 97 new servers (54 TB usable) 2003: 1.3 TB usable in one server at 13 kCHF Total usable capacity today: ~ 200 TB
1997: ~700 GB SCSI/Sparc
2000/2001: 750 GB PC/EIDE (1)
2000/2001: 750 GB PC/EIDE (2)
2002: 670 GB PC/EIDE
2 systems
Assumed price per GB usable
0
50
100
150
200
250
300
350
400
450
1994 1995 1996 1997 1998 1999 2000 2001 2002
Time
CH
F /
GB
Disks only
Complete systems
SCSI/RISC
EIDE/PC
Real price per GB incl. infrastructure
0
5
10
15
20
25
30
35
40
45
50
Oct-
00
Nov-0
0
Dec-0
0
Jan-
01
Feb-0
1
Mar
-01
Apr-0
1
May
-01
Jun-
01
Jul-0
1
Aug-0
1
Sep-0
1
Oct-
01
Nov-0
1
Dec-0
1
Jan-
02
Feb-0
2
Mar
-02
Apr-0
2
May
-02
Jun-
02
Jul-0
2
Aug-0
2
Sep-0
2
Oct-
02
Nov-0
2
Dec-0
2
Jan-
03
Feb-0
3
Date
CH
F /
GB
Capacity per server
0
500
1000
1500
2000
2500
3000
3500
Oct-
00
Nov-0
0
Dec-0
0
Jan-
01
Feb-0
1
Mar
-01
Apr-0
1
May
-01
Jun-
01
Jul-0
1
Aug-0
1
Sep-0
1
Oct-
01
Nov-0
1
Dec-0
1
Jan-
02
Feb-0
2
Mar
-02
Apr-0
2
May
-02
Jun-
02
Jul-0
2
Aug-0
2
Sep-0
2
Oct-
02
Nov-0
2
Dec-0
2
Jan-
03
Feb-0
3
GB
Gross
Usable
Today’s servers: Specifications
19” rackmount, IA32 processor(s), 1 GB, 2x80 GB system disks, GigE (1000BaseT), redundant power supplies
>500 GB usable space on data disks Hardware RAID offering redundancy Hot-swap disk trays
Performance requirements network – disk: 50 MB/s reading from server @ 500 GB 40 MB/s writing to server @ 500 GB
5 years on-site warranty
Lessons learnt
Capacity is not everything, for good performance need CPU, memory, RAID cards Good OS and application software Network connectivity Large number of spindles
Firmware of RAID controllers and disks critical Redundancy (RAID) is a must, required
performance possible only with mirroring (RAID 1) so far
Outlook
Good price/performance has risen interest in other application domains at CERN AFS and MS DFS servers Web servers, mail servers Software servers (Linux installation) Data base servers (Oracle, Objectivity/DB)
Access pattern of physics analysis likely to change Investigating different file systems (XFS), RAID 5 (in
software), … Architecture constantly being reviewed
Alternatives investigated: data disks scattered over large number of batch nodes; SAN
Conclusion
Architecture of early 1990s still valid May even carry us into LHC era…
Important improvements made Price/performance Reliability (RAID)
Will review architecture soon (2003) New application areas New access patterns for physics analysis