progress s o f t w a r e linux and the progress rdbms gus björklund ([email protected]) wizard...

PROGRESSS O F T W A R E

Linux And The Progress RDBMS

Gus Björklund ([email protected])WizardProgress Software

PUG Challenge 2002, Veldhoven, the Netherlands


Engine Crew

© 2002, Progress Software Corporation3

PROGRESSS O F T W A R EAbstract

The Linux operating system can be the right choice for many Progress RDBMS installations. This talk describes how an Intel-based system can be a highly cost-effective and powerful database server running the Progress RDBMS on RedHat Linux 7.2 on a powerful but inexpensive computer. We will show how to configure the system, what to do, and some things to avoid. Informal benchmark results from the Spring of 2002 are used to illustrate the performance of various configurations.

Time: 90 minutes



Please ask questionsif I do not explain something clearly



Spring 2002Linux Benchmark Team

Bravepoint Dan Foreman John Harlow

Progress Gus Björklund


PROGRESSS O F T W A R EGoals

Prove that Linux is stable and viable in the real world

Prove that price/performance is excellentProduce configuration recommendationsAnd finally…...


PROGRESSS O F T W A R EWhy Linux ?

Commodity Priced Hardware Inexpensive Operating System Linux is no longer a toy Currently being used in production (details to

follow) in mission critical applications Excellent alternative to Microsoft Believers in Open Source We don’t drink Microsoft Kool-Aid


PROGRESSS O F T W A R EWhy Not Linux ?

The constant temptation to apply the “patch of the day”

Credibility (the gap between perception of reality and reality)

Track record is relatively short


PROGRESSS O F T W A R EWhy Not Linux

It’s not like an OS where you call up your friendly 24 hr/day support line and they stick with fixing your problem; we posted a note to one of the Linux lists during our testing and never got a response; There are two ways to go: the slow, plodding, conservative method the complex, inter-dependent, patch-of-

the-day method


PROGRESSS O F T W A R EHistory

August 25, 1991: Linus Torvalds posts to comp.os.minix that he has his experimental kernel running gcc and bash. “just a hobby, won’t be big and professional” Version 0.01 sources posted September 1991

1.0 stable kernel release was March 1994 SCO OpenServer binaries of Progress were being

run on Linux as early as 1995 Not supported by PSC

First Progress release for Linux (RedHat 6.2) was Version 8.3C in Spring 2000


PROGRESSS O F T W A R EExample Production Linux Site

Transplatinum Financial services provider for large trucking

companies Progress V9.1B Linux 2.2.19-6.2.1 DB Size 12GB 137 Users Only problem encountered: “getting the

correct Linux patch combinations installed”



Benchmark HardwareIn Zero Gravity


PROGRESSS O F T W A R EBenchmark Server Hardware

Disks - 10 IBM Deskstar IDE (not SCSI) disk drives, 60 gigabytes each

3Ware 7810 Disk Controller with H/W RAID

Memory - 2 gigabytes, PC133, SDRAMCPUs - 2, 1 gigahertz Pentium IIIASUS Motherboard


PROGRESSS O F T W A R EHardware Cost (August 2001)

Item Price

Server Case $528

Hot Swap Power Supply $338

10 IBM Deskstar 60 GB IDE disks $1890

8 Hot swap cases for disks $200

3Ware 7810 RAID controller $385

Motherboard, RAM, CD, Floppy, Video $891

Shipping $161

Total $4413



Hardware Cost (1991)Sequent SMP Unix System

Item Price

16 x 20 megahertz 80386 cpu, 64 MB RAM, cabinet

$318,500

176 megabytes additional RAM $195,200

21 Fujitsu Eagle 200 MB ? disks $246,000

Dynix/PTX Operating System $46,000

Installation $8,090

Miscellaneous items $23,500

Total $837,290


PROGRESSS O F T W A R EThen Versus Now

Result Then: 170 tps $4,925 per tps

Result Now: 580 tps $7.60 per tps

3.4 x more performance1 / 648 the price per tps


PROGRESSS O F T W A R ESoftware Details

Linux 2.4.18 KernelStock except for the RAID array driversProgress V9.1C09



Multi-million dollar benchmarking lab in a secret location



Benchmark LaboratoryUnderground Bunker


PROGRESSS O F T W A R EBenchmark Team (Partial)


PROGRESSS O F T W A R EBenchmarks

Lies, True Lies, and BenchmarksATM benchmarkDidn’t compare performance to NT,

W2K, or any other Intel Operating System


PROGRESSS O F T W A R EATM Benchmark

Simulates ATM withdrawal transaction read, update account read, update branch read, update teller create history

Execute as many times as possible in given time Produces heavy update workload


PROGRESSS O F T W A R EATM Database

4 tables account: 20,000,000 records branch: 20,000 records teller: 2,000 records history: create 1 per transaction

3.2 gigabyte total size 2 extents, 1,600 megabytes each 2,300 megabytes initial data 175 megabytes indexes


PROGRESSS O F T W A R ECreating The 3.2 GB Database

prostrct create to allocate space about 2 minutes

Creating the initial records via 4GL 35 min

procopy to make backup 10 minutes, including creating extents


PROGRESSS O F T W A R EOther Times

Index rebuild (eventually) 9 minutes, 10 seconds proutil atm -C idxbuild all -TB 31 -TM 32 -B 250 -T /bi -t

sort file size: 381,086,720 bytes

dbanalysis 6 minutes


PROGRESSS O F T W A R EDocumentation


PROGRESSS O F T W A R EBaseline Run

Database size 3 gigabytes

Everything on one disk8k database block size but otherwise

untuned “out-of-the box installation”


PROGRESSS O F T W A R EBaseline Run

Database size 3 gigabytes 4 tables

account, branch, teller, historyEverything on one disk8k database block size but otherwise

untuned “out-of-the box installation”Result: 30 tps (4 users)


PROGRESSS O F T W A R ETop 8 Progress Tuning Items

1. BI on a separate disk2. -spin 50,0003. -B 32000

Later we tried 640004. BI Cluster Size (-bi) 16 mb



5. Page Writers (4)6. BI Writer7. BI Buffers (25+)8. 8K Database Block Size



5. Page Writers (4)6. BI Writer7. BI Buffers (25+)8. 8K Database Block Size

Results before: 30 tps (4 users)Results after: 61 tps (4 users)


PROGRESSS O F T W A R ELinux Changes

Reiser FS some of the best and worst numbers too erratic so we didn’t continue

EXT2 versus EXT3 no difference observed

noatime no difference observed

Larger shared memory segments (32mb default raised to128 mb) 2% gain


PROGRESSS O F T W A R EAdditional Progress Changes

-semsets no statistically significant difference

Index rebuild of the test database 3% gain



Doubled -B from 32000 to 64000 (8k blocks)

Before: 557 tps (54 users)After: 581 tps (58 users)



File System Block Size/DB Block Size rated from best to worst: 4k/8k 4k/4k 1k/8k - not tested

prostrct create took too long 1k/1k- not tested

prostrct create took too long


PROGRESSS O F T W A R EDisk Array Changes

JBOD 8 disks 479 tps (24 users)

RAID 0 Striped 8 disks, 64k stripe size, bi on stripe set 339 tps (26 users)

Moved BI file to internal (non-RAID) drives 481 tps (32 users)


PROGRESSS O F T W A R EDisk Array Changes

RAID 0 8 disks; just striping -B 64000, index rebuild, raise shmmax 581 tps (58 users)

Stripe & Mirror 3 x 2 for data disks 1 pair for BI 304 tps (60 users)


PROGRESSS O F T W A R EDisk Setups Compared

Configuration tps % of base

1 60 GB internal IDE disk (base) 61 100

JBOD, 8 disks 479 785

RAID 0, 8 disks, data + bi 339 555

RAID 0, 8 data + separate bi disk 481 788

RAID 0, tune + separate bi disk 581 952

RAID 10, 3 pair data, 1 pair bi 304 498


PROGRESSS O F T W A R EConventional Wisdom

There’s no such thing! Direct I/O parameter reduced performance

(551 118) but chopped off the spikes in the longest transaction duration; resorted to manual syncing similar to pre-directio days

Enabling On-board disk cache DECREASED Performance (506 474)

RAID 0 Stripe Size 1mb was much better than 64k


PROGRESSS O F T W A R ECost Of After Imaging

Before: 289 tps (92 users)After: 278 tps (82 users)3.8% reduction


PROGRESSS O F T W A R EConstraints

Time - infinite number of benchmarking possibilities

The unexpected: 3 hours to configure the RAID array from JBOD to RAID 10

Ran out of beer on the last night Occasional Disagreement between the

Members of the Benchmarking Team BI Buffers What do we do next? Where do we eat tonight?


PROGRESSS O F T W A R ELinux Virtual Memory

bdflushIt’s in fluxHurriedly overhauled in 2.4.10Will be overhauled again in 2.4.19, soonDirect I/O does not produce results

similar to other Unixen


PROGRESSS O F T W A R EMiscellaneous Stuff

Can’t control the amount of OS Cache at least, we could not


PROGRESSS O F T W A R EWhat we Didn’t Do

RAID 5 - because as you know it’s Evil Software striping & mirroring Raw partitions Non-standard file systems (JFS, etc) 2.5 kernel Timeslice parameters NAS Storage Areas Records per Block



Yes, but what about Windows



Digression:Linux Vs. Windows Performance

Progress 9.1A05 Windows 2000 Advanced Server @$3,500 USD RedHat Linux 6.2, 2.2.14 kernel Same exact hardware for both

IBM Netfinity 8500, 8 x 550 mhz Pentium III 8 gigabytes RAM 40 x 9 Gigabyte SCSI disks 32 megabytes of disk cache

In a previous benchmark, we compared Linuxagainst Windows



Linux Vs Windows PerformanceResults (9.1A05)

Windows: 3,655 tpsLinux 4,508 tps

Linux was 23 % fasterLinux had more idle cpu ( ~ 15 %)


PROGRESSS O F T W A R EConclusions

Linux provides excellent performance Could probably run 100 MFG/PRO users

System was easy to set up, easy to useSystem cost was very lowWe had no way to backup the system !!

493 GB of disk in the array Tape robot could cost $12,000 or more

Better than Windows on Intel


PROGRESSS O F T W A R EWhat Should We Test Next Time ?

Next Linux Benchmark Oct 28, 2002Should we

compare Linux vs Windows with 9.1D? compare AMD vs Intel ? compare filesystems in detail? try BIOS “tweaking”?

Ideas?



Vragen

progress s o f t w a r e linux and the progress rdbms gus björklund ([email protected]) wizard...

Documents