storage performance council (spc) storage performance council (spc) [email protected]

Storage Performance Council (SPC)Storage Performance Council (SPC)www.storageperformance.org

[email protected]

Page: 2 CMG 2005Storage Performance Council (SPC) www.StoragePerformance.org

Presentations and Speakers

An Introduction to the SPCLeah Schoeb (Sun Microsystems, Inc.)

SPC-1C PreviewWalter E. Baker (SPC Administrator & Auditor)

SPC-3 Development StatusMel Boksenbaum (Hitachi Data Systems)

SPC-2 AnnouncementBruce McNutt (IBM Corporation & SPC-2 Chair)

SPC-2 “Wave One” ResultsWalter E. Baker (SPC Administrator & Auditor)

Q&ASandy Wilson (Fujitsu & SPC Steering Committee Chair)


An Introduction to the SPC

Leah SchoebSun Microsystems, Inc.


Council MissionCouncil Mission

The SPC is a non-profit corporation founded in 1998 to

define, standardize, and promote the first industry standard storage benchmarks

to disseminate objective, verifiable performance data

to developers, product managers, and customers of computer systems


SPC ObjectivesSPC Objectives

Drive performance improvement in the computer systems industry for storage

Insure that customers can accurately compare products in a multi-vendor world

SPC

ResultsDissemination

Storage Industry Performance Experts

New BenchmarkDevelopment

Benchmark Maintenance

Test ToolsBenchmark Kits

PerformanceAnalysis Tools

Results ValidationPeer Review

Audit

Service andSupport

SPC

ResultsDissemination

Storage Industry Performance Experts

New BenchmarkDevelopment

Benchmark Maintenance

Test ToolsBenchmark Kits

PerformanceAnalysis Tools

Results ValidationPeer Review

Audit

Service andSupport

Establish a level “playing field” for all manufacturers

Publicize benchmark results

Ensure accuracy and authenticity of published results


SPC Organizational Structure

Membership

Full Members AssociateMembers

Board of Directors- Elects Steering Committee- Approves Standards- Reviews Compliance

Steering Committee-

-Management-Direction

- Administration

Administrator

BenchmarkSubcommittees

Standing Committees

AcademicMembersNo Cost


SPC MembershipSPC Membership


Application Environment

SPC-1OLTP

LDQVOD

LFPDatabase

Emai

l SPC-2

App - hApp -

i

SPC-x

Applications are clustered based on characteristics of load presented to the I/O Subsystem

SPC-x benchmark will be defined for each cluster, starting with SPC-1

SPC BenchmarksSPC Benchmarks


SPC-1C Preview

Walter E. BakerGradient Systems, Inc.

SPC Administrator and Auditor


SPC-1 and storage components

There is a distinct need to provide product performance comparison for storage configuration components such as disk drives, HBAs/controllers, small storage subsystems (single enclosure), etc.

SPC-1 provides that type of product performance comparison, but is typically used for large, complex storage configurations


SPC-1C (Component)

SPC-1C will be the first industry-standard performance benchmark to provide product performance comparisons for individual storage components such as disks, HBAs/controllers, small storage subsystems (single enclosure), storage software, etc.

SPC-1C will be identical to SPC-1 in workload and reporting requirements.

SPC-1C will have reduced audit and filing fees.


SPC-1C Producer Audience

Storage component vendors comprise the primary producer audience for SPC-1C.

SPC-1C will provide performance evaluation and comparison for storage components such as:

• Disk drives

• HBAs/controllers

• Small Storage Subsystems (single enclosure)

• Processors used in the above components

• Storage software such as Logical Volume Managers


SPC-1C Consumer Audiences

Storage solution providers (vendors, resellers, etc.), as well as end users, are the primary consumer audiences for SPC-1C.

SPC-1C will allow storage solution providers to evaluate the storage components that comprise the larger storage solution offered by the provider.• Disks used in an array

• HBAs/Controllers

• Small Storage Subsystems

• Logical Volume Managers


SPC-1C Configuration Components

SPC-1C configurations will consist of the following two components:

• One HBA/Controller (single/dual/quad…).

• One of the following disk drive configurations:

– One, two, or four disks in a “standalone” configuration.

»An external enclosure may be used but only to provide power and/or connectivity for the disks.

– A small storage subsystem (any form factor) with a maximum of sixteen disks.


SPC-1C Availability Test

An optional SPC-1C Availability Test will replace the required SPC-1 Persistence Test.

Applicable in SPC-1C configurations that provide uninterrupted operation and “hot” rebuild/recovery in the case of a single physical drive failure.


SPC-1C Availability Test

The SPC-1C Availability Test will report:

• If processing continues in the case of a single physical drive failure.

• The level of performance when operating in “degraded mode” after the failure.

• The time required to recover from the failure.

• The level of performance during the recovery.


SPC-3 Development Status

Mel BoksenbaumHitachi Data Systems


Description of effortDescription of effort

A protocol independent benchmark that measures storage performance as seen by applications

SPC-1 and SPC-2 characterize lower level components in the storage hierarchy

• Storage from the bottom up

SPC-3 will take the end-user view of the virtualized storage environment.

• Storage from the top down


Environments and workloadsEnvironments and workloads

File Services

Expansion/Business Use

Traversal/Backup

Database

OLTP

Decision Support

Workflow

Refinement/Digital Film Editing

Traversal/Playback


Critical QuestionsCritical Questions

Draft Workload Definition available

www.storageperformance.org

Is a measurement at the application level useful?

What factors make it interesting?

Scale? Volume, users, bandwidth

Constraints? Price, latency, protocol

Topology? NAS, SAN, DAS

Your input is invited and eagerly awaited!

[email protected]


SPC-2 Announcement

Bruce McNuttIBM Corporation

Chair, SPC-2 Technical Subcommittee


Announcing SPC-2Announcing SPC-2 (Dec 6, 2005)(Dec 6, 2005)

New test of storage system performance for environments that require large scale data movement.

Built using a series of application-oriented sequential workloads.

Audited results will be posted today on the SPC web site for 6 tested storage products.

Available now to SPC members/associate members.


Read

(pct)

Write

(pct)

All

(pct)

Random 29 32 61

Sequential 11 28 39

All 40 60 100

Represents environments such as OLTP, database, andmail server:

SPC-1: Predominately random accessSPC-1: Predominately random access


SPC-2: Large Block, Sequential I/OSPC-2: Large Block, Sequential I/O

Sequential tests: valuable to many customers

Customers with dedicated servers.• Data warehouse & Data mining• Scientific Processing• Media

Customers with a mix of applications including sequential.

• Banking & Financial, Insurance, Manufacturing, Medical, Government, many others…

• To such customers, the sequential component of performance is often the most visible.


SPC-2: A SPC-2: A collectioncollection of sequential tests of sequential tests

Why a collection?

• Types of sequential processing can be very different.

• It is easy to include a variety of tests.

• Since the hardware imposes a ceiling on all data rate measurements, a variety of measurements will tend to reinforce each other.


Sequential processing tested in SPC-2Sequential processing tested in SPC-2

Large file processing: simple sequential processing of one or more large files.

Large database queries: scans or joins of large relational tables.

Video on demand: individualized playback from a digital film library.


SPC-2 measurements (Example)SPC-2 measurements (Example)

Data Rate -- Video On Demand

4400 Streams

MI

0

2,000

4,000

6,000

0:00

0:05

0:10

0:15

0:20

0:25

0:30

0:35

0:40

0:45

0:50

0:55

1:00

1:05

1:10

1:15

1:20

1:25

1:30

1:35

1:40

1:45

1:50

1:55

2:00

2:05

2:10

2:15

2:20

Time within Test Run, min

Da

ta R

ate

, MB

/se

c

Large File Processing - Data Rate

0

500

1,000

1,500

2,000

2,500

3,000

3,500

4,000

4,500

5,000

Write 256KiB Read/Write256KiB

Read 256KiB Write1024KiB

Read/Write1024KiB

Read1024KiB

Da

ta R

ate

, MB

/se

c

1 Stream 12 Streams 24 Streams 48 Streams 96 Streams

Large Database Query - Data Rate

0

500

1,000

1,500

2,000

2,500

3,000

3,500

4,000

4,500

5,000

64KiB w/ 1IO/Stream

64KiB w/ 4IOs/Stream

1024KiB w/ 1IO/Stream

1024KiB w/ 4IOs/Stream

Dat

a R

ate,

MB

/sec

1 Stream 12 Streams 24 Streams 48 Streams 96 Streams


Watch This SpaceWatch This Space

Additional results are expected to accumulate rapidly

Watch for them on the SPC Web site:

• http://www.StoragePerformance.org

Aiming for release to non-SPC members in 1H2006


SPC-2 “Wave One” Results

Walter E. BakerGradient Systems, Inc.

SPC Administrator and Auditor



SPC-2 “Wave One” results submitted by four member companies:

• Fujitsu Limited

• Hewlett-Packard Company

• IBM Corporation

• Sun Microsystems, Inc.



Six storage products:

• Fujitsu Storage Systems ETERNUS 6000 Model 900

• HP StorageWorks 8000 Enterprise Virtual Array

• IBM TotalStorage SAN Volume Controller 3.1

• IBM TotalStorage DS8300

• Sun StorEdge 6130 Array

• Sun StorEdge 3510 FC Array


SPC-2 Reported Data

SPC-2 MBPS™SPC-2

Price-Performance Total Price ASU CapacityData

Protection Level

Data RateNumber of Streams

Data Rateper Stream Price-Performance

LFP CompositeWrite Only:

1024 KiB Transfer 256 KiB Transfer

Read-Write:1024 KiB Transfer

256 KiB Transfer Read Only:




LDQ Composite1024 KiB Transfer Size

4 I/Os Outstanding 1 I/O Outstanding

64 KiB Transfer Size4 I/Os Outstanding 1 I/O Outstanding



SPC-2 Reported DataPriced Storage Configuration Name:

The above SPC-2 MBPS™ value represents the aggregate data rate of all three SPC-2 workloads:Large File Processing (LFP), Large Database Query (LDQ), and Video On Demand (VOD)

SPC-2 Large File Processing (LFP) Reported Data

The above SPC-2 Data Rate value for LFP Composite represents the aggregate performance of all three LFP Test Phases: (Write Only, Read-Write, and Read Only).

SPC-2 Large Database Query (LDQ) Reported Data

The above SPC-2 Data Rate value for LDQ Composite represents the aggregate performance of the two LDQ Test Phases: (1024 KiB and 64 KiB Transfer Sizes).

SPC-2 Video On Demand (VOD) Reported Data


SPC-2 Reported Data

SPC-2 MBPS™SPC-2


Protection Level



Overall Composite:

• Large File Processing (LFP)

• Large Database Query (LDQ)

• Video on Demand Delivery (VOD)


SPC-2 Reported Data

SPC-2 MBPS™SPC-2


Protection Level



• SPC-2 MBPS™:

– 263.01 / 343.72 / 1,025.62 / 2,129.47 / 3,217.53 / 3,517.75

• SPC-2 Price-Performance ($ per SPC-2 MBPS™):

– $137.52 / $201.41 / $384.31 / $415.37 / $563.93 / $679.23

• Range of disk devices (spindles) configured:

– 14 / 36 / 112 / 280 / 512 / 672


SPC-2 Reported DataLarge File Processing (LFP)



LFP CompositeWrite Only:


Read-Write:1024 KiB Transfer

256 KiB Transfer Read Only:


SPC-2 Large File Processing (LFP) Reported Data

The above SPC-2 Data Rate value for LFP Composite represents the aggregate performance of all three LFP Test Phases: (Write Only, Read-Write, and Read Only).

• SPC-2 LFP Composite Data Rate (MB/s):

– 221.66 / 248.33 / 833.19 / 1,875.53 / 2,573.57 / 2,707.28

• SPC-2 Price-Performance ($ per MB/s):

– $163.19 / $278.77 / $471.61 / $473.06 / $732.76 / $849.18


SPC-2 Reported DataLarge Database Query (LDQ)



LDQ Composite1024 KiB Transfer Size

4 I/Os Outstanding 1 I/O Outstanding

64 KiB Transfer Size4 I/Os Outstanding 1 I/O Outstanding

SPC-2 Large Database Query (LDQ) Reported Data

The above SPC-2 Data Rate value for LDQ Composite represents the aggregate performance of the two LDQ Test Phases: (1024 KiB and 64 KiB Transfer Sizes).

• SPC-2 LDQ Composite Data Rate (MB/s):

– 323.60 / 389.60 / 1,118.66 / 2,273.09 / 3,697.33 / 4,389.76

• SPC-2 Price-Performance ($/MB/s):

– $111.78 / $177.69 / $352.34 / $394.92 / $452.32 / $591.08


SPC-2 Reported Data

Video on Demand Delivery (VOD)



SPC-2 Video On Demand (VOD) Reported Data

• Number of Streams:

– 310 / 500 / 1,500 / 2,848 / 4,300 / 4,400

• SPC-2 VOD Data Rate (MB/s):

– 243.78 / 393.22 / 1,124.99 / 2,239.77 / 3,381.70 / 3,460.23

• SPC-2 Price-Performance ($/MBs):

– $148.37 / $176.06 / $350.36 / $394.92 / $573.31 / $646.25


SPC Q&A

Sandy WilsonFujitsu Computer Systems Corporation

Chair, SPC Steering Committee


SPC Q&A

Why are there not more SPC member companies releasing SPC-2 results for this announcement?

What type of testing was done, prior this initial release of the benchmark specification?

Why is the benchmark only available to SPC members and not to the general public?

storage performance council (spc) storage performance council (spc) [email protected]

Documents

chair spc

spc benchmarks

spc membership slide

c component spc

spc leah schoeb

spcx applications

application environment

io subsystem spcx benchmark