transforming data center infrastructure from … · 2017-10-19 · limited to the platform...

TRANSFORMING

DATA CENTER

INFRASTRUCTURE

FROM ENTERPRISE TO

TACTICAL EDGE

White Paper By:

Jim Shaw

Executive Vice President of Engineering

Crystal Group Inc.

October 2017

Crystal Group Inc. | 850 Kacena Road., Hiawatha, IA | 800.378.1636 | crystalrugged.com

of 12

Table of Contents

INTRODUCTION .................................................................................... 3

INSTANT ACCESS ANYWHERE ........................................................... 3

DATA CENTER EVOLUTION ................................................................ 4

VIRTUAL RAID ON CPU ........................................................................ 4

NVME RAID ........................................................................................... 4

CPU-ATTACHED SSDS WITH VMD ...................................................... 5

BOOTABLE RAID .................................................................................. 7

MANAGEMENT AND MAINTENANCE .................................................. 7

RELIABLE RAID DATA PROTECTION .................................................. 8

INTELLIGENCE AT THE EDGE ............................................................. 9

ROBUST, RELIABLE, AND RUGGED REQUIREMENTS .................... 10

ABOUT THE AUTHOR ......................................................................... 11

ABOUT CRYSTAL GROUP INC. ......................................................... 11

of 12

INTRODUCTION

The digital universe is rapidly expanding, as organizations and individuals everywhere use

digital devices to capture, share, use, and store data. Organizations worldwide increasingly

rely on fast, frequent, and reliable access to growing volumes of data. Data centers from the

enterprise to the tactical edge are modernizing to meet current and future data demands,

transforming critical data-handling and storage infrastructures with the use of modern, high-

performance hardware, software, and hybrid technologies.

Organizations and individuals alike are generating, sharing, and storing more data today than

at any other point in history. Data volumes are doubling every two years, in an exponential

growth trend that shows few signs of slowing.

Data is also more in demand than ever before. As industries worldwide undergo a digital

transformation, data is integral to an ever-increasing number of applications. Mission-critical,

safety-critical, and critical-infrastructure applications require reliable access to real-time data

(RTD). Instant access to data – whenever and wherever needed – enables fast, informed, and

even automated decision making.

INSTANT ACCESS ANYWHERE

Data needs to be instantly available whenever and wherever needed in a wealth of military,

aerospace, power, homeland security, search and rescue, law enforcement, and public safety

applications. Rapid and reliable access to data is imperative and can mean the difference

between life and death in these and other markets.

To meet growing demands for real-time data access, data centers are actively being deployed

in remote locations, ranging from military forward operating bases (FOBs) to remote power

stations, and on mobile platforms, including manned and unmanned ground vehicles, ships,

aircraft, spacecraft, and even the International Space Station (ISS).

Digital devices are capturing, transmitting, storing, and accessing data in virtually all corners

of the globe and in orbit. The size of data sets and pool of data sources are growing

increasingly larger, as is the need to access data faster and more frequently.

Real-time data requirements are pushing faster growth in modular, mobile, and remote data

centers, designed to provide more frequent access and faster response times in the field.

Rugged computer systems are delivering enterprise-class data center capacities and

functionality to the field, such as in remote power stations, and to the tactical edge like today’s

digital, multi-domain battlefield.

All these trends are changing the face of the data center, an essential element of modern

workflows.

of 12

DATA CENTER EVOLUTION

Data continues to grow in volume, variety, and velocity – known as the 3Vs of big data – and

data centers are under considerable pressure to keep pace. Data centers of all sizes and in

all locales, from the enterprise to the edge, are evolving to meet rapidly growing and changing

data demands.

Data centers in the enterprise and at the edge are investing in modern technologies to

increase compute, storage, and networking capacities – by 30 percent to 40 percent annually.

Data centers are adopting a hybrid approach combining capable hardware and software, such

as hardware and software RAID technologies, and tapping into the Internet of Things (IoT)

and public, private, and hybrid clouds.

The goal is to expand data center capacities and capabilities without growing the physical or

carbon footprint, maintenance demands, and power or thermal management requirements. In

short, organizations worldwide have been waiting for groundbreaking innovations, rather than

incremental improvements, in data center technology.

Growing data demands are driving the need for disruptive data center technologies that

enable high-capacity, high-bandwidth, and low-latency data storage and data management in

the most efficient and streamlined way and in as small a space as possible.

VIRTUAL RAID ON CPU

Intel® has introduced new technologies targeted at optimizing data center infrastructures, of

all sizes and in all locales, to meet current and future data demands. Intel Virtual RAID on

CPU (Intel VROC) is poised to transform data centers, in much the same way RAID –

Redundant Array of Independent Disks or Redundant Array of Inexpensive Disks – did

decades ago.

Intel VROC is a hybrid RAID solution that combines the attributes of software RAID and

hardware RAID systems. It uses the CPU to calculate RAID logic on the software side, for

example, while aggregating Non-Volatile Memory Express (NVMe)-based solid-state drives

(SSDs) behind an integrated hardware controller, known as the Intel® Volume Management

Device (VMD).

Intel VROC enhances the benefits of NVMe SSD and VMD technologies to boost performance,

reduce latency, and streamline management and maintenance.

NVME RAID

The NVM Express (NVMe) communications interface or protocol is designed specifically to

support solid-state storage devices such as flash memory, and to overcome challenges

associated with legacy storage buses, including serial ATA (SATA) and serial attached SCSI

(SAS). Developed by a consortium of technology companies under NVM Express Inc., NVMe

of 12

is a popular choice for high-performance computing (HPC) systems for enabling faster job

completion, cluster utilization, and six time (6X) performance improvement over SATA.

NVMe takes advantage of Peripheral Component Interconnect Express, PCI Express or PCIe,

to achieve fast data transfer speeds. PCIe Gen 3, for example, is capable of speeds of up to

985 megabytes per second per lane. Offering high throughput and low latency, NVMe helps

to eliminate storage bottlenecks, benefitting big data and data-intensive workloads and

empowering fast decisions based on real-time data.

Beneficial for industries that need to comply with data backup or replication requirements and

regulations, NVMe provides fast data transfer speeds to help meet strict or tight deadlines

even as data volumes continue to grow in size. It also boasts a low total cost of ownership

(TCOS) for virtualization environments by increasing virtual machine (VM) density, supporting

more virtual servers on one physical server.

CPU-ATTACHED SSDS WITH VMD

Intel VROC takes NVMe-based devices to the next level by enabling NVMe SSDs to connect

directly to the central processing unit (CPU) using Intel VMD, a hardware controller integrated

into Intel Xeon Scalable Processors inside the CPU PCIe root complex. Intel VROC leverages

the Xeon Scalable Processor hardware architecture to enable NVMe RAID, and to support

scalability with flexible drive configurations.

Each Intel Xeon Scalable Processor features three

VMDs, each of which can manage x16 PCIe lanes and

support up to four NVMe SSDs. The processor handles

PCIe bifurcation, splitting the x16 PCIe lanes into

subgroups without the need for a multiplexer (PLX) chip.

With four SSDs connected, for example, each device

would be allotted x4 PCIe lanes.

Using Intel VROC, a single Intel Xeon Scalable

Processor is capable of supporting up to 12 NVMe SSDs

directly attached to the CPU, and up to six RAID arrays.

That number doubles when adding PCIe switches or

dual-socket system configurations.

The Intel UltraPath Interconnect (Intel UPI), a coherent

interconnect for scalable multi-processor systems,

enable data RAID across multiple VMDs and across

CPUs. Intel Xeon Scalable Processors that support Intel

UPI provide two or three links for connecting to other Intel

Xeon processors with a high-speed, low-latency path to

the other CPU sockets. Image: Intel Corp.

of 12

With Intel VROC, NVMe SSDs connect directly to the CPU to achieve full performance

potential, with reduced latency and increased bandwidth, while also avoiding the cost,

complexity, and power consumption of traditional hardware RAID host bus adapter (HBA)

cards between the drives and the CPU.

Traditionally, NVMe drives had to go through the Direct Media Interface (DMI) which was

limited to the Platform Controller Hubs (PCH) x4 PCIe connection, limiting bandwidth to

approximately 2 gigabytes per second (GBps) for DMI 2.0 and 3.6GBps for DMI 3.0. An

external host bus adapter could be used to provide more bandwidth, but would also increase

system cost, complexity, and power usage.

VROC eliminates this bottleneck, bypassing the PCH and connecting directly to the Intel Xeon

Scalable Processor VMDs. Doing so offers a full x16 PCIe interface providing nearly 16GBps

of bandwidth per VMD. Real-world benchmarks from multiple sources have yielded more than

13GBps throughput and input/output operations per second, or IOPS, in excess of 97,000 in

sequential read testing in four NVMe SSD RAID 0 configurations.

Tests have confirmed 12GBps to 13GBps speeds via IOMeter with PCIe x16 cards that house

up to four M.2 NVMe drives per card. Users can add cards to the system until all PCIe lanes

are consumed (x16 per card). The latest Xeon Scalable Processors (Purley/Skylake-SP)

support up to 48 PCIe lanes per processor.

NVMe SSDs with Intel VROC quickly and efficiently handle data-heavy and compute-intensive

workloads, resulting in fewer hardware queues. In traditional systems, multiple read and write

disk operations can overlap in the same period. Eight access operations at the same time

of 12

results in a queue depth of eight, for example. Using Intel VROC, NVMe SSDs have direct

access to the CPU, including Intel Xeon Scalable Processors, to address operations faster

and avoid hardware queues with a buildup of outstanding access operations.

BOOTABLE RAID

Intel VROC enables NVMe bootable RAID without a host bus adapter (HBA). Intel VMD can

use up to x16 PCIe lanes and up to four NVMe devices to achieve an incredibly fast, bootable

RAID array. This capability enables a lightning-fast operating system (OS) capable of tackling

even the most demanding and data-intensive workloads. Intel VROC works in conjunction

with Intel’s NVMe devices and select third-party NVMe SSDs to enable bootable RAID

functionality.

Intel VROC is part of the Intel Rapid Storage Technology enterprise (Intel RSTe) 5.x driver

family, which provides a graphical user interface (GUI) with which to configure, manage, and

check the status of Intel VROC NVMe RAID volumes. Intel VROC uses Intel VMD to provide

several features that Intel RSTe legacy NVMe RAID does not offer, including: bootable RAID,

surprise hot-plug, LED management, closed RAID5 write loops, and support for third-party

SSDs.

Traditional systems require the use of a separate drive for the operating system (OS). Booting

from Intel VROC RAID enables the OS to achieve incredible read/write speeds. Boot support

is restricted to a single VMD, but data arrays can span across multiple VMDs and even

multiple CPUs via Intel’s UltraPath Interconnect (UPI), offering exciting options for building

large NVMe RAID volumes.

To enable different RAID modes, Intel has developed a dongle that plugs into a proprietary

port on the system’s motherboard. Intel offers three different VROC licensing options: Pass-

Through, Standard, and Premium. Pass-Through mode has no RAID support, except RAID 0

for the Intel SSD DC P3608 Series, and enables the connection of stand-alone NVMe SSDs

to the VMD through enabled PCIe lanes with no dongle required. The Standard hardware key

enables RAID 0/1/10, while the Premium key allows RAID 0/1/5/10.

MANAGEMENT AND MAINTENANCE

The simplicity, ease, and efficiency of installation, management, and maintenance weigh

heavily in the decision whether to adopt any new data center solution. Data protection is,

likewise, a predominant concern, particularly for mobile and remote data centers, including

those deployed on the battlefield and in the national power grid.

Systems that take advantage of NVMe SSDs with Intel VROC and Intel Xeon Scalable

Processors are designed for fast and easy deployment, such that they can be up and running

quickly in enterprise and tactical edge environments. Using Intel VROC, data center

administrators can create and delete RAID volumes in pre-OS and OS environments,

configure RAID settings with either a user interface or a command line, and manage the low-

of 12

maintenance system locally or remotely via centralized controllers, such as through a Web-

based RESTful agent framework.

Intel VROC’s LED management feature supports SSD indicator lights on the backplane of the

server enclosure to offer instant feedback and simplified maintenance. The LEDs blink

different patterns to indicate the status of each SSD in the RAID array. Intel VROC follows the

blinking patterns defined by the International Blinking Pattern Interpretation (IBPI) standard.

Using status-indicator LEDs, administrators can quickly identify the RAID status – including

normal, initialization, degraded, or fail – and also locate a specific drive among potentially

hundreds of SSDs. A particularly valuable tool in large enterprise installations as well as

remote, mobile, and edge applications, VROC can trigger automatic e-mail notifications to

alert administrators of events that require attention.

RELIABLE RAID DATA PROTECTION

In addition to streamlining management and maintenance, Intel VROC helps to boost system

availability and data protection in both enterprise and deployed operational environments.

Intel VROC supports surprise hot-plug functionality, enabling administrators to replace a failed

drive without having to reboot the server. Specifically, Intel VMD enables hot swap

replacement of NVMe SSDs from the PCIe bus without shutting down the system. Hot-swap

capabilities combined with LED status indicators help further boost the reliability, availability,

and serviceability (RAS) features of Crystal Group Rugged Servers and hyperconverged

infrastructure (HCI) solutions with NVMe SSDs.

Host insert and surprise removal can be quite common in mobile, remote, and edge storage

systems deployed in military, critical infrastructure, public safety, and myriad other challenging

environments. Such events can result in the corruption and loss of key data, upon which users

Ima

ge

: In

tel C

orp

.

of 12

rely to make important, real-time decisions. Intel VROC’s hot-swap and data protection

capabilities help support host insert and surprise removal events, preventing data loss.

Software RAID is vulnerable to the RAID5 Write Hole, an unfortunate phenomenon whereby

a write operation to a RAID 5 volume is interrupted by a blue screen (known as the Blue

Screen of Death or BSoD) or power loss, resulting in data corruption and loss. Most hardware

RAID systems require a backup power supply – which adds cost, power consumption, heat,

and noise – to avoid data loss from such events.

The susceptibility of software RAID to data corruption and loss from system crashes and

power loss have prevented many data center professionals from adopting the technology and

opting instead for hardware RAID solutions. Intel VROC features, specifically journaling and

double fault protection, provide a welcome remedy to data loss challenges and make a

compelling case for the use of hybrid solutions.

Intel VROC uses a logging system and does not require the use of a backup power unit, such

as a costly uninterruptable power supply (UPI), to protect data when power loss occurs

unexpectedly. VROC’s journaling feature helps to ensure data remains safe even when a

RAID 5 volume is in a degraded state and suffers from power loss. Intel VROC RAID 5 double

fault protection is dependent upon the use of NVMe SSDs with power loss protection.

INTELLIGENCE AT THE EDGE

Aerospace, defense, energy, homeland

security, border control, public safety, law

enforcement and a wide variety of other

mission-critical, safety-critical, and critical-

infrastructure applications are taking data

centers to the edge, in permanent, semi-

permanent, and temporary structures and

vehicles in remote locations and even

harsh environments. Data center administrators are also proactively and reactively, as the

case may be, modernizing enterprise and edge data infrastructures of all sizes in all types of

locations.

The end goal in virtually all current data center scenarios is to do more with less – that is, to

increase rack density, data storage capacity, reliability, scalability, and data protection while

reducing the size, power consumption, and thermal management needs of systems and the

time, effort, and cost required for management and maintenance tasks.

Intel VROC’s performance and features augment the already attractive benefits of

hyperconverged infrastructure (HCI) systems, a popular choice for modernizing data centers

while streamlining management and maintenance. HCI tightly integrates compute, storage,

of 12

networking, and virtualization resources in a single, purpose-built hardware rack or transit box

supported by a single vendor.

HCI consolidates multiple devices in a single compute, storage, and networking cluster, and

adds advanced software to take advantage of the Internet of Things, the cloud, and resource

virtualization. A single HCI rack, for example, can effectively replace at least three separate

devices to deliver more features and functionality in the same or smaller physical size, weight,

and power consumption footprint than traditional and legacy systems. Intel VROC, NVMe

SSDs, and Intel Xeon Scalable Processors can boost the already impressive performance,

capacity, and scalability of HCI systems, which are modernizing data centers in the enterprise

and at the tactical edge.

ROBUST, RELIABLE, AND RUGGED REQUIREMENTS

Crystal Group is pairing its high-performance, rugged server, HCI, workstation, and data

storage hardware with Intel’s next-generation data center technologies – including Intel VROC,

SSDs, and Xeon Scalable Processors – to meet rapidly growing data demands and ensure

fast, reliable access to critical information in fixed-site, remote, and mobile applications across

aerospace, defense/military, industrial, and commercial markets.

The move to Crystal Group hardware with solid-state drives (SDDs) can provide considerable

advantages compared to hard disk drives (HDDs), including: greater storage capacity and

speeds, increased stability and ruggedness, reduced power consumption and heat dissipation,

critical redundancy and failover protection, and enhanced mean time between failures (MTBF).

Crystal Group’s full line of commercial off-the-shelf (COTS) and custom-engineered Rugged

Server, Embedded Computer, Rugged Storage, and HCI solutions are now available with

Intel’s latest data center (DC) storage and processing technologies, including Intel VROC, the

Intel SSD DC Family, and Intel Xeon Scalable Processors. The all-new line of Crystal Group

Fully Optimized Rugged Computer Equipment (FORCE), including FORCE Rugged Servers,

leverage Intel Xeon Scalable Processors and optional Intel SSDs and Intel VROC

technologies to provide next-generation data handling and storage capabilities in a rugged,

reliable system.

Designed to bring enterprise-level data center capabilities to the edge, the Crystal Group

FORCE Rugged Server line of rugged rack-mount servers marry the

latest Intel Xeon Scalable Processors with high-speed networking, enhanced I/O, platform

security, and thermal management in a modular, customizable high-performance computing

(HPC) system to bring fast, reliable processing power and storage capacity to remote and

mobile environments.

Crystal Group FORCE Rugged Servers employ single or dual Intel Xeon Scalable Processors,

each able to support up to 24 cores and 48 PCIe lanes to provide a substantial boost in

performance; Intel Advanced Vector Extensions 512 (AVX-512) ultra-wide vector processing

of 12

to accelerate repetitive tasks; and a Platform Controller Hub (PCH) with 10 Gigabit Ethernet

(10GbE) ports for high-speed communication, up to 14 SATA 3 ports for expanded storage

capacity, and QuickAssist Technology (QAT) for enhanced security, authentication, and

compression. Crystal Group FORCE RS1104, RS2608, and RS3712 Rugged Servers feature

a lightweight chassis in 1U, 2U, and 3U sizes with 19-, 22-, and 25-inch depths, a choice of

ATX or eATX motherboard and air- or liquid-cooled thermal management, and a wide variety

of front-end configuration options, including 2.5-inch drives, sanitize button for security, serial

and USB ports, CMOS battery, pump assembly for liquid cooling, circuit breaker, card reader,

USB sound card, DVD, and LCD.

Crystal Group Rugged Servers, HCI systems, and electronics solutions are designed and built

to provide high performance, reliability, scalability, and flexibility in an efficient size, weight,

power consumption, and thermal management footprint from the enterprise to the edge.

Crystal Group is an Intel Technology Provider and award-winning Intel Internet of Things

Solutions Alliance Member, and its growing product portfolio follows the Intel Roadmap.

ABOUT THE AUTHOR

Jim Shaw is the Executive Vice President of Engineering at Crystal Group. Since 2006, Jim has led the engineering department and the new product development team. Jim’s revolutionary design prowess led to the birth of the rugged series (RS) chassis for the military and industrial computing markets. During Jim’s tenure at Crystal Group the company has expanded its rugged product lines with embedded, storage, displays, switches, carbon fiber options, and custom power supply designs. Jim holds a Bachelor of Science degree in Mechanical Engineering from Iowa State University

and a Masters of Business Administration from the University of Iowa. Prior to joining Crystal

Group, Jim held a management position in engineering at Rockwell Collins, located in Cedar

Rapids, Iowa. While at Rockwell Collins, he was honored three times as an Engineer of the

Year nominee for his work in high performance electronics packaging. He has authored or co-

authored eight international patents.

ABOUT CRYSTAL GROUP INC.

Over the past 30 years, Crystal Group has designed, manufactured, tested, and delivered

rugged, reliable computer and electronics systems that leverage the most advanced hardware

and software available. As a Platinum Intel Technology Provider and award-winning Intel

of 12

Internet of Things Solutions Alliance Member, Crystal Group adheres closely to the Intel

Roadmap to bring the very latest Intel innovations to aerospace and defense, automotive,

commercial, industrial, power, and other discerning customers quickly and efficiently.

Crystal Group Inc., an employee-owned small business located in Hiawatha, Iowa, USA, is a

technology innovation leader specializing in both custom and COTS products for defense,

government and industrial markets since 1987. Crystal Group designs and manufactures

installation-ready rugged servers, displays, networking devices, embedded systems, power

supplies and storage devices that fit critical applications in demanding environmental

conditions.

The company is certified to quality management standards AS9100C:2009 and ISO

9001:2008. Crystal Group products meet and exceed MIL-STDs 810, 167-1, 461, MIL-S-901,

IEEE and IEC industrial standards. Additionally, the company offers integration services,

configuration management, product life-cycle planning and 5+ year warranties.

© 2017 Crystal Group Inc. All rights reserved. All marks are properties of their respective

owners. Design and specifications are subject to change.

transforming data center infrastructure from … · 2017-10-19 · limited to the platform...

Documents