hp dbc reference architecture technical overvie · the hp dbc reference architecture (dbc ra) is a...

Technical white paper

HP DBC Reference Architecture technical overview Reference architecture for running hundreds to thousands of I/O demanding workloads on a Microsoft private cloud

Table of contents

Executive summary 3

Business drivers 3

Introduction to the HP DBC Reference Architecture 4

Benefits 4

Solution architecture 7

Hardware components 7

Software components 13

Guest workload support 15

Built-in high availability 15

Disaster recovery 16

Performance and scalability 17

Power consumption under load 19

Ordering and purchasing the DBC Reference Architecture 22

Assessment and planning 22

Purchasing and build 24

Deploying the DBC Reference Architecture 24

Hardware configuration 24

Host OS and configuration steps before System Center installation 32

System Center management stack configuration 40

DBC Reference Architecture bill of materials 43

DBC Reference Architecture Base configuration 43

DBC Reference Architecture Full configuration 45

Appendix 47

Base DBC RA racking, wiring and cabling diagrams 47

Full DBC RA wiring and cabling diagrams 50

DBC RA storage block SAS cabling 53

DBC RA power cabling 54

For more information 55

3

Executive summary

The HP DBC Reference Architecture (DBC RA) is a reference architecture solution for database consolidation, provisioning, and running thousands of I/O demanding database workloads in a private cloud model. The DBC RA is designed to solve the challenges of consolidation and database sprawl from massively increasing volumes of data across a company, and meet rapidly changing business requirements by provisioning new databases on-demand where-ever and when-ever.

To facilitate database management in the data center and enable private cloud deployments, HP has developed the DBC RA based on mutual HP and Microsoft® experiences in Microsoft SQL Server consulting and architecting infrastructure for large enterprises and feedback from existing customers. At its core, the DBC RA provides a foundation for building a high-performance Microsoft Hyper-V virtualization platform that has been optimized to consolidate and provision hundreds to thousands of database workloads while providing extremely high availability at all levels – from the underlying network and storage fabrics up to the virtual machine (VM) layer.

The DBC RA is built on the HP Converged Infrastructure, including the HP BladeSystem architecture, HP Virtual Connect fabric, and HP P2000 G3 storage arrays. By designing on the HP BladeSystem, the DBC RA can be sized and scaled in a modular fashion, simplifying scaling up and out as additional resources are required. Additionally the HP BladeSystem architecture helps to not only reduce the footprint of the solution but also reduce the environmental requirements through advanced power and thermal capabilities. HP Virtual Connect provides the converged fabric for the DBC RA and the ability to specifically allocate network ports and associated bandwidth per the DBC RA requirements; a very important capability when building a virtualization platform due to the complex networking requirements. Coupling these technologies with the HP P2000 G3 Modular Smart Array (P2000) storage subsystem, the DBC RA provides an extremely dense platform for the deployment of enterprise databases that require high levels of storage performance.

At the core of the DBC RA are the HP ProLiant BL460c (or BL465c) Gen8 blade servers. This latest generation of blade servers provides a number of advancements in intelligence, control, and performance, helping drive down the operational expenses of the DBC RA platform. Innovations such as HP Integrated Lights-Out 4 (iLO 4), HP Active Health System, HP Smart Update, and HP SmartMemory help customers save substantial administrative time and effort in managing and supporting the DBC RA platform; enabling customers to reinvest that time savings in other higher value activities.

The DBC RA is designed to give customers the flexibility to implement the solution themselves or to leverage HP’s services to assemble the hardware and even implement the solution on-site. The combination of an optimized architecture with services implementation makes it possible to speed deployment, quickly provision SQL Servers, simplify management, and ultimately reduce IT costs. This document describes the benefits of the DBC RA, describes the solution architecture, outlines some typical use cases, and describes a reference example for building out a Microsoft private cloud reference architecture solution.

Target audience: This document is intended for technical decision-makers and solution architects.

Business drivers

Data in most enterprises is ubiquitous and massively increasing, used in line-of-business applications, in departmental servers, and in e-commerce platforms. This, coupled with the ease of deploying database platforms such as Microsoft SQL Server, often leads to the proliferation of databases of all sizes running on a variety of operating systems and database versions.

Pervasive database sprawl creates a broad range of problems for the business, including the following:

Agility – Today’s database implementations tend to be inconsistent and isolated, which makes it almost impossible to adapt quickly to changing business conditions. And customers are asking IT to provision new databases to support new business strategies or asking IT to grow current database resources as they run out of storage.

Manageability – The very nature of database sprawl – with databases from multiple vendors and databases from the same vendor at different revision levels – makes it difficult to manage databases effectively and efficiently.

Cost – Hundreds and thousands of databases inevitably lead to power, cooling, and real estate inefficiencies at a time when IT management is being asked to reduce costs and implement green solutions. The cost of operating and maintaining these databases – often legacy implementations – is also high.

Confidence – With database sprawl comes the very real risk that you will not be able to find the data you need when you need it.

4

The challenge becomes finding a solution that eliminates database sprawl, simplifies implementation, reduces capital and operating expenses, and delivers high levels of confidence. The HP DBC Reference Architecture provides the platform to achieve these goals and help you regain control of your database infrastructure.

Introduction to the HP DBC Reference Architecture

The new model for IT delivery is the private cloud, which transforms existing data center infrastructures into a single cloud of shared compute resources within which application services are provided as utilities. The combination of infrastructure convergence and resource sharing helps get applications up-and-running faster, while dynamic scaling and provisioning can keep you on top of rapidly changing business needs.

The private cloud is deployed within your own data center, with services restricted to specified classes of users and business units. Its key features are as follows:

Pooled resources – Key resources are pooled and abstracted into units that allow you to dynamically provision and scale applications and resources.

Self-service – Applications and resources are delivered to business users as services via a simple, interactive portal, enabling automated provisioning, monitoring and chargeback.

Elasticity – Resources can quickly be expanded or contracted through automation or workflows, allowing a rapid response to changing business needs.

Usage-based – Resource utilization can be metered in this service-based implementation; thus, end-users are only billed for resources they have actually used.

To create a solution that can help you move in a single step from an aging infrastructure to the benefits of a private cloud, HP and Microsoft began by carefully analyzing the performance needs – in particular, the I/O requirements – of a large number of existing database implementations. This analysis led to the development of profiles for the most commonly used database workloads and, in turn, drove the design for an innovative, enterprise-class solution that can be used to consolidate and provision such workloads. The result is the DBC Reference Architecture solution, a high IOPS, fault-tolerant design built on the HP Converged Infrastructure that forms the foundation of a Microsoft private cloud infrastructure and a platform for your SQL consolidation initiatives.

Benefits

There are a number of benefits that the DBC RA solution provides for organizations looking to deploy a private cloud infrastructure to support high I/O demanding workloads such as SQL Server and other database transactional workloads. These benefits help to drive down both acquisition and operational costs and reduce your total cost of ownership (TCO).

Time to value

The DBC RA is based on a pre-tested, validated architecture design, thus eliminating the time-consuming tasks associated with designing, testing, and certifying the solution on-site. This can significantly lower the time to order the solution to when it is in place and ready to use by the business. Since the DBC RA can be delivered already racked and cabled using HP Factory Express, customers can spend time on the higher value tasks of setting up the management environment, configuring customized workflows and enabling automation of common IT tasks, rather than setting up the underlying infrastructure components.

Additionally, the Microsoft Assessment and Planning (MAP) toolkit has logic to facilitate SQL Server workload discovery and provide placement recommendations for your database instances onto the DBC RA. This can save time in both evaluating the type of infrastructure required for a consolidation effort as well as providing guidance in how to place workloads onto the DBC RA.

Optimized and balanced architecture

When designing a solution to support transactional based workloads it can often be a challenge to ensure there is sufficient I/O capacity in the design to meet the requirements of the workloads while also efficiently utilizing the other server and network resources. One of the significant driving factors for customers looking at SQL consolidation initiatives is that there is often a high percentage of SQL Servers in the data center that run at very low CPU utilizations; consuming energy and floor space while performing very little work.

5

The DBC RA platform has been designed from the ground up to effectively utilize all of the resources (processing, network, I/O and capacity) required by typical database consolidation workloads and has been validated to deliver a sustained 60,000 IOPS1 per rack. This provides customers with a standardized environment optimized to support their SQL Server and other database needs.

Flexible

Building on the HP Converged Infrastructure building blocks, the DBC RA provides a modular foundation for deploying Microsoft private cloud architectures. The DBC RA is built on building blocks designed for linear performance scalability, minimal-downtime, and growth ability. With HP Virtual Connect Flex-10 technology, the DBC RA utilizes a single fabric that can be configured to meet the specific requirements for virtualization. Providing the flexibility to define individual networks and allocate bandwidth to those networks according to the utilization and availability requirements, while dramatically reducing the cabling and wiring complexity and associated fabric costs.

This building block methodology enables the DBC RA to easily scale from a base half-rack configuration to a full-rack design. And as more resources are required, additional racks can be easily incorporated into the management envelope and grouped together as new resource pools.

The DBC RA also provides the flexibility to build this solution using your own in-house IT staff or engage with experienced HP consultants to customize and tailor the design to meet the demands of your business.

Simplified management

The DBC RA solution is built on HP Converged Infrastructure components and provides the infrastructure layer for deploying a Microsoft private cloud. By deploying Microsoft System Center to manage the DBC RA, customers can manage the entire lifecycle of both the DBC RA as well as the workloads running on the DBC RA solution.

With HP Insight Control for System Center (IC-SC) customers get deeper insight and monitoring control into the hardware from System Center Operations Manager (SCOM). IC-SC provides hardware alerting and event information in a single pane of view within System Center Operations Manager. In Figure 1 you can see a view of the HP BladeSystem platform within SCOM.

Figure 1. HP BladeSystem platform within SCOM

1 Based on a random workload with 8 KB blocks, 60% reads/40% writes

6

HP ProLiant Gen8 blade technology The latest generation of HP blade server technology, the HP ProLiant Generation 8 servers, features embedded automation and intelligence that cut lifecycle operations tasks, and reduce data center overhead and downtime costs. These new features and capabilities help further simplify management of the DBC RA, help lower compute costs, and increase performance. These features include HP SmartMemory, iLO 4, HP Active Health System, HP Agentless management, and HP Intelligent Provisioning features.

With HP SmartMemory technology, servers can identify and verify that system memory has passed enhanced levels of qualification and testing. This prevents a common cause of downtime and service interruption due to lower quality memory DIMMs requiring replacement. Additionally certain Gen8 servers with HP SmartMemory can run memory at lower voltages (1.35 vs. 1.5) without impacting performance. This helps lower power consumption up to 20% while still maintaining equal performance.

HP iLO has been a standard component for HP servers for many years, simplifying server setup, health monitoring and remote server management. With the release of the Gen8 server family, iLO 4 has been enhanced to provide greater control and awareness of the system health and management and is the core element enabling features such as the HP Active Health System, Agentless management and HP Intelligent Provisioning.

HP iLO 4 now includes embedded storage, replacing the traditional HP Smart Start DVD by including the support and configuration utilities directly on the servers shipping from the factory. This not only includes the common setup utilities but also includes the drivers and firmware required for installing the OS using HP Intelligent Provisioning. A simple setup wizard allows you to download updated drivers and firmware as needed and to link back to HP for additional support through HP Insight Online (described in the following section).

The HP Active Health System is a diagnostic tool which is used to monitor and record all the changes in server hardware and system configuration, changes in temperature and voltage, and alerts. It is used to diagnose problems and resolve system failures. This information can be used by HP support engineers if there is a service issue. And since the system is built on iLO, there is no impact on server performance to gather and collect this information.

Additionally, the iLO architecture enables the HP Agentless management capability which collects core management functions directly from iLO for all internal system components, including health monitoring and alerts. No configuration, drivers or agents are required to manage the servers using IC-SC inside the SCOM management console.

These Gen8 technologies coupled with the performance improvements in the Gen8 platforms, and other enhanced features such as additional thermal sensors to provide more efficient cooling instrumentation yields a platform that can simplify management and lower operational costs in both the deployment and on-going management of the DBC RA.

For more information on the HP ProLiant Gen8 technology please visit hp.com/go/proliant.

Simplified support and lifecycle maintenance

With the HP ProActive Insight architecture, HP ProLiant Gen8 servers continuously monitor more than a thousand system parameters to optimize application performance and proactively decrease downtime, while providing organizations insight into every aspect of their IT infrastructure. With this continuous monitoring along with HP Insight Online (a comprehensive cloud based management portal for viewing system health, asset, and warranty information), the DBC RA provides simplified monitoring and support capabilities.

The embedded HP Active Health System collects logs for every action performed on the server. In the event of a service issue, these logs provide valuable time savings when performing root cause analysis. And customers can log into the portal to see the service events on these systems in real-time. Additionally, as part of the HP Insight Online and iLO technology, customers can leverage the embedded remote support feature that provides Phone Home Service Support for the HP servers in the DBC RA (without any additional software installation). This feature can then be extended to provide phone home support for the entire DBC RA by installing the HP Insight Remote Support (IRS) software in your environment.

For server and storage upgrades, HP has established a set of tools to make updating driver and firmware changes to complex solutions a more streamlined and simplified process. With the HP Service Pack for ProLiant (SPP) and HP Smart Update Manager (HP SUM), customers can download tested and validated firmware/driver bundles on a periodic basis from HP and push those updates out to the components in the DBC RA in a controlled manner. This allows administrators to target updates on a server by server basis and control migration of the workloads to eliminate any workload downtime associated with planned maintenance activities. Additionally, the redundancy built into the DBC RA design provides multiple data paths for updating infrastructure components such as the network switches without needing to take the solution offline.

For more information on the HP SPP and HP SUM tools visit hp.com/go/spp.

http://www.hp.com/go/proliant

http://www.hp.com/go/spp

7

Solution architecture

The DBC RA solution is based on a modular server and storage building block design and is composed of the following two main configurations:

Base configuration

4 x HP ProLiant BL460c Gen8 Servers*, for a total of 48 cores and 1 TB RAM

2 x HP P2000 G3 storage array with external D2700 disk enclosures for 29 TB raw storage (198 spindles, 30,000 IOPS)

60 Gbps network bandwidth upstream

Full configuration

8 x HP ProLiant BL460c Gen8 Servers*, for a total of 96 cores and 2 TB RAM

4 x HP P2000 G3 storage array with external D2700 disk enclosures for 57 TB raw storage (396 spindles, 60,000 IOPS)

60 Gbps network bandwidth upstream

* Note HP ProLiant BL465c Gen8 blade servers can be substituted for the BL460c Gen8 platform if there is a specific requirement for an AMD based processor. See the bill of materials section below for details on the BL465c Gen8 blade configuration.

In addition, a Base configuration can be upgraded to a Full configuration and additional DBC RA racks can be added, to extend the resource pools for your implementation as needed.

Each of these configurations is configured as a Windows® Failover Cluster resource and has Microsoft Hyper-V enabled as the virtualization technology. This provides the infrastructure resource pool to provision hundreds of VM instances for consolidation and new VM provisioning.

The following sections describe the core DBC RA solution architecture in more detail including the hardware and software design, high availability and disaster recovery elements, performance, scalability and more.

Hardware components

The DBC RA is designed to deliver the resources needed to support a mix of typical OLTP workloads (based on Small, Medium, and Large profiles) along with similar applications that require high random IOPS levels comparable to an OLTP workload. That is, the architecture is balanced to provide sufficient processing cores, memory capacity and I/O capability to sustain a particular IOPS level, which is the key requirement for this workload.

The DBC RA is comprised of building blocks (servers and storage) that allow the solution to scale linearly between a Base version and a Full version, as shown in Figure 2. If additional resources are needed you can deploy more racks as required.

The Base version is designed to provide support for 30,000 sustained IOPS, which will accommodate 100 – 200 database instances with mixed workloads (based on HP and Microsoft research of the resulting workload demands and mixes of several large enterprise organizations that have undergone a database consolidation project). The Full version supports 60,000 sustained IOPS and 200 – 400 database instances.

8

Figure 2. Base and Full configurations of the DBC RA, which balance density with server and storage resources

Servers

The server building block consists of four HP ProLiant BL460c Gen8 blade servers deployed within an HP BladeSystem c3000 enclosure. Each blade has 12 cores, 256 GB RAM, and two 300 GB hard disk drives (HDDs). As noted above, BL465c Gen8 AMD based blade servers can also be utilized as the core compute nodes for the DBC RA.

In the DBC RA design and testing, each server is running Windows Server 2008 R2 SP1 Datacenter edition with Hyper-V enabled. With the Datacenter edition, users have the right to run an unlimited number of VMs on each server. This will be the best option when running the DBC RA given the anticipated number of SQL workloads that can be consolidated onto this platform. Note that you will still need to acquire or validate the necessary licenses for running Microsoft SQL Server in the VMs which will vary based on existing customer licensing agreements.

9

Along with the Hyper-V role, failover clustering is also enabled on each server. A DBC RA rack is configured as either a 4-node (base) or 8-node (full) cluster configuration. Each additional rack then becomes its own Failover Cluster resource that can be managed as additional resources with the System Center Virtual Machine Manager environment.

Storage

Storage is also designed in a building block fashion for the DBC RA solution. Each core storage building block in the DBC RA consists of the following components:

Single P2000 10 Gb iSCSI storage system with 24 spindles, selected to maximize the performance and disk count of each storage block

Three HP D2700 Disk Enclosures, each with 25 spindles, attached to the P2000 storage

Total of 99 146 GB 15,000 rpm small form factor (SFF) HDDs

Approximately 15,000 IOPS2

Note Larger capacity drives can be selected if there are additional storage capacity requirements. However this may lower the total IOPS capacity of each storage block if the larger drives run at 10K RPM versus 15K RPM.

For a base configuration there are two storage blocks and for a full configuration there are four storage blocks. The storage fabric chosen in the DBC RA design is 10Gb iSCSI. This specifically provides capability for configuring guest clusters between two VMs (requiring direct access to LUNs on the storage array). This capability enables a higher degree of availability and is described in further detail in the “Guest clustering for workload mobility” section below.

To achieve the published storage performance numbers for this DBC RA design requires strict adherence to the design and configuration of the storage subsystem. Table 1 below provides the detail for the DBC RA storage block design.

Table 1. P2000 storage block VDisk and Volume configuration

VDisk Name Drive Count RAID level Sub-VDisk Count Volume Name(s) Volume Size

VDisk #1 16 10 8 CSV1

Quorum*

1150 GB**

1GB

VDisk #2 16 10 8 CSV2 1150 GB**

VDisk #3 16 10 8 CSV3 1150 GB**

VDisk #4 16 10 8 CSV4 1150 GB**

VDisk #5 16 10 8 CSV5 1150 GB**

VDisk #6 14 10 7 Avail*** N/A

*The Quorum drive is only configured for the first storage block, array #1 and is used for the host Windows Failover Cluster quorum drive.

**The capacity of the Cluster Shared Volume s(CSVs) are based on the 146GB, 15K RPM drive and will be larger if larger drives are selected for the DBC RA.

***VDisk6 provides available capacity for supporting guest clustering requirements and any System Center management stack requirements.

Further information on the specific storage design is described in the “Hardware configuration” section below.

2 Based on a random workload with 8 KB blocks, 60% reads/40% writes

10

Integrated storage is an important factor in the DBC RA model, rather than an external SAN for the following reasons:

Less risk

Shorter time-to-value

Predictable performance

Ease of configuration

However, customers can extend the DBC RA capability by connecting to existing SAN environments for additional storage capacity if needed.

Networking

When designing Hyper-V host servers, there are a number of distinct networks that are required for the solution to be in accordance with accepted best practices for building a Microsoft private cloud.

Thus, the following networking design is recommended for each server in the DBC RA:

Teamed production networks

Teamed backup networks

Teamed management networks

Live VM migration network

Cluster Shared Volume (CSV) network

Redundant iSCSI networks for Multipath I/O (MPIO)

To implement this network configuration requires 10 individual network connections, and for ideal optimization, an ability to support a range of network bandwidths.

This networking configuration has been simplified in the DBC RA by the use of HP Virtual Connect Technology, which makes it easy to create the desired number of networks and allocate the appropriate bandwidth to each. Each server in the DBC RA is configured with two dual-port HP Virtual Connect FlexFabric Adapters that provide an aggregate bandwidth of 40 Gb. Each port can be carved into four FlexNICs3, for a total of 16 FlexNICs per server. Each FlexNIC is allocated a share of the available bandwidth according to its specific network role.

The benefits of this design include:

Ability to carve up individual networks and allocate bandwidth optimally for each role, reducing the number of required switches and physical network cables from each server dramatically

Wire-once connectivity internally within the DBC RA and externally to the customer’s upstream network

Convergence of iSCSI storage traffic and network traffic within the DBC RA

Connectivity options There are several options for connecting the system to a customer’s environment or linking multiple racks together. There are three available uplinks on each Flex-10 module in the c3000 enclosure for connecting to the upstream network. These can operate at either 1Gbs or 10Gbs speeds depending on the customer’s requirements and can be aggregated together to provide higher uplink speeds. The recommendation is to utilize 10Gbs uplinks to ensure that the uplink pipes do not become a source of network congestion.

There are six transceiver options for connectivity:

1Gb Copper

1Gb SX Fibre

10Gb SR Fibre

10Gb LR Fibre

10Gb LRM Fibre

10Gb Copper using DAC cables

3 The system treats FlexNICs in the same way as physical NICs.

11

Each Flex-10 module has 8 uplink ports. One port on each module will be used for the management network, two ports (Ports 7 and 8) provide redundant crosslink connections between the two Flex-10 modules, and two ports on each module provide connectivity to the 10Gb iSCSI storage fabric. This leaves a remainder of 3 uplink ports on each module, or 6 ports aggregate across both modules for uplink connectivity. The recommendation is to utilize four of the uplink ports for the production network traffic and dedicate two of the uplink ports for the backup network. This provides an aggregate bandwidth of 60 Gbps; 40 Gbps for production and 20 Gbps for backup traffic if utilizing the base network design.

Connectivity within the DBC RA is achieved via the following redundant switches:

HP 2910 1 Gb switches – Management traffic

The 1 Gb Top of Rack (TOR) switches are intended to be utilized for the internal management network. If there are multiple networks in your environment, management of these systems can be consolidated across the 1 Gb management fabric.

HP 6600 10 Gb switches – iSCSI storage fabric

Each of the P2000 G3 iSCSI storage arrays has redundant connections to these switches, and the switches themselves have redundant connections to the Virtual Connect Flex-10 modules in the c3000 enclosure. This forms a completely redundant, isolated 10 Gb storage fabric.

For additional details on the wiring and connectivity options please see the cabling and wiring guides for the DBC RA in the Appendix.

Power

The DBC RA utilizes the HP Intelligent Power Distribution Units (iPDU) which bring an increased level of precision, control, and automation to power distribution. With the HP iPDUs, operations staff have the ability to get exact power consumption at a core, stick or outlet level and get precise reporting of the power utilization and consumption requirements of the HP DBC RA environment. Additionally the HP iPDUs can send out SNMP based alerts which allow monitoring of the power consumption for the DBC RA from the SCOM console or from an HP Systems Insight Manager environment for the data center.

For fault tolerance, each DBC RA rack includes four HP iPDUs that are connected to the customer’s utility supply via multiple feeds. This configuration is designed to sustain the loss of a single power feed in the data center. Figure 3 below shows the overview page for one side (two PDUs) of the DBC RA.

12

Figure 3. Overview Page for HP iPDU Primary and Secondary

This page provides a graphical representation of the total load on each iPDU segment and details into the load on each segment (each iPDU core has 6 segments). Additional detail at the outlet level can be displayed by clicking on the status page link as shown in Figure 4.

13

Figure 4. Status Page for HP iPDUs

The Status page displays the status of the iPDU by load segment. In addition to the Current, Load, and Voltage displayed on the Overview page, the Status page shows the Power, Watts, and Power Factor, down to the Outlet level. This allows the user to see exactly how much power is being used by a particular device.

In the DBC RA design (and associated bill of materials below) two iPDU options are highlighted. However, it is important to select the appropriate PDU option based on the regional and data center power requirements. For example, the appropriate international replacement iPDU is the HP 7.3kVA 32A Single Phase INTL Core Intelligent Modular Power Distribution Unit.

Software components

The DBC RA provides the underlying infrastructure for running both SQL Server and other workloads requiring high levels of performance and availability. To complete the solution, a software stack that can deliver the management, monitoring, operational control, and lifecycle requirements needed to run a Microsoft private cloud infrastructure is required. To deliver these capabilities System Center 2012 with HP Insight Control for System Center can be installed on the DBC RA.

System Center 2012 is a comprehensive management platform consisting of capabilities for infrastructure management, service delivery and automation, and application management and control. The System Center 2012 product includes a number of core components that can be installed including the following:

App Controller (AC) – Provides aggregated self-service portal capabilities for managing and provisioning resources to internal private cloud infrastructures managed by System Center Virtual Machine Manager and external resources hosted in SQL Azure.

Operations Manager (SCOM) – Core monitoring component for System Center to monitor services, devices, and operations through a single console. Provides a framework to install management packs to manage and monitor different components of the environment such as SQL Server. Additionally, there are management packs from HP as part of the Insight Control suite (described below) that provide alerting and monitoring capabilities for the underlying HP components.

Orchestrator (ORC) – Workflow management tool which enables Runbook automation tasks to be developed to automate various processes and operations by IT. Orchestrator can execute standalone Runbook tasks or be called by other System Center components such as Service Manager to automate workflow operations for provisioning a service, creating a new VM, and various other tasks.

14

Service Manager (SM) – Provides automated service and support management for System Center including a new service level self-service portal that can be integrated with existing IT systems for chargeback and billing, ticketing, and change management tasks. Service Manager is integrated with other System Center components (Orchestrator, SCOM, and System Center Virtual Machine Manager) to enable fully automated provisioning and execution of common IT tasks in an end-to-end workflow operation.

Virtual Machine Manager (SCVMM) – Manages the physical infrastructure resources including server, storage, and network that make up your private cloud resource pool. Provides fabric level configuration of the network and leverages SMI-S to communicate with and configure storage components directly.

Data Protection Manager (DPM) – Backup and recovery with continuous data protection capability

Endpoint Protection (EP) – Anti-malware and security capabilities for endpoint PCs. Can be integrated with System Center Configuration Manager to deploy security clients and configuration settings for Windows Firewall.

Configuration Manager (SCCM) – Performs hardware and software baseline inventory and can be utilized to deploy operating systems, software applications, and firmware/driver updates.

The System Center 2012 management stack can be installed on a core set of VMs running on the DBC RA infrastructure, through which the services are delivered to the business users. Alternatively, the management stack can be installed outside the DBC RA on physical servers or in another VM environment.

Depending on the desired capabilities for the DBC RA the specific components that are installed and utilized may vary. For the DBC RA, the following configuration is recommended:

Highly Available VMs:

– SCVMM

– SCOM

– SCOM Reporting Server

– Service Manager Mgt Server

– Service Manager Portal Server

– Orchestrator

– App Controller

Guest Clustered SQL Server VMs:

– SQL instances

o SCVMM DB instance

o SCOM DB instance

o SCOM DW instance

o SC SM DB instance

o SC SM DW instance

o Orchestrator DB instance

o App Controller DB instance

With these components the DBC RA provides a management environment with integrated self-service portal capabilities, service provisioning and lifecycle management, and charge-back capabilities on top of the core private cloud infrastructure management and monitoring capability provided by SCVMM and SCOM.

Beyond the components shown above, there are a number of other components that can be installed to provide additional capabilities for the management platform including:

Windows Deployment Services (WDS) /PXE Server

Windows Server Update Services (WSUS)

DPM Server

EP Server

SCCM Server

15

For more information on Microsoft System Center 2012 please see microsoft.com/systemcenter.

HP Insight Control for Microsoft System Center

HP Insight Control for Microsoft System Center provides essential infrastructure management, making it easy to deploy, migrate, monitor, control, and optimize an IT infrastructure from a single, simple management console. Within the DBC RA, HP Insight Control for Microsoft System Center seamlessly integrates unique HP ProLiant and HP BladeSystem manageability features into System Center Operations Manager, exposing the native capabilities of servers and enclosures to System Center. This increased visibility makes it easier to perform root cause analysis to subsystem and component levels.

For the DBC RA, installation of the HP BladeSystem and ProLiant Management Packs for SCOM provides the following capabilities:

Proactive hardware monitoring, managing and alerting of hardware health and intelligently responding to hardware events on servers running Windows.

The HP ProLiant Agentless Management Pack manages the health of ProLiant Gen8 servers without the need for loading OS-based SNMP agents or WBEM providers.

HP SCVMM 2012 Integration Kit provides HP WinPE and production drivers for ProLiant servers to assist with OS deployment via SCVMM 2012.

The HP ProLiant Updates Catalog can be used by SCVMM 2012 to provide simplified Windows driver and firmware updates via a rotating, automated workflow for Hyper-V clusters.

There is a single license that covers all the Insight Control functionality including IC-SC, which is included in the bill of materials section below.

In addition the HP Storage Management Pack for System Center provides seamless integration with SCOM 2012 by integrating predefined discovery and state monitoring policies, event processing rules and tasks. This comprehensive integration solution complements HP Insight Control for Microsoft System Center, and allows administrators to proactively streamline IT operations and increase systems availability by monitoring ProLiant server environments and HP Storage products through a common console.

Guest workload support

One of the key elements of a virtualization strategy for SQL consolidation is that by virtualizing at an OS level, you are free to mix and match different OS and SQL versions as necessary for the applications running in your environment. This platform will support any version of SQL Server that is supported inside a Hyper-V environment. This ranges from SQL Server 2000 up to the recently released SQL Server 2012, giving administrators the utmost levels of flexibility and control in consolidation and new SQL Server deployments.

Built-in high availability

The DBC RA has been designed to provide extremely high availability at all levels – from the underlying network and storage fabrics to the VM layer. Thus, the DBC RA can tolerate the failure of any active component, ensuring there is minimal impact to database workloads in the event of planned or unplanned downtime. High availability can be extended to the solution level through the live migration of active workloads between VMs, with zero downtime.

In the hardware design, there is full redundancy for switches and PDUs, while server components such as power supplies, fans, and storage are hot swappable. The DBC RA also provides a fully redundant network stack, with multiple switches for internal storage and management traffic; at the server level, NIC teaming and MPIO can be used to maximize throughput and availability. High availability between the DBC RA and your upstream network infrastructure is achieved through Link Aggregation Control Protocol (LACP) bindings in conjunction with redundant Virtual Connect Flex-10 modules.

In addition, host clustering (requirement) and guest clustering (optional) can then be leveraged to enhance availability, protecting VMs or specific workloads from planned maintenance and unexpected hardware failures, while enabling load balancing. More information on clustering follows.

http://www.microsoft.com/systemcenter

16

Host clustering for VM mobility

To enhance the availability of VMs, virtualization hosts in a DBC RA should be configured through Windows Failover Clustering as a four-node (Base version) or eight-node (Extended version) cluster.

Installed in the parent partition of each virtualization host, the cluster service allows VMs to move from server to server (though not from one cluster to another in a multi-rack DBC RA configuration). For example, if downtime is planned, a VM can be moved seamlessly from one node to another server via the Live Migration feature of SCVMM; alternatively, if the downtime is unplanned, VMs undergo a fast restart on a surviving server.

Use cases for host clustering include:

Zero-downtime host patching: Supporting hardware changes or software updates to the parent partition

Load balancing: Migrating a VM to a different server when the original server’s resources are being saturated

Guest clustering for workload mobility

Optionally, you can create guest clusters to provide higher levels of availability for specific workloads within a DBC RA configuration. In this scenario, the cluster service runs within VMs in a two-node cluster and manages the movement of a mission-critical workload from one VM to the other, as required.

Use cases for guest clustering include:

Zero-downtime VM patching: Supporting updates to the OS or application running in the VM

Automatic recovery: Failing over a database instance from one VM to another in the event of a failure

Guest clustering is a powerful option that is most useful for protecting mission-critical workloads. To support VM level guest clustering an iSCSI storage fabric is part of the core DBC RA design.

Disaster recovery

The DBC RA offers multiple levels of hardware high-availability designed to provide protection in the event of a component or even node failure. However, to minimize the risk of the devastating losses that can result from the failure of an entire site, you can develop a disaster recovery (DR) solution designed to protect the resources you consider critical – from as little as a few VMs to the entire DBC RA workloads – by replicating these resources at a second, geographically-separated location.

The DBC RA can be aligned with typical corporate DR plans by combining built-in technologies with proven DR methodologies. DPM can be used in conjunction with native database technologies and third-party solutions such as SteelEye DataKeeper to provide a range of DR options.

For example, through its backup and restore capabilities, DPM can provide basic DR for less critical workloads. At its simplest, a DPM server at the primary site ensures that all workloads are backed up and can then be replicated to the secondary site; in the event of a disaster, the secondary DPM server is used to restore servers and workloads, and, subsequently, rebuild the primary DPM server.

For more critical workloads, native database features can typically be utilized to provide a significantly faster time-to-recovery, while minimizing the potential for data loss. Proven, high availability technologies like transaction log shipping and database mirroring can be used in the same way as in a physical database implementation, ensuring no extra effort is required to implement a robust DR strategy. With log shipping, transaction logs are backed up at the primary site and then copied to the secondary, where they are applied after a specified delay; in the event of a site failure, the warm copy4 at the secondary site can be manually restored. With mirroring5, you maintain two copies of the same database, one at the primary site and a hot copy at the secondary site. All insert, update, and delete operations are streamed to the secondary site, where they are applied as quickly as possible. In the event of a site failure, the secondary site can be brought online without data loss; clients can recover quickly by reconnecting to the mirrored instance. In addition, the latest SQL Server release – SQL Server 2012 – provides AlwaysOn capabilities and an enhanced database mirroring solution.

If you need the automatic reconnection of applications to databases following a failover, you should consider a solution6 such as DataKeeper, which is based on a Microsoft failover cluster, with one node in the primary DBC RA and a second at a different location. The key feature in this scenario is the ability of DataKeeper to take two separate (non-shared) disk systems, mirror these systems using disk-level replication, and create a shared disk resource. Since DataKeeper is able

4 Restoring a warm copy may cause some data loss; restoring a hot copy causes little or no data loss. 5 Mirroring is achieved at the level of physical log records, while replication is achieved at the logical level. 6 Requires the database instance to support Microsoft failover clustering

http://us.sios.com/products/steeleye-datakeeper-windows/

17

to keep the databases in sync, a lights-out failure at the primary location results in the database VM failing over to the secondary location and continuing to run. Following the failover, the database still remains within the same cluster – though on a different node; thus, the application is unaware of the failover event. With this automatic reconnection capability, DataKeeper can be used to provide the highest levels of protection.

Note that the secondary node in a DataKeeper cluster does not have to be a VM. For example, rather than deploying a DBC RA at the secondary site, you could utilize a large server such as an HP ProLiant DL980 to run a virtualized or non-virtualized database instance.

Performance and scalability

The DBC RA is designed to satisfy both scale-up (within a rack) and scale-out (multiple racks) requirements in a linear fashion. The base (half-rack) configuration can deliver 30,000 sustained IOPS. As noted earlier this IOPS measurement is based on a random 60/40 R/W workload with 8KB block sizes. As more resources are required, the base configuration can be scaled-up to a full-rack by adding the necessary infrastructure and growing the cluster from 4 to 8 nodes. The full-rack configuration can then deliver a sustained 60,000 IOPS. As resource requirements grow beyond 60,000 IOPS, additional racks can be added achieving linear scalability for each new resource pool.

These IOPS levels can be achieved with any database that meets the following criteria:

The database instance can be virtualized using Windows Server 2008 R2 Hyper-V

The workload matches one of the VM profiles used as key design criteria for the DBC RA (see below)

Note Though the DBC RA was originally optimized for SQL Server, it can support various other I/O-intensive workloads and applications.

VM profiling and sizing recommendations

Unlike traditional application sizing exercises, there is no single workload that can be used to represent all of the various permutations that will exist from customer to customer. Some customers will require hundreds and hundreds of small VMs with very low utilization rates. While other customers may require fewer, more heavily used VMs to support their specific OLTP workload requirements.

For sizing and modeling analysis there were three VM models created – small, medium, and large – as outlined in Table 2.

Table 2. VM modeling for small, medium, and large workloads

Workload Virtual CPUs (vCPUs)

RAM Storage I/O operations per second (IOPS)

Small 1 4 GB 100 GB 50 IOPS

Medium 2 8 GB 200 GB 400 IOPS

Large 4 16 GB 400 GB 1,600 IOPS

For example, a Base (half-rack) configuration is designed to support between 100 - 200 mixed workloads using different ratios of small, medium, and large VMs while still reserving sufficient resources to accommodate growth and recover from the loss of at least one server for HA requirements. The Full (full-rack) RA configuration is estimated to support between 200 - 400 mixed workloads. As with any sizing exercise, these numbers will vary greatly depending on your specific mix and workloads running in the VMs.

There have been a number of stress and modeling tests in order to characterize the number of databases that can be supported on a DBC RA configuration. Results show conclusively that IOPS values scale linearly as the number of workloads increases; for example, with Small workloads, the Base configuration was able to support as many as 432 VMs and the Full configuration as many as 865 VMs.

18

However, this testing did not take into account the following characteristics which would be fundamental to consider in a production deployment:

Support a mix of small, medium, and large workloads

Growth capacity

Support for the failover of one – and, in some circumstances, two – servers

Thus, the recommended VM densities are as follows:

Base configuration: A mix of 100 – 200 small, medium, and large VMs

Full configuration: A mix of 200 – 400 small, medium, and large VMs

Storage performance

To demonstrate the I/O capabilities of the DBC RA a SQL Server workload was applied to a full configuration architecture using medium and large profile SQL Server VMs, as outlined above in Table 2. In order to utilize all of the available disks, the available space on VDisk #6, as outlined in Table 1, was used on storage blocks 2, 3, and 4 to create three additional CSV volumes. This increased the total CSV volume count to 23.

Table 3 below outlines CSV ownership by blade server and the number of large and medium SQL Server VMs created.

Table 3. Blade Server CSV and VM Allocation

Blade server CSV volumes Large VMs Medium VMs

1 1 – 2 3 6

2 3 – 5 4 9

3 6 – 8 4 9

4 9 – 10, 21 4 9

5 11 – 13 4 9

6 14 – 15, 22 4 9

7 16 – 18 4 9

8 19 – 20, 23 4 9

Total 31 69

This heavy workload showed the DBC RA’s ability to sustain an I/O requirement of over 60,000 IOPS with a 20 ms response time. The total disk transfers/sec. counters were recorded for each of the eight blade servers in Table 4.

Table 4. Total Transfers/sec.

Blade server Total Disk Transfers/sec.

CSV Volumes

1 6,408 2

2 7,691 3

3 7,804 3

19

Blade server Total Disk Transfers/sec.

CSV Volumes

4 7,655 3

5 7,602 3

6 8,095 3

7 8,248 3

8 7,821 3

Total 61,324 23

And in a true deployment scenario, this ratio would also include a larger percentage of small and lightly used VMs, significantly increasing the total number of VMs running on the DBC RA.

Power consumption under load

One of the key benefits listed above is that the HP iPDUs provide advanced power consumption and reporting; including monitoring of power draw at the core, load segment, stick, and outlet level. This gives administrators the data needed to understand exactly how the power draw is balanced in the DBC RA and if there are any areas of the architecture which are operating at a non-optimal level or are unbalanced.

Using the SNMP management capabilities on the HP iPDUs a PowerShell script was used to collect the power draw from each iPDU at 15 second intervals. A comparison of the DBC RA at idle (no workload but all devices powered on) with the DBC RA under heavy workload showed a very reasonable power consumption increase of only 14%. Figure 5 compares the power consumption at idle vs. heavy load using the total power draw for all four iPDUs.

Figure 5. Power Consumption Idle vs. Heavy Load

20

Figure 6 displays the Overview page for the HP iPDUs while under the load described above. This is a view of the primary and secondary iPDUs on one side of the rack, exhibiting approximately half of the total rack power draw under this load.

Figure 6. Overview Page for iPDUs under heavy load

For fault tolerance, each DBC RA rack includes four HP iPDUs that are connected to the customer’s utility supply via multiple feeds. This configuration is designed to sustain the loss of a single power feed in the data center. Under the heavy workload one of the power feeds supplying two of the iPDUs on one side of the rack were unplugged to demonstrate the high availability of the DBC RA. Figure 7 below depicts the Overview page displaying the total load percentage for the remaining 2 iPDUs, along with the associated active critical alarm notification from the loss of the power feed.

21

Figure 7. HP iPDU Overview Page after one utility feed is disconnected

The screenshots show an increase of the primary PDU load percentage from 19.66% (Figure 6) to 31.79% (Figure 7) and the secondary PDU load percentage from 21.49% to 42.89% as these iPDUs replace the power lost from the disconnected iPDUs. The “Active Critical Alarm” is triggered from the loss of communication with the disconnected iPDUs as shown in Figures 8 and 9.

Figure 8. HP iPDU Alarms Page after one utility feed is disconnected

22

The event log shows the loss of communication with the primary HP iPDU at IP address 172.16.2.45 on the disconnected power feed.

Figure 9. HP iPDU Event Log after one utility feed is disconnected

Ordering and purchasing the DBC Reference Architecture

The intent of the DBC RA is to provide a pretested and validated hardware infrastructure that can support consolidation of high I/O workloads like SQL Server as well as provide the foundational infrastructure for configuring a Microsoft private cloud. At a high-level, there are three phases that need to occur when ordering a DBC RA platform: Assessment and planning, Purchasing and build, and On-site deployment.

Assessment and planning

To help you consolidate your particular workloads on to the DBC RA, Microsoft has customized the Microsoft Assessment and Planning (MAP) Toolkit, which is freely available for download at no charge7. The MAP 7.0 toolkit is aware of the specific DBC RA hardware configuration and can provide intelligent recommendations to help in capacity planning based on this architecture.

The tool begins by performing an inventory of your existing physical database servers and then collects performance metrics for each server in order to categorize its workload. After a suitable collection period, the tool is able to assess the following:

Which database instances are suitable candidates for consolidation on a DBC RA (that is, which database workloads can be categorized as x-Small, Small, Medium, Large, or x-Large)?

How many servers are required to host the virtualized database instances? This determination requires a sophisticated set of checks and heuristics developed by HP and Microsoft engineers based on years of real-world experience with database implementations.

Which server would be the ideal host for a particular VM? Because of its ability to match the requirements of individual VMs with the resources available on each server, the MAP toolkit can recommend an ideal host for each VM.

How many racks are required to support the required number of host servers?

Figure 10 shows an example output of the MAP 7.0 toolkit after the inventory and analysis stage has been completed using the Microsoft Database Consolidation Appliance mode which has knowledge of the core hardware infrastructure in the DBC RA.

7 http://technet.microsoft.com/en-us/solutionaccelerators/dd537566.aspx

http://technet.microsoft.com/en-us/solutionaccelerators/dd537566.aspx

23

Figure 10. MAP toolkit output

The MAP tool has provided a recommendation for the number of hosts (based on the Gen8 HP blade servers) required to consolidate the SQL instances that have been discovered and properly inventoried. As you can see in Figure 11 below, the tool also provides information on how the existing SQL workloads fit into the five workload modeling categories. Note that reports on this information can also be generated in a more usable spreadsheet format.

Figure 11. MAP toolkit output

24

For more information and to download the MAP 7.0 toolkit and perform your assessment and analysis today go to http://technet.microsoft.com/en-us/solutionaccelerators/dd537566.aspx. Additionally, HP Technical Services Consulting offers a SQL Assessment and Planning service to facilitate this activity and help you through the SQL migration and planning phases.

Purchasing and build

After completing the assessment phase and gaining a better understanding of the workloads running in your environment, the next step would be purchasing the required number of DBC RA racks to support your workload requirements. Listed in the “DBC Reference Architecture bill of materials” section of this document is a complete list of all the required components for both a base and full DBC RA configuration.

The DBC RA can be ordered as a collection of parts and assembled at the customer site. However, it is highly recommended that the DBC RA be ordered along with Factory Express level 4 or Factory Express level 5 integration. With Factory Express services, the components of the DBC RA will be fully integrated (racked/wired) in the factory. This can include racking of the hardware components and wiring of the network and storage fabric leveraging the additional material included in the appendix of this RA. By incorporating Factory Express into the order, the DBC RA will arrive on-site and ready to configure. For more information on HP Factory Express please visit hp.com/go/factoryexpress.

Since this is a reference architecture design, you do have choice and flexibility in how to order and customize this platform. Common changes to the RA design include moving to larger disk capacities, choosing between AMD-based or Intel®-based servers, and increasing memory capacity. However, it is always important to keep in mind that these changes will change the performance and resource footprint for the configuration.

Deploying the DBC Reference Architecture

The DBC RA is designed to provide a pre-configured and validated infrastructure platform for SQL consolidation and private cloud deployment based on best practices and recommendations for building a Microsoft private cloud. However, HP recognizes that each customer has unique configuration and setup requirements to integrate and deploy a solution of this caliber in the data center. For organizations with significant in-house technical experience this on-site setup procedure can be performed by themselves. As an alternative, HP Technical Services Consulting (as well as other third party consulting organizations) can deliver services tailored and customized to a customer’s specific deployment requirements for the DBC RA platform.

The following sections provide a high-level overview of the key steps required to build out the DBC RA platform once the solution has been racked and wired together (either on-site or by leveraging Factory Express services). Each deployment may vary depending on specific customer implementation choices and so should be evaluated accordingly.

Hardware configuration

The first stage of setup for the DBC RA infrastructure is to configure the following hardware components.

HP Network switches

The following high-level setup steps should be performed to setup the network switches.

6600-24G and 2910-24G 1. Set switch IP address information.

2. Configure switch passwords.

3. Set switch name, time, and time zone.

4. Enable spanning tree.

6600-24G only 1. Configure two Link Aggregation Control Protocol (LACP) Trunks; one on ports 1 and 2, and one on ports 23 and

24. These will form the trunks for the crossover connection and between the two VC modules

2. Enable jumbo frames support on the storage fabric.

http://technet.microsoft.com/en-us/solutionaccelerators/dd537566.aspx

http://www.hp.com/go/factoryexpress

25

HP Onboard Administrator

The following high-level setup steps should be performed to setup the HP Onboard Administrators (OA):

1. Set OA IP address information.

2. Configure OA users and passwords.

3. Set enclosure name and information.

4. Set power mode.

5. Setup enclosure bay IP addressing (EBIPA) (optional):

a. This will preconfigure IP addresses on the iLO and VC modules if DHCP is not enabled in the network.

HP Virtual Connect

The Virtual Connect setup will depend on the exact wiring and configuration of the network design for the deployment. The following steps provide an example driven by the base DBC RA design

1. Import enclosure and create VC domain.

2. Set VC users and passwords.

3. Configure global VC settings such as VC or factory assigned MAC addresses.

4. Add VC networks:

a. Enable Smart Link for each of the networks.

b. Enable VLAN tunnel mode for each network or Mapped mode as appropriate for the specific configuration.

c. For the base DBC RA design, define the following VC networks with the associated details:

i. Mgt-A

1. Uplink – Bay1 x1

ii. Mgt-B

1. Uplink – Bay2 x2

iii. iSCSI-A

1. Uplinks – Bay1 x2, x3

iv. iSCSI-B


v. Production-A


vi. Production-B


vii. CSV

1. No Uplink

viii. LiveMigration

1. No Uplink

ix. Backup-A

1. Uplink - Bay1 X6

x. Backup-B

1. Uplink – Bay2 X6

5. Create 4 or 8 Server Profiles accordingly

a. Each server profile should be configured as shown in Figure 12

26

Figure 12. VC Server Profile

Note that this is showing the mapping of each network to the VC FlexNIC adapters on each blade in the Mapping column. You can see that in this layout, each of the teamed adapters uses a FlexNIC from the on-board (LOM) and mezzanine (MEZZ) cards for optimal HA.

HP P2000 G3 storage arrays

Achieving the published storage performance numbers for this DBC RA design requires strict adherence to the design and configuration of the storage subsystem. As described above, for a base DBC RA configuration there are two storage blocks and for a full DBC RA configuration there are four blocks. At this stage, all of the blocks can be configured identically.

Each storage block consists of 99 drives as shown in the following table.

Table 5. P2000 storage block VDisk and Volume configuration

VDisk Name Drive Count RAID level Sub-VDisk Count Volume Name(s) Volume Size

VDisk #1 16 10 8 CSV1

Quorum*

1150 GB**

1GB

VDisk #2 16 10 8 CSV2 1150 GB**

VDisk #3 16 10 8 CSV3 1150 GB**

VDisk #4 16 10 8 CSV4 1150 GB**

VDisk #5 16 10 8 CSV5 1150 GB**

VDisk #6 14 10 7 Avail*** N/A

*The Quorum drive is only configured for the first storage block, array #1 and is used for the host Windows Failover Cluster quorum drive.

**The capacity of the CSVs are based on the 146GB, 15K RPM drive and will be larger if larger drives are selected for the DBC RA.

***VDisk6 provides available capacity for supporting guest clustering requirements and any System Center management stack requirements.

27

In each storage block there are five global spares that are allocated to handle drive failures without a loss of service. Additionally, to ensure that the loss of a D2700 drive enclosure does not impact availability each Sub-VDisk mirror pair is created using a drive from a different drive enclosure. The following figures (Figures 13-18) show how the drives should be laid out for each VDisk on the storage blocks.

Figure 13. VDisk1 drive layout

28



29



30


To setup a storage block based on the layout shown above, perform the following steps:

1. Login to the HP Storage Management Utility (SMU) homepage.

2. Setup users and passwords.

3. Assign appropriate IP addresses to network and host interfaces if DHCP is not being used.

4. Configure Jumbo Frames on the Host Interfaces.

5. Set system name and time.

6. Create VDisks – VDisk1 to VDisk6 as described and depicted above.

a. The specific drive configurations depicted above are also called out here for each VDisk. The format is X.Y where X is the enclosure number and Y is the drive bay:

i. VDisk1 – 8 sub-VDisk mirror sets with the following drive pairings:

1. 1.1, 2.1

2. 3.1, 4.1

3. 1.2, 2.2

4. 3.2, 4.2

5. 1.3, 2.3

6. 3.3, 4.3

7. 1.4, 2.4

8. 3.4, 4.4

31

ii. VDisk2 – 8 sub-VDisk mirror sets with the following drive pairings:

1. 1.5, 2.5

2. 3.5, 4.5

3. 1.6, 2.6

4. 3.6, 4.6

5. 1.7, 2.7

6. 3.7, 4.7

7. 1.8, 2.8

8. 3.8, 4.8

iii. VDisk3 - 8 sub-VDisk mirror sets with the following drive pairings:

1. 1.9, 2.9

2. 3.9, 4.9

3. 1.10, 2.10

4. 3.10, 4.10

5. 1.11, 2.11

6. 3.11, 4.11

7. 1.12, 2.12

8. 3.12, 4.12

iv. VDisk4 - 8 sub-VDisk mirror sets with the following drive pairings:

1. 1.13, 2.13

2. 3.13, 4.13

3. 1.14, 2.14

4. 3.14, 4.14

5. 1.15, 2.15

6. 3.15, 4.15

7. 1.16, 2.16

8. 3.16, 4.16

v. VDisk5 - 8 sub-VDisk mirror sets with the following drive pairings:

1. 1.17, 2.17

2. 3.17, 4.17

3. 1.18, 2.18

4. 3.18, 4.18

5. 1.19, 2.19

6. 3.19, 4.19

7. 1.20, 2.20

8. 3.20, 4.20

vi. VDisk6 - 7 sub-VDisk mirror sets with the following drive pairings:

1. 1.21, 2.21

2. 3.21, 4.21

3. 1.22, 2.22

32

4. 3.22, 4.22

5. 1.23, 2.23

6. 3.23, 4.23

7. 3.24, 4.24

b. To create a VDisk select the Provisioning -> CreateVDisk drop-down menu option.

i. Set the name, RAID10 level, and sub-VDisk count.

ii. Select the appropriate drive pairings for the VDisk.

7. Once the VDisks have been created, create the CSV volumes as outline in the table above.

a. For VDisks 1-5, there will be a single CSV volume, VDisk6 is left unallocated at this time.

b. To create a volume, click on the appropriate VDisk and select Provisioning -> Create Volume from the drop-down menu.

i. Set the name and size as described above.

ii. Uncheck the option to Map the volumes at this time.

iii. Select Apply.

c. Repeat this procedure to create all of the necessary volumes for the storage block.

8. Assign global spares.

a. There will be five global spares assigned to the following drives (using the same X.Y format as above).

i. 1.24, 2.24, 2.25, 3.35, 4.25

b. To assign a global spare select Provisioning -> Manage Global Spares.

c. Assign the appropriate drives as global spares, the only remaining available drives should be the five allocated as spares.

9. Repeat steps 1-8 for each storage block in the base or full DBC RA configuration

Host OS and configuration steps before System Center installation

Once the hardware is setup and configured, the next stage is to setup the host servers and configure the Windows Failover Cluster as follows:

1. Provision base OS image.

2. Install Hyper-V and Failover Cluster role.

3. Install the latest HP Service Pack for ProLiant (SPP) package to install the latest drivers and firmware for the blade server – hp.com/go/SPP.

a. Note – If the SPP is applied before the Hyper-V role is enabled then the Network Configuration Utility (NCU) must not be installed. The NCU must only be installed after the Hyper-V role is enabled.

4. Enable MPIO capability for the P2000 G3 arrays.

a. To enable MPIO for the P2000 arrays in Windows 2008 R2, open up a cmd prompt and run the following cmd:

i. mpclaim -r -i -d "HP P2"

5. Configure the networking names to match the VC profiles. This will facilitate the process of assigning static IP addresses and setting up the appropriate network teams.

a. By default each of the network adapters will have a name “Local Area Connection N” with N being a numerical value. To understand which network adapter is associated with a specific VC network requires mapping the MAC address from the adapter to the VC profile information.

b. Open up the VC management interface and click on the “Edit Profile” option for a specific server profile. Figure 12 above shows an example of the server profile for bay 1.

http://www.hp.com/go/SPP

33

c. For each network in the server profile there is a corresponding MAC address in the MAC column. Find the corresponding network on the OS and rename the network to match the VC profile Network Name value.

6. Disable the networks that are not utilized in the VC profile. By default, there is a corresponding adapter for each of the 16 VC FlexNIC adapters. In the base design, only 10 adapters have networks assigned. Once all 10 of these networks have been renamed, disable the remaining 6 adapters.

7. Create network teams for the appropriate networks. In the base design, there are three networks which require teaming: management, production, and backup networks. In step 5 the networks were renamed to simplify this step.

a. Open up the NCU tool by double-clicking on the HP NCU icon in the bottom-right taskbar

b. In the main HP NICs window you will see all 10 of the available NICs to team together. Create the following 3 teams with the appropriately named adapter as shown in the list and Figure 19 below:

1. Mgt-Team

2. Mgt-A network

3. Mgt-B network

ii. Production-Team

1. Production-A network

2. Production-B network

iii. Backup-Team

1. Backup-A network

2. Backup-B network

Figure 19. HP NCU configuration per blade

c. Once the three teams have been created, promiscuous mode should be enabled on each to allow multiple VLANs to pass over the teamed adapter.

i. To enable promiscuous mode select one of the teams and click properties.

ii. Select the Settings tab and the VLAN Promiscuous property.

34

iii. Set the value to Enabled as shown in Figure 20

Figure 20. Enabling promiscuous mode in HP NCU

8. After the NIC teams have been created, assign any static IP address settings (IP, DNS, etc.) necessary to each of the networks on the host. If following the base design, the following 7 networks will be created:

a. Production-Team

b. Backup-Team

c. Mgt-Team

d. CSV

e. LiveMigration

f. iSCSI-A

g. iSCSI-B

9. The next step in the setup process is to configure the appropriate Hyper-V switches for each host. It is important to pay careful attention to the naming of these switches as they must be consistent across each host. For the base RA design, the following Hyper-V switches with associated properties should be created:

a. Mgt-Team

i. External connection type

ii. Allow mgt OS to share this network adapter

b. Production-Team



c. Backup-Team


35


d. iSCSI-A



e. iSCSI-B



10. Rename the newly created adapter references created as part of the Hyper-V virtual switch creation. The default naming convention is to use the same naming convention and add “-Host” to the suffix.

a. For example, for the Mgt-Team host adapter, the name would be “Mgt-Team-Host” to distinguish between the Mgt-Team virtual switch adapter.

11. Modify the network binding order and unbinding services on the iSCSI networks.

a. The network binding order should be set as follows:

i. Mgt-Team

ii. CSV

iii. LiveMigration

iv. Production-Team

v. Backup-Team

vi. iSCSI-A

vii. iSCSI-B

b. For the iSCSI-A and iSCSI-B networks, the following protocols should be disabled

i. iPv6

ii. File and Print Sharing for Microsoft Networks

iii. Client for Microsoft Networks

iv. QoS Packet Scheduler

12. The final step involves setting up any custom firewall rules or other settings on each host as necessary. Minimally, the following firewall rule should be enabled to allow SCOM to communicate with the HP System Management Homepage (SMH) on each server. With the HP IC-SC management packs, SCOM’s capabilities are extended to include direct links to the SMH on each server. The SMH provides event, monitoring, and alert information for the server and its various sub-components. To enable the firmware rule run the following command from a command prompt:

a. netsh advfirewall firewall add rule name="ICMP Allow incoming V4 echo request" protocol=icmpv4:8,any dir=in action=allow

13. Steps 1-12 should be repeated for each blade, 4 in a base configuration and 8 in a full configuration. And any additional setup required on each host should be configured at this time. Once the initial configuration has been completed, the next steps are as follows:

14. Join each of the hosts to the appropriate Active Directory (AD) domain.

15. Configure iSCSI initiator sessions.

16. Provision P2000 volumes to each host.

36

17. Validate and create Windows Failover Cluster.

iSCSI initiator setup

Once the hosts have been joined to the AD domain, the specific iSCSI connections can be configured using the Microsoft iSCSI initiator.

To enable multi-path storage access with an optimal number of paths for both availability and performance, the iSCSI session configuration as shown in Figure 21 is recommended.

Figure 21. iSCSI initiator session and path configuration

For each host, there will be 4 paths to each storage block in this design and a total of 8 and 16 paths respectively for a base or full DBC RA configuration. To create the appropriate initiator sessions perform the following steps:

1. Open up the iSCSI initiator configuration window on the host.

2. Click on the Discovery tab and select Discover Portal… to add a target.

a. In the Discover Target Portal enter the IP address for port A1 (as shown in the figure above) in the storage block.

b. Select the Advanced… option.

i. Set the Local Adapter value to “Microsoft iSCSI Initiator”.

ii. Set the Initiator IP to the IP address corresponding to the iSCSI-A adapter.

iii. Click Ok.

c. Click Ok.

3. Click on the Targets tab. Select the target just added and click on Properties.

4. Click on the Add Session option.

a. For each storage block there are 4 sessions to initiate including:

i. iSCSI-A -> A1

ii. iSCSI-A -> B1

iii. iSCSI-B -> A2

iv. iSCSI-B -> B2

37

b. Select the check box to enable Multi-Path.

c. Click on the Advanced… button.

i. Set the Local adapter option to “Microsoft iSCSI Initiator”.

ii. Set the Initiator IP.

1. This will be either the iSCSI-A or iSCSI-B adapter IP.

iii. Set the Target IP accordingly.

1. This will be one of the P2000 target IP addresses (A1, A2, B1, or B2).

iv. Click Ok.

d. Click Ok.

5. Repeat step 4 to setup all four sessions as diagrammed in the figure above.

6. Repeat steps 2-5 for each storage block, 2 blocks in a base DBC RA and 4 blocks in full DBC RA.

Note: Figure 22 shows a view of the Favorite Targets display in a base DBC RA once all 8 sessions have been created.

Figure 22. iSCSI target sessions after configuration

For a full DBC RA there would be an additional 8 target connections.

Storage provisioning and formatting

In the initial hardware configuration stage, the P2000 VDisks and Volumes were created. Once the iSCSI sessions have been created, the P2000 volumes can now be presented to each host. In a base DBC RA there are 11 volumes and in a full DBC RA there are 21 volumes. Table 6 shows the volume name and LUN number to use for each volume.

38

Table 6. Volume name and LUN number mapping

Array Number Volume Name LUN Number DBC RA Type

#1 Quorum 200 Base/Full

#1 CSV1 1 Base/Full

#1 CSV2 2 Base/Full

#1 CSV3 3 Base/Full

#1 CSV4 4 Base/Full

#1 CSV5 5 Base/Full

#2 CSV1 6 Base/Full

#2 CSV2 7 Base/Full

#2 CSV3 8 Base/Full

#2 CSV4 9 Base/Full

#2 CSV5 10 Base/Full

#3 CSV1 11 Full only










For a base DBC RA the following order can be used to configure the storage:

1. Provision the volumes from storage array #1 (see steps in “Volume provisioning steps” section below).

2. Format the LUNs on blade #1.



By formatting the LUNs after each array is provisioned, it is easier to format each LUN with a name according to the specific array it is located on. Alternatively, each array can be provisioned first and then the formatting done at the end using the associated LUN numbers to correctly identify which LUN presented to the OS is from which array. Figure 23

39

shows the Details tab for the Quorum LUN, note the LUN 200 value in the Location Information property field to identify the LUN accordingly to the value set in the P2000 when the LUN is provisioned.

Figure 23. LUN ID identification

For a full DBC RA configuration the order and steps from the base DBC RA are repeated with the additional two arrays added as follows:





Volume provisioning steps The steps to provision a volume are as follows:

1. Log into the HP Storage Management Utility for the first P2000.

2. In the left tree, expand the Logical -> Hosts entry and the 4 or 8 host servers should now be visible.

3. Right-click on a host and select Provisioning -> Manage Host Mappings.

4. To set a host mapping perform the following steps:

a. Select a LUN radio button.

b. This will then display a Map check box to enable.

c. Set the appropriate LUN number (as shown in the list above).

d. Click Apply.

5. Repeat step 4 for each LUN.

40

6. Repeat steps 3-5 for each host on the array.

Cluster validation and creation

At this stage the cluster validation and creation process can be performed. Before continuing it is recommended to verify that each host has visibility to all of the drives provisioned in the previous step. This may require several disk rescan operations.

Once the hosts have been validated complete the following steps:

1. Login to blade #1.

2. Open up the Server Manager -> Features -> Failover Cluster Manager.

3. Click on the “Create a Cluster” option.

4. Enter the 4 or 8 host server names.

a. Specify a cluster name.

b. Assign the appropriate cluster IP address to the management network.

c. Uncheck the other networks besides the management network.

d. Run the cluster validation process to ensure that there are no issues in the configuration.

5. Complete the cluster creation.

Once the cluster has been setup there are several additional configuration steps that should be performed as follows:

1. Rename the cluster networks to match the role accordingly.

a. For example, Cluster Network 1 -> Management

2. Disable cluster communications on the two iSCSI networks (iSCSI-A and iSCSI-B).

3. Set cluster network metrics:

a. iSCSI-A = 10100

b. iSCSI-B = 10300

c. CSV = 1000

d. LiveMigration = 1100

4. Enable cluster shared volumes for the cluster LUNs.

At this stage, the Windows Failover Cluster is configured and the next phase of installing the System Center management environment can begin.

System Center management stack configuration

Depending on the existing customer configuration and desired end state, the specific installation requirements for the management stack will be highly variable. For example, some of the pivot points for the installation include whether there is an existing management infrastructure in-place, whether System Center will be installed on separate physical servers, or whether the management stack will be installed on virtual machines inside the DBC RA itself.

Additionally, a full Microsoft private cloud configuration design is predicated on installing a core set of System Center roles while other System Center roles are optional components which enhance core private cloud functionality.

For the base DBC RA design, the System Center management stack components are installed inside VMs running on the DBC RA resource cluster. To support the SQL Server DB requirements while maintaining high availability and scalability, a single SQL guest cluster can be utilized with different SQL instances for each of the System Center roles. For the default DBC RA management environment this will require nine SQL instances as outlined in the following table, Table 7.

41

Table 7. System Center SQL instances

System Center Role SQL Instance Name

SCVMM SCVMMDB

Orchestrator SCODB

App Controller SCACDB

SCOM DB SCOMDB

SCOM Data Warehouse SCOMDW

Service Manager DB SCSMDB

SharePoint SCSPFarm

Service Manager Data Warehouse SCSMDW

Service Manager Analysis Services SCSMAS

To support the LUN requirements the SQL guest cluster will require 19 LUNs, a DB and Log LUN for each SQL instance, and a cluster quorum drive. These LUNs can be presented from the available storage in VDisk6 on each storage block. This VDisk does not have any volumes allocated for host CSV usage. For the management stack LUN requirements, volumes should be created and directly presented to the SQL VMs via iSCSI.

Figure 24 below depicts the core SC2012 management VMs on the first blade server and Figure 25 depicts the second VM for the SQL guest cluster node on an alternate server.

Note The relationship of the VM to the physical blade is only a pictorial representation and in reality the VMs will reside on different servers in the cluster as controlled by SCVMM dynamic optimization and workload migration.

42

Figure 24. System Center management infrastructure

Figure 25. System Center management infrastructure

43

DBC Reference Architecture bill of materials

DBC Reference Architecture Base configuration

Table 8. DBC RA base configuration components

Quantity Part Number Description

Rack and Network Infrastructure

Note – There may be slight variations in the part numbers based on regional specific factory racking designations

1 BW904A HP 642 1075mm Shock Intelligent Rack

5 AF547A HP 5xC13 Intlgnt PDU Ext Bars G2 Kit

1 BW906A HP 42U 1075mm Side Panel Kit

9 C7535A HP Ethernet 7ft CAT5e RJ45 M/M Cable

2 J9145A HP 2910-24G al Switch

2 J9008A HP 2-port 10GbE SFP+ al Module

22 487655-B21 HP BLc SFP+ 3m 10GbE Copper Cable

2 J9265A HP 6600-24XG Switch

2 J9269A HP 6600 Switch Power Supply

1 BW930A HP Air Flow Optimization Kit

2 AF070A HP 10pk Carbt 1U Universal Filler Panel

1 BW891A HP Rack Grounding Kit

Select appropriate PDU option

4 AF520A HP Intelligent Mod PDU 24a Na/Jpn Core

4 AF525A HP 7.3kVA 32A Single Phase INTL Core

HP BladeSystem - c3000 Enclosure

1 508668-B21 HP BLc3000 CTO Encl

2 455880-B21 HP BLc VC Flex-10 Enet Module Opt

2 453154-B21 HP BladeSystem 1Gb RJ-45 SFP Opt Kit

4 500172-B21 HP 1200W CS HE Power Supply Kit

2 507082-B21 HP BLc Single Active Cool 100 Fan Option

1 439034-B23 HP c-Class All FIO 8 Icm 1yr 24x7 Lic

6 AF590A HP 2m 15A C20 C13 Univ Jpr Cord

1 488100-B21 HP BLc3000 Dual DDR2 Onboard Admin Kit

44


HP BladeSystem – Hyper-V cluster nodes

Select Blade Server Model – BL460c Gen8 or BL465c Gen8

BL460c Gen8 Servers

4 641016-B21 HP BL460c Gen8 10Gb FLB CTO Blade

4 667804-L21 HP BL460c Gen8 E5-2667 FIO Kit

4 667804-B21 HP BL460c Gen8 E5-2667 Kit

64 672631-B21 HP 16GB 2Rx4 PC3-12800R-11 Kit

8 652564-B21 HP 300GB 6G SAS 10K 2.5in SC ENT HDD

4 684212-B21 HP FlexFabric 10Gb 2P 554FLB FIO Adptr

4 647590-B21 HP FlexFabric 10Gb 2P 554M Adptr

4 589251-B21 MS WS08 R2 DataCnt2CPU FIO Npi Eng SW

BL465c Gen8 Servers

4 634975-B21 HP BL465c Gen8 10Gb Flb CTO Blade

4 660080-L21 HP BL465c Gen8 6274 2.2GHz 16c FIO Kit

4 660080-B21 HP BL465c Gen8 6274 2.2GHz 16c Kit

64 684066-B21 HP 16GB 2Rx4 PC3-12800R-11 Kit





HP P2000 G3 storage blocks

2 AW597B HP P2000 G3 10GbE iSCSI MSA Dual Controller SFF Array System

6 AJ941A HP D2700 SFF Disk Enclosure, twenty-five 2.5" drive bays

198 512547-B21 HP 146GB 6G SAS 15K 2.5 inch DP ENT HDD

2 407337-B21 HP Ext Mini SAS 1m Cable

45

DBC Reference Architecture Full configuration

Table 9. DBC RA full configuration components


Rack and Network Infrastructure

Note – There may be slight variations in the part numbers based on regional specific factory racking designations

1 BW904A HP 642 1075mm Shock Intelligent Rack

5 AF547A HP 5xC13 Intlgnt PDU Ext Bars G2 Kit

1 BW906A HP 42U 1075mm Side Panel Kit

13 C7535A HP Ethernet 7ft CAT5e RJ45 M/M Cable

2 J9145A HP 2910-24G al Switch

2 J9008A HP 2-port 10GbE SFP+ al Module

30 487655-B21 HP BLc SFP+ 3m 10GbE Copper Cable

2 J9265A HP 6600-24XG Switch

2 J9269A HP 6600 Switch Power Supply

1 BW930A HP Air Flow Optimization Kit

1 BW891A HP Rack Grounding Kit

Select appropriate PDU option

4 AF520A HP Intelligent Mod PDU 24a Na/Jpn Core

4 AF525A HP 7.3kVA 32A Single Phase INTL Core

HP BladeSystem - c3000 Enclosure

1 508668-B21 HP BLc3000 CTO Encl

2 455880-B21 HP BLc VC Flex-10 Enet Module Opt

2 453154-B21 HP BladeSystem 1Gb RJ-45 SFP Opt Kit

4 500172-B21 HP 1200W CS HE Power Supply Kit

2 507082-B21 HP BLc Single Active Cool 100 Fan Option

1 439034-B23 HP c-Class All FIO 8 Icm 1yr 24x7 Lic

6 AF590A HP 2m 15A C20 C13 Univ Jpr Cord

1 488100-B21 HP BLc3000 Dual DDR2 Onboard Admin Kit

46


HP BladeSystem – Hyper-V cluster nodes

Select Blade Server Model – BL460c G8 or BL465c G8

BL460c Gen8 Servers

8 641016-B21 HP BL460c Gen8 10Gb FLB CTO Blade

8 667804-L21 HP BL460c Gen8 E5-2667 FIO Kit

8 667804-B21 HP BL460c Gen8 E5-2667 Kit

128 672631-B21 HP 16GB 2Rx4 PC3-12800R-11 Kit





BL465c Gen8 Servers

8 634975-B21 HP BL465c Gen8 10Gb Flb CTO Blade

8 660080-L21 HP BL465c Gen8 6274 2.2GHz 16c FIO Kit

8 660080-B21 HP BL465c Gen8 6274 2.2GHz 16c Kit

128 684066-B21 HP 16GB 2Rx4 PC3-12800R-11 Kit





HP P2000 G3 storage blocks

4 AW597B HP P2000 G3 10GbE iSCSI MSA Dual Controller SFF Array System

12 AJ941A HP D2700 SFF Disk Enclosure, twenty-five 2.5" drive bays

396 512547-B21 HP 146GB 6G SAS 15K 2.5 inch DP ENT HDD

4 407337-B21 HP Ext Mini SAS 1m Cable

47

Appendix

The following Appendix information contains the racking, wiring, and cabling information for the base and full DBC configurations

Base DBC RA racking, wiring and cabling diagrams

Figure 26. Base DBC RA racking diagram

48

Figure 27. Base DBC RA iSCSI storage fabric wiring diagram

49

Figure 28. Base DBC RA management fabric wiring

50

Full DBC RA wiring and cabling diagrams

Figure 29. Full DBC RA racking diagram

51

Figure 30. Full DBC RA iSCSI storage fabric wiring diagram

52

Figure 31. Full DBC RA management fabric wiring diagram

53

DBC RA storage block SAS cabling

Figure 32. P2000 G3 10Gb iSCSI fault tolerant cabling

HP StorageWorks P2000 G3 MSA System Cable Configuration Guide

http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c02254377/c02254377.pdf

http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c02254377/c02254377.pdf

54

DBC RA power cabling

Figure 33. DBC RA Base and Full Power cabling

55

For more information

HP DBC for SQL Server Consolidation hp.com/solutions/microsoft/dbc

HP BladeSystem hp.com/go/bladesystem

HP P2000 Storage hp.com/go/msa

HP Factory Express hp.com/go/factoryexpress

Microsoft Assessment and Planning (MAP) Toolkit information

microsoft.com/en-us/download/details.aspx?id=7826

Microsoft Private Cloud Solutions microsoft.com/en-us/server-cloud/readynow

To help us improve our documents, please provide feedback at hp.com/solutions/feedback.

Get connected hp.com/go/getconnected

Current HP driver, support, and security alerts delivered directly to your desktop

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omiss ions contained herein.

Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation. Intel is a trademark of Intel Corporation in the U.S. and other countries. AMD is a trademark of Advanced Micro Devices, Inc.

4AA4-2741ENW, Created July 2012; Updated October 2012, Rev. 2

http://www.hp.com/solutions/microsoft/dbc

http://www.hp.com/go/bladesystem

http://www.hp.com/go/msa

http://www.hp.com/go/factoryexpress

http://www.microsoft.com/en-us/download/details.aspx?id=7826

http://www.microsoft.com/en-us/server-cloud/readynow

http://www.hp.com/solutions/feedback

http://www.hp.com/go/getconnected

hp dbc reference architecture technical overvie · the hp dbc reference architecture (dbc ra) is a...

Documents