onecore™ storage sdk quick start guide - broadcom · linux ocs_fc_ramd linux fc/fcoe initiator...

P008909-17A Rev. A Connect • Monitor • Manage

OneCore™ Storage SDKQuick Start Guide

10.6 Release

Revision 4.5August 10, 2015

OneCore Storage SDK Quick Start Guide, 10.6 Release P008909-17A Rev. ARevision 4.5 August 10, 2015

2

Copyright © 2012–2015 Emulex. All rights reserved worldwide. No part of this document may be reproduced by any means or translated to any electronic medium without the prior written consent of Emulex.

Information furnished by Emulex is believed to be accurate and reliable. However, no responsibility is assumed by Emulex for its use; or for any infringements of patents or other rights of third parties which may result from its use. No license is granted by implication or otherwise under any patent, copyright, trade secret or related rights of Emulex.

Emulex, the Emulex logo, Emulex BladeEngine, Emulex InSpeed, Emulex LightPulse, Emulex OneCommand, Emulex OneConnect, and Emulex SLI are registered trademarks, and Emulex Advanced-8, Emulex Connect, Emulex CrossLink, Emulex Engine, Emulex Edge, Emulex ExpressLane, Emulex GreenState, Emulex OneCore, Emulex Pilot, Emulex SURF, Emulex Universal Multi-Channel, Emulex vEngine, Emulex Virtual Fabric, Emulex Virtual Network Exceleration, Emulex vPath, and Emulex vScale, are trademarks, of Emulex. All other brand or product names referenced herein are trademarks or registered trademarks of their respective companies or organizations.

Emulex provides this documentation “as is” without any warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability or fitness for a particular purpose. Emulex may make improvements and changes to the product described in this manual at any time and without any notice. Emulex assumes no responsibility for its use, nor for any infringements of patents or other rights of third parties that may result. Periodic changes are made to information contained herein; although these changes will be incorporated into new editions of this documentation, Emulex disclaims any undertaking to give notice of such changes.

Emulex, 3333 Susan Street

Costa Mesa, CA 92626

Table of Contents

3

Table of Contents

List of Tables .............................................................................6

1 Introduction............................................................................7

1.1 Overview................................................................................7

1.2 SDK Drivers and Utilities.............................................................8

1.3 Emulex Reference Documentation ................................................9

1.3.1 OneCore Storage SDK Documentation ....................................... 9

1.3.2 Browser-Based API Documentation .........................................10

1.4 Abbreviations ........................................................................ 10

2 Directory Structure................................................................. 12

2.1 Tree Structure of the .tgz File ................................................... 12

2.1.1 doc Directory...................................................................13

2.1.2 driver Directory ................................................................13

2.1.3 tools Directory .................................................................15

3 Building and Installing the RAMD Drivers....................................... 16

3.1 BSD FC/FCoE Target RAMD Driver (BSD ocs_fc_ramd) ....................... 16

3.1.1 OS Requirements ..............................................................16

3.1.2 Building .........................................................................17

3.1.3 Installing ........................................................................173.1.3.1 Module Parameters .......................................................... 17

3.1.3.2 Loading the Driver ........................................................... 21

3.2 Linux FC/FCoE I+T RAMD Driver (Linux ocs_fc_ramd)....................... 22

3.2.1 OS Requirements ..............................................................22

3.2.2 Chip Platform Support ........................................................22

3.2.3 Building .........................................................................223.2.3.1 General Instructions ......................................................... 22

3.2.3.2 ARM-specific Instructions for Ubuntu 14.04.............................. 23

3.2.3.2.1 Prerequisites..................................................... 23

3.2.3.2.2 Building ........................................................... 23

3.2.3.2.2.1 Cross-compile Environment Setup............................................................... 23

3.2.4 Installing ........................................................................243.2.4.1 Module Parameters .......................................................... 24

3.2.4.2 Module Parameter Examples ............................................... 30

3.2.5 Creating Device Special Files ................................................30


Table of Contents

4

3.2.6 Performance Optimization ...................................................30

3.3 Linux iSCSI I+T RAMD Driver (Linux ocs_iscsi_ramd) ........................ 31

3.3.1 OS Requirements ..............................................................31

3.3.2 Using the iSCSI Target and iSCSI Initiator..................................313.3.2.1 Using the iSCSI Target ....................................................... 31

3.3.2.2 Using the iSCSI Initiator ..................................................... 31

3.3.3 Building and Installing the Linux ocs_iscsi_ramd Driver .................323.3.3.1 Building ........................................................................ 32

3.3.3.2 Installing....................................................................... 32

3.3.3.2.1 Module Parameters ............................................. 32

3.3.4 Open-iSCSI Interface (ocs_openiscsi) Module .............................363.3.4.1 Overview ...................................................................... 36

3.3.4.2 Building ........................................................................ 36

3.3.5 Creating Device Special Files ................................................37

3.3.6 Performance Optimization ...................................................37

3.3.7 Using CHAP .....................................................................373.3.7.1 Enabling CHAP on Linux ..................................................... 37

3.3.7.2 Enabling CHAP on Windows................................................. 39

3.3.7.2.1 Mutual CHAP Set-up ............................................ 39

3.3.7.2.2 Enabling CHAP for Discovery Sessions........................ 40

3.3.7.2.3 Enabling CHAP for Normal Sessions .......................... 41

3.4 User Space FC/FCoE Target RAMD Driver (Uspace ocs_fc_ramd) ......... 44

3.4.1 OS Requirements ..............................................................44

3.4.2 Unpacking and Building.......................................................44

3.4.3 Installing the Kernel Module .................................................45

3.4.4 Running the User Space Application ........................................453.4.4.1 Module Parameters .......................................................... 45

3.4.4.2 Module Parameter Example ................................................ 45

3.4.5 CPU Affinity and Performance Optimization ..............................45

3.4.6 Terminating the User Space Application...................................46

3.5 User Space iSCSI Target RAMD Driver (Uspace ocs_iscsi_ramd) ........... 47

3.5.1 OS Requirements ..............................................................47

3.5.2 Unpacking and Building.......................................................47

3.5.3 Installing the Kernel Module .................................................48

3.5.4 Running the User Space Application ........................................483.5.4.1 Module Parameters .......................................................... 48

3.5.4.2 Module Parameter Example ................................................ 48

3.5.5 CPU Affinity and Performance Optimization ..............................49

3.5.6 Terminating the User Space Application...................................49


Table of Contents

5

3.5.7 Using CHAP .....................................................................49

4 User Space ioctl() RPC Interface ................................................. 50

4.1 Overview.............................................................................. 50

4.2 elxsdkutil --host Option............................................................ 50

4.2.1 Examples........................................................................50

5 Building and Installing the elxsdkutil Utility .................................. 52

5.1 Prerequisites......................................................................... 52

5.2 Building ............................................................................... 52

5.3 Installing .............................................................................. 52

5.3.1 Device Special Files ...........................................................53

6 Linux-Specific Scripts .............................................................. 54

6.1 ocs_config.py Script ................................................................ 54

6.1.1 Device Special Files ...........................................................546.1.1.1 Creating Device Special Files Manually ................................... 54

6.1.2 Performance Tuning...........................................................55

Appendix A. Module Parameter Extras ........................................... 56

A.1 Module Parameters Per Driver Comparison ................................... 56

Appendix B. Viewing Parameter Values .......................................... 58



List of Tables

6

List of TablesTable 1-1 OneCore Storage Drivers and Utilities ....................................... 8

Table 2-1 driver Directory ................................................................13

Table 2-2 tools Directory..................................................................15

Table 3-1 BSD ocs_fc_ramd Module Parameters.......................................17

Table 3-2 Linux ocs_fc_ramd Module Parameters.....................................24

Table 3-3 Linux ocs_iscsi_ramd Module Parameters ..................................32

Table 3-4 CHAP Secrets ...................................................................37

Table A-1 Module Parameter Comparison Between OneCore Storage Drivers ...........................................................................56

1 Introduction1.1 Overview

7

1 Introduction

1.1 OverviewThis document provides an overview of the OneCore™ Storage software development kit (SDK), including a list of the available reference and production drivers, utilities, file organization, and documentation information. It also provides instructions for building and installing the OneCore Storage reference RAMD drivers and the Emulex OneCore Storage SDK utility (elxsdkutil).

The OneCore Storage drivers are based on a common driver framework and operate on the Linux or FreeBSD operating system. These drivers are modular in design and can be tailored to fit customer product-specific requirements and needs.

The OneCore Storage drivers provide implementations of FC/FCoE and iSCSI drivers for Emulex’s Service Level Interface - 4 (SLI®-4) family of products, such as:

XE100-Series 10Gb/40Gb Ethernet Converged Network I/O controller (IOC) and its adapters: OCe14000-series of 10GbE and 40GbE Converged Network Adapters

(CNAs), such as the OCe14102-UX and OCe14102-UM. XE201 IOC, combinations of 8 Gigabit Fibre Channel (GFC), 16 GFC, and

10 Gigabit Ethernet (GbE), and its adapters: LPe15004 quad-port Advanced-8 8GFC host bust adapters (HBAs) LPe16000-series of HBAs (such as LPe16000B, LPe16202, and LPe16004)

For a complete list and exact part numbers, contact your Emulex support or sales representative.

Note: OneCore Storage drivers do not support older devices that use the SLI-3 standard.


1 Introduction1.2 SDK Drivers and Utilities

8

1.2 SDK Drivers and UtilitiesThe SDK includes the following production drivers, reference drivers, and utilities:

Table 1-1 OneCore Storage Drivers and Utilities

Type OS Name Description Page

Production Drivers

Linux ocs_fc_lio Linux FC/FCoE initiator and target production driver that uses the LIO (TCM) target.

This driver is in “Technical Preview.” For more information, see the latest OneCore Storage LIO Driver Guide.

-

Linux ocs_fc_scst Linux FC/FCoE initiator and target production driver that connects to the SCST target, and to the Linux SCSI mid-layer for its initiator support.

For more information, see the latest OneCore Storage SCST Driver Guide.

-

Linux ocs_iscsi_scst Linux iSCSI target production driver that connects to the SCST through a SCST shim back-end target interface. This driver does not include initiator support.

This driver is in “Technical Preview.” For more information, see the latest OneCore Storage SCST Driver Guide.

-

Reference Drivers

Linux ocs_fc_ramd Linux FC/FCoE initiator and target reference driver that connects to the Emulex RAMD target, and to the Linux SCSI mid-layer for its initiator support.

22

Linux ocs_iscsi_ramd Linux iSCSI initiator and target reference driver that connects to the Emulex RAMD target, and to the Linux SCSI mid-layer for its initiator support.

31

Linux Uspace ocs_fc_ramd

The Linux user space FC/FCoE target reference driver that connects to the Emulex RAMD target. This driver does not include initiator support.

44

Linux Uspace ocs_iscsi_ramd

The Linux user space iSCSI target reference driver that connects to the Emulex RAMD target. This driver does not include initiator support.

47

FreeBSD ocs_fc_cam FreeBSD FC/FCoE initiator and target reference driver that connects to FreeBSD’s Common Access Methods (CAM) framework for back-end connectivity to initiators and targets.

For more information, see the latest OneCore Storage BSD CAM Driver Guide.

-

FreeBSD ocs_fc_ramd FreeBSD FC/FCoE target reference driver that connects to the Emulex RAMD target. This driver does not include initiator support.

16


1 Introduction1.3 Emulex Reference Documentation

9

1.3 Emulex Reference Documentation

1.3.1 OneCore Storage SDK Documentation OneCore Storage BSD CAM Driver Guide – contains BSD ocs_fc_cam

driver-specific information. OneCore Storage Driver Design Manual – provides the architecture and design

details for the common front-end FC/FCoE and iSCSI components of the reference and production drivers. It also provides specific details for the reference drivers that use the RAMD as its target back-end, such as the Linux ocs_fc_ramd driver.

OneCore Storage FAQs – provides answers for frequently asked questions (FAQs) for the OneCore Storage drivers and utilities.

OneCore Storage LIO Driver Guide – contains Linux ocs_fc_lio driver-specific information.

OneCore Storage Performance Tuning Application Note – contains details to improve Emulex adapter performance when using the OneCore Storage Linux drivers in a multi-core CPU environment. It includes descriptions for CPU

Utilities Linux/Free BSD

elxocsutil The elxsdkutil utility can be built for use with FreeBSD or Linux. The elxocsutil utility provides the ability to manage profiles on the OCe14000-series of adapters. It uses the Configuration Space IOCTL Interface (CSII), along with PCI information from sysfs, to access the device. As such, the elxocsutil utility is able to query and configure the device without requiring any assistance from a device driver.

For more information, see the latest OneCore Storage Utilities Manual.

-

Linux/Free BSD

elxsdkutil The elxsdkutil utility can be built for use with FreeBSD or Linux. The elxsdkutil utility and its source code provide a working example of common management operations, including firmware download, diagnostic dumps, device queries, and so on.


52

Linux/FreeBSD

mgmt The mgmt utility provide command line access to the mgmt interface in the OneCore Storage drivers. Information about the driver is communicated in the form of an XML document.


-

Table 1-1 OneCore Storage Drivers and Utilities (Continued)

Type OS Name Description Page


1 Introduction1.4 Abbreviations

10

affinity, CPU frequency scaling performance settings, and OCS driver port to NUMA node mapping. The ocs_config.py script, located in the tools/bin directory, incorporates these settings.

OneCore Storage SCST Driver Guide – contains Linux ocs_fc_scst driver and Linux ocs_iscsi_scst driver specific information.

OneCore Storage SDK Quick Start Guide – (this document) OneCore Storage SDK Release Notes – lists known issues, resolved issues, and

technical tips for a particular OneCore Storage SDK release. OneCore Storage Utilities Manual – describes the usage and commands of the

elxocsutil, elxsdkutil, and mgmt utilities. It also includes helpful SDK information in relation to the SLI-4 specifications.

1.3.2 Browser-Based API DocumentationThe OneCore Storage drivers and elxsdkutil utility include an HTML-based reference manual that is generated from the source code using the Doxygen tool. This documentation provides links for navigating through the source code; displaying files and the relationships between files; listing modules; and providing descriptions for functions and variables. To access <driver-specific> HTML-based documentation, you must initially extract the “<driver-specific>-doc.tgz” package into the <driver-specific> directory. For example, to access the Linux ocs_fc_ramd driver’s HTML files:

1. Open the ocs_sdk_documentation.html file (located at the same level as the driver and tools directories).

2. In the table, extract the Linux ocs_fc_ramd driver’s ocs_fc_ramd-doc.tgz files into the “driver/linux/ocs_fc_ramd/” directory. This places the *.html files under the “driver/linux/ocs_fc_ramd/doc/html/” directory.

3. In the table, click on the Linux ocs_fc_ramd driver’s HTML files link. This is the link to the driver's overview page (index.html), which provides access to the other HTML files.

1.4 AbbreviationsABTS abort sequence

API application programming interface

CAM common access method

CPU central processing unit

DMA direct memory access

ELS extended link service

FC Fibre Channel

FCoE Fibre Channel over Ethernet

FCP Fibre Channel Protocol for SCSI

GbE Gigabit Ethernet


1 Introduction1.4 Abbreviations

11

GFC Gigabit Fibre Channel

HAL hardware abstraction layer

HBA host bus adapter

HTML HyperText Markup Language

I/O input/output

iSCSI Internet Small Computer System Interface

LUN logical unit number

N_Port node port

NPIV N_Port ID virtualization

OCS OneCore™ Storage

OS operating system

PCI Peripheral Component Interconnect

R2T Ready to Transfer

RPI remote pair indicator

RSCN registered state change notification

SCSI Small Computer System Interface

SGL scatter/gather list

SLI Service Level Interface

SRRS single request response sequence

VPI virtual port indicator

WQE work queue entry

XRI exchange resource indicator


2 Directory Structure2.1 Tree Structure of the .tgz File

12

2 Directory Structure

2.1 Tree Structure of the .tgz FileThe OneCore Storage SDK release package is contained in a zip-compressed, tar archive file (.tgz). The top-level organization (tree structure), after unpacking the .tgz file, has three main directories (doc, driver, and tools). The top three levels are shown below, and described in the following sections:

doc

- ocs_bsd_cam_driver_guide.pdf

- ocs_driver_design_manual.pdf

- ocs_faqs.pdf

- ocs_lio_driver_guide.pdf

- ocs_performance_tuning_app_note.pdf

- ocs_scst_driver_guide.pdf

- ocs_sdk_quick_start_guide.pdf

- ocs_sdk_release_notes.pdf

- ocs_utilities_manual.pdf

driver

- bsd

- common

- ocs_cam

- ocs_fc_cam

- ocs_fc_ramd

- common

- doxy

- hal

- ihal

- ocs

- ocs_fc

- ocs_ini_stub

- ocs_iscsi

- ocs_ramd

- ocs_template

- linux

- common

- ocs_fc_lio

- ocs_fc_ramd

- ocs_fc_scst

- ocs_iscsi_ramd



13

- ocs_iscsi_scst

- ocs_openiscsi

- ocs_uspace

- uspace

- common

- ocs_fc_ramd

- ocs_iscsi_ramd

tools

- bin

- get_port_login_data.sh

- ocs_config.py

- ocs_config.sh

- ocs_cpu_info.py

- user

- common

- elxocsutil

- elxscdul

- elxsdkutil

- hba_api

- mgmt

2.1.1 doc DirectoryThe doc directory contains PDFs of the documents listed in Section 1.3.1, OneCore Storage SDK Documentation, on page 9.

Note: The document filenames may differ in each release and the doc directory may contain additional files.

2.1.2 driver DirectoryThe driver directory contains four main subdirectories: bsd (FreeBSD-specific files), common (files shared between FreeBSD and Linux), linux (Linux-specific files), and uspace (User Space-specific files).

Table 2-1 driver Directory

driver Subdirectories Description

bsd common Common code that is shared by the BSD drivers

ocs_cam The FreeBSD CAM related code for back-end connectivity to initiators and targets

ocs_fc_cam The FreeBSD ocs_fc_cam driver-specific files and Makefile



14

common doxy Common files for presenting Doxygen-generated html files

hal FC/FCoE-specific HAL and SLI-4 code

ihal The iSCSI-specific HAL and SLI-4 code

ocs Common code and headers shared by the OneCore Storage drivers

ocs_fc The FC/FCoE transport code

ocs_ini_stub Linux initiator stub code for driver builds that do not include initiator functionality

ocs_iscsi iSCSI transport code

ocs_ramd RAMD code that is shared by the ocs_fc_ramd and ocs_iscsi_ramd drivers

ocs_template This code allows a target or initiator that is compiled as a separate module to link in to the OCS front-end. The OCS front-end loads first, providing access to the OCS SCSI API. Then another module can load, register as a driver, and provide the initiator or target function.

linux common Common code that is shared by the Linux drivers. It includes ocs_ml_fc (code and headers that provide the integration of the Linux FC/FCOE front-end into the Linux SCSI mid-layer).

ocs_fc_lio Linux ocs_fc_lio driver-specific files and Makefile

ocs_fc_ramd Linux ocs_fc_ramd driver-specific files and Makefile

ocs_fc_scst Linux ocs_fc_scst driver-specific files and Makefile

ocs_iscsi_ramd Linux ocs_iscsi_ramd driver-specific files and Makefile

ocs_iscsi_scst Linux ocs_iscsi_scst driver-specific files and Makefile

ocs_openiscsi Files for the ocs_openiscsi module that provides the iSCSI initiator function for Open-iSCSI. This module attaches to the Linux ocs_iscsi_ramd driver using the functions provided by the ocs_template files.

ocs_uspace Linux ocs_uspace kernel module required in support of the user space driver

uspace common Common code that is shared by the user space drivers

ocs_fc_ramd Linux user space ocs_fc_ramd driver-specific files and Makefile

ocs_iscsi_ramd Linux user space ocs_iscsi_ramd driver-specific files and Makefile

Table 2-1 driver Directory (Continued)

driver Subdirectories Description



15

2.1.3 tools DirectoryThe tools directory contains the bin and user subdirectories.

Table 2-2 tools Directory

tools Subdirectories Description

bin get_port_login_data.sh

This script dumps port login information to standard out. See the “Scripts” chapter in the OneCore Storage Utilities Manual, 5.2.1 Release, or later.

ocs_config.py or ocs_config.sh

The ocs_config.py or ocs_config.sh script:• Creates device special files. • Employs configuration settings for CPU affinity and CPU frequency

scaling to improve performance for the OneCore Storage Linux drivers. Also uses OCS driver port to NUMA node mapping to minimize latency by running OCS driver processes on local memory. See the OneCore Storage SDK Performance Tuning Application Note (March 2015, or later version).

For more information, see Section 6.1, ocs_config.py Script, on page 54.

ocs_cpu_info.py A supporting for the ocs_config.py script.

user common Contains a few common files for tools.

elxocsutil Contains the files for the Emulex OneCore Storage utility (elxocsutil). The elxocsutil utility provides the ability to manage profiles on the OCe14000-series of adapters. It uses the Configuration Space IOCTL Interface (CSII), along with PCI information from sysfs, to access the device. As such, the elxocsutil utility is able to query and configure the device without requiring any assistance from a device driver.

The elx_pt driver files are located in the “tools/user/elxocsutil/driver” directory.

elxcdul Library and sample application to extract a core dump image from an Emulex FCoE adapter.

elxsdkutil Contains the Emulex SDK Utility (elxsdkutil) source code and Makefile. See Section 5, Building and Installing the elxsdkutil Utility, on page 52. For commands and design details, see the “elxsdkutil Utility” chapter in the OneCore Storage Utilities Manual.

hba_api This hba_api library is an implementation of hbaapi as defined in the Fibre Channel HBA API (FC-HBA) standard, Revision 14 (ANSI INCITS Project T11/1568-D).

This library provides a standard API, which can be called to gather information about Fibre Channel HBAs that are controlled by the OneCore Storage SDK drivers.

For more information, see the “README” document in this directory.

mgmt Contains the files for the Management (mgmt) utility. For details, see the “mgmt Utility” chapter in the OneCore Storage Utilities Manual.


3 Building and Installing the RAMD Drivers3.1 BSD FC/FCoE Target RAMD Driver (BSD ocs_fc_ramd)

16

3 Building and Installing the RAMD DriversThis section provides instructions for building and installing the OneCore Storage reference drivers that use the RAMD target back-end. They include the following drivers:

“BSD FC/FCoE Target RAMD Driver (BSD ocs_fc_ramd)” on page 16 “Linux FC/FCoE I+T RAMD Driver (Linux ocs_fc_ramd)” on page 22 “Linux iSCSI I+T RAMD Driver (Linux ocs_iscsi_ramd)” on page 31 “User Space FC/FCoE Target RAMD Driver (Uspace ocs_fc_ramd)” on page 44 “User Space iSCSI Target RAMD Driver (Uspace ocs_iscsi_ramd)” on page 47

For the drivers that use a CAM, LIO/TCM, or SCST back-end, see their respective manuals for building and installing instructions:

BSD ocs_fc_cam – OneCore Storage BSD FC/FCoE I+T CAM Driver Guide Linux ocs_fc_lio – OneCore Storage LIO/TCM Driver Guide Linux ocs_fc_scst and ocs_iscsi_scst – OneCore Storage SCST Driver Guide

Note: In this section, the instructions for building and running applications assume that you are logged in as root.

3.1 BSD FC/FCoE Target RAMD Driver (BSD ocs_fc_ramd)

The BSD ocs_fc_ramd driver is an FC/FCoE reference driver that uses the Emulex RAMD target. As a target reference driver, it implements a non-persistent RAM disk backing store. It includes support for a number of SCSI commands, sufficient to allow most FC/FCoE initiators to discover the RAM disk, log in, and perform I/Os.

This driver does not have initiator functionality.

3.1.1 OS RequirementsFor the BSD ocs_fc_ramd driver:

Use FreeBSD 10.1 or later versions. Install the latest operating system patches for FreeBSD (to enable the latest

functionality).

For more OS requirements, see the OneCore Storage SDK Release Notes (included with the OneCore Storage SDK release package).



17

3.1.2 BuildingTo build the BSD ocs_fc_ramd driver:

1. In a FreeBSD system capable of performing kernel builds, extract the files into a work directory.

2. Change to the ocs_fc_ramd directory and use the make utility:make depend all

The make utility builds the ocs_fc_ramd.ko loadable kernel object module.

3.1.3 Installing

3.1.3.1 Module ParametersSet parameters on a per port basis prior to loading the driver by using the kenv program:

kenv hint.ocs_fc_ramd.<port>.<parameter>=”<value>”

The following example sets the number of LUNs to four and enables state machine logging on ports 0 and 1:

kenv hint.ocs_fc_ramd.0.num_luns=4

kenv hint.ocs_fc_ramd.1.num_luns=4

kenv hint.ocs_fc_ramd.0.logmask=0x11

kenv hint.ocs_fc_ramd.1.logmask=0x11

The driver supports several load-time parameters available through the following kernel environment variables:

Table 3-1 BSD ocs_fc_ramd Module Parameters

Module Parameter Description

auto_xfer_rdy_size This setting determines when the firmware automatically sends an FCP transfer ready (FCP_XFER_RDY) to the initiator.

It is the maximum size, in bytes, of a write command received from the initiator. For any write command less than or equal to <auto_xfer_rdy_size>, an FCP_XFER_RDY frame is automatically generated by the firmware.

Possible values are 2048 to 65536 (64K). Default is 0 (disabled).

ddump_saved_size Indicates the size, in bytes, of a saved driver dump. Generating a saved driver dump requires additional driver code changes. The size can be any positive number. Default is 0 (disabled).



18

dif_separate Note: This parameter is only applicable if the p_type parameter is “1” or “2”.

This parameter specifies the memory model for T10-PI operations:• For the XE201: When set to 1, the T10 DIF separate model is used.

When set to 0, the inline model is used (default).• For the XE100-series: Only the separate model is supported and

dif_separate must be set to 1. Do not set to this parameter to 0. T10 DIF inline is not supported. Default is 0.

enable_hlm Enables high login mode (HLM) when set to 1. Default is 0 (disabled). When HLM is disabled, then one remote port indicator (RPI) per login is allocated.

Enabling HLM may be necessary when an application may exceed the SLI-4/adapter RPI constraints. When HLM is enabled, the number of logins is limited only by driver resources (memory).

When HLM is enabled, remote connections that share the same service parameters can share an RPI. After an RPI and service parameters are registered with the adapter, subsequent requests (target send/receive/response requests, initiator send command requests) will specify the destination FC_ID in the request.

If enabled, then RPIs are allocated as follows:• A unique RPI is allocated for each of the first hlm_group_size logins

from remote nodes with compatible service parameters.• Subsequent login requests in excess of the hlm_group_size will use

(share) one of the previously allocated RPIs on a round robin basis.

explicit_buffer_list Host dynamically assigns SGLs as needed when set to 1. Driver pre-registers fixed-value SGLs when set to 0 (default).

external_loopback Enables external loopback mode when set to 1. This mode allows a port to discover and log in to itself when the Tx and Rx signals are connected through a loopback hood. The default is 0 (external loopback mode disabled).

global_ramdisc Creates a global ramdisc shared across all ports when set to 1 (default). Creates a ramdisc for each port when set to 0.

hal_war_version Firmware version that overrides the version reported by the firmware. This parameter is used for HAL workaround matching.

The default is the version reported by the firmware (that is, hal_war_version is an empty string).

Typically used for test purposes.

hlm_group_size Sets the high login mode group size. Possible values are 1–8192. Default is 8. This low default value is chosen to facilitate testing of the feature; in a typical scenario, it is expected that a higher value would be specified.

initiator Enables initiator functionality when set to 1. Default is 0 (disabled). Since this is a target-only driver, initiator=1 is not supported.

Table 3-1 BSD ocs_fc_ramd Module Parameters (Continued)




19

logdest Indicates the location of the logging destination:• bit[0] - Set to 1 to enable system logging (default).• bit[1] - Set to 1 to enable RAM logging. Default is 0 (disabled). Also see

the ramlog_size parameter.

loglevel Indicates the type and amount of log output. Each successive loglevel includes the information of the lower levels. For example, loglevel=3 (default) includes LOG_INFO, LOG_WARN, LOG_ERR, and LOG_CRIT information. Valid values are 0 - 5:• 0 - LOG_CRIT: Logs critical information. The hardware is not usable.

The chip is unresponsive or in a Unrecoverable Error (UE) state.• 1 - LOG_ERR: Logs error information that may prevent normal

operation, such as out-of-memory or start-up failures.• 2 - LOG_WARN: Logs warnings and error information. The driver is

functional, but the requested operation may not have worked as expected (for example, limits may have been exceeded or I/O errors may have occurred).

• 3 - LOG_INFO: Logs informational messages. Operation is normal, but something of interest may have occurred, such as a driver start or link up/down.

• 4 - LOG_TEST: Logs test information. Operation is normal, but an error may have occurred and has been handled successfully.

• 5 - LOG_DEBUG: Logs debugging information for development and test. Operation is normal.

The default is “3”, LOG_INFO, if no number is selected.

logmask Flag bits to enable logging:• bit[0] - Set to 1 to enable node state machine traces.• bit[1] - Set to 1 to enable ELS traces.• bit[2] - Set to 1 to enable SCSI command traces.• bit[3] - Set to 1 to enable SCSI back-end target traces.• bit[4] - Set to 1 to enable domain and sport state machine traces.• bit[5] - Set to 1 to enable the logging of messages with a status of SCSI

Task Set Full (Queue Full) or Busy.

The default value for each of the bits is 0 (that is, logging is disabled by default). Values are also defined in the ocs_debug.h file.

num_luns Number of LUNs within the ramdisc. Default is 1.

num_scsi_ios Indicates the number of I/Os allocated within the driver that is used for the SCSI API. This value is independent of the number of I/Os supported by the ASIC.

Possible values are any positive numbers. Default is 8192.

This default value is sufficient for most environments. In an environment with a large number of initiators, increasing the num_scsi_ios value may be needed to prevent Task Set Full responses. To determine a num_scsi_ios value, the following rough equation may be helpful:

(# of initiators * queue_depth * number of LUNs) + 32





20

num_vports Specifies the number of NPIV ports to create. The default is 0.

The NPIV port WWNN is a type 2 address derived from a factory-supplied MAC address. The NPIV WWPN is derived by OR’ing the NPIV port ID (also known as VPI) into bits 59:48 of the WWNN.

p_type Specifies the type of T10 Data Integrity Feature (DIF). Possible values include:• 0 – No protection (default)• 1 – Type 1 protection• 2 – Type 2 protection

For details on T10 DIF, see Section 4.20, Protection information model, in the latest SCSI Block Commands -3 (SBC-3) standard.

For T10 DIF separate or inline settings per controller, see the dif_separate parameter.

ramdisc_blocksize RAMdisc block size in bytes. Default is 512.

ramdisc_size RAMdisc size in bytes in the form of <number>[k|K|m|M|g|G]. Default is 50M.

ramlog_size Indicates the size of the RAM logging buffer, an integer in bytes. Default is 1048576 (1M). Also see the logdest parameter.

rq_threads Indicates the number of Receive Queue (RQ) processing threads that the driver creates. Default is 0.

RQ threads are used to distribute the work of incoming SCSI command frames across CPUs. On slow CPU machines (for example, 2.1 GHz), using multiple RQ threads (for example, rq_threads=2) may increase CPU performance. On fast CPU machines (for example, 3.3 GHz), using multiple RQ threads has little to no effect on performance.

speed Sets the link speed. Value is in megabits per second. Possible values include: • 0 – Auto-speed negotiation (default)• 4000 (4GFC)• 8000 (8GFC)• 10000 (10GbE)• 16000 (16GFC)

stub_res6_rel6 Indicates the type of response for the legacy Reserve(6) (0x16) and Release(6) (0x17) SCSI commands.• 0 – Driver returns a CHECK CONDITION of ILLEGAL REQUEST, Invalid

Command Operation Code (05/20/00) for the Reserve(6)/Release(6) commands (default).

• 1 – Driver returns a GOOD status for the Reserve(6)/Release(6) commands, however no actual Reserve/Release functionality is implemented.

target Enables target functionality when set to 1 (default). Since the BSD ocs_fc_ramd driver is a target-only driver, target=0 is not supported.





21

3.1.3.2 Loading the DriverTo load the driver, as root:

kldload ./ocs_fc_ramd.ko

Note: This kldload command must be executed in the same directory as the ocs_fc_ramd.ko file, otherwise, you must include the complete path to the ocs_fc_ramd.ko file.

target_io_timer Note: This parameter is only applicable when the target parameter is 1.

The timeout value, in seconds, for target commands. The default is 0 (target_io_timer disabled).

topology Specifies FC topology:• 0 – Auto-select; first tries N_Port, then loop. (default)• 1 – N_Port (point-to-point)• 2 – Loop

wwn_bump Local port’s WWNs are XOR’d with this value prior to registering service parameters with the adapter. This is used to alter the WWN values for the purpose of testing point-to-point negotiations. The value can be hexadecimal (using “0x”) or decimal.




3 Building and Installing the RAMD Drivers3.2 Linux FC/FCoE I+T RAMD Driver (Linux ocs_fc_ramd)

22

3.2 Linux FC/FCoE I+T RAMD Driver (Linux ocs_fc_ramd)

The Linux ocs_fc_ramd driver is an FC/FCoE reference driver that uses the Emulex RAMD target. As a target reference driver, it implements a non-persistent RAM disk backing store. It includes support for several SCSI commands, sufficient to allow most FC/FCoE initiators to discover the RAM disk, log in, and perform I/Os.

The Linux ocs_fc_ramd driver also provides initiator support to the SCSI mid-layer through the SCSI mid-layer shim.

3.2.1 OS RequirementsUse a RHEL 7.1 distribution.

3.2.2 Chip Platform SupportThe ocs_fc_ramd driver supports the Intel x86 family of processors and ARM processors.

For ARM-specific building instructions for Ubuntu, see section 3.2.3.2, ARM-specific Instructions for Ubuntu 14.04, on page 23.

3.2.3 Building

3.2.3.1 General InstructionsTo build the Linux ocs_fc_ramd driver, perform the following steps:

1. In a RHEL 7.1 system that is capable of performing Linux driver builds, extract the ocs_fc_ramd files into a work directory.

# create a build directory

mkdir build_dir

cd build_dir

# extract the tar file

tar xfv <tarfile>

2. Change to the ocs_fc_ramd directory, and build the ocs_fc_ramd driver:# move to the directory containing the Makefile

cd driver/linux/ocs_fc_ramd

# build it

make

The make utility builds the ocs_fc_ramd.ko loadable kernel object module.



23

3.2.3.2 ARM-specific Instructions for Ubuntu 14.04

3.2.3.2.1 Prerequisites

Prior to building the driver for ARM64 on Ubuntu 14.04, two modifications are needed:

Modify the MODULE_LICENSE() declaration from “Proprietary” to “GPL” in the following file:

driver/linux/common/ocs_lnx.c .

This modification is a workaround a temporary problem.

Modify the Linux header file to include/linux/skbuff.h on line 2255 to cast the expression compared to “off” to an int, that is:

(int)(frag->page_offset + skb_frag_size(frag))

3.2.3.2.2 Building

Native Build for ARM64

make

Cross-compile for ARM64

See the following Cross-compile Environment Setup section for steps to set up the cross-compile build environment

make CROSS=arm64 KERN_SRC=<path to ARM64 kernel source>

For example:

KERN_SRC=/opt/ubuntu/linux-3.13.0/debian/build/build-generic/

3.2.3.2.2.1 Cross-compile Environment Setup

Instructions to set up a cross-compile build environment for ARM64 on a x86_64 host running Ubuntu 14.04.

Note that the kernel version is modified as part of the kernel build. For example, the Makefile may show kernel version 3.13.11.4, but the resulting kernel will have a vermagic of 3.13.0-32-generic.

1. Install the build tools, if necessary:fetch a version matching the current kernel

$ sudo apt-get build-dep linux-image-$(uname -r)

or fetch a specific version

$ sudo apt-get build-dep linux-image-3.13.0-36-generic

$ sudo apt-get install kernel-wedge

2. Install the ARM compiler:$ sudo apt-get install gcc-aarch64-linux-gnu

3. Get the Linux source:$ cd /opt/ubuntu

$ apt-get source linux-image-$(uname -r)



24

or

$ apt-get source linux-image-3.13.0-36-generic

4. Install the fakeroot package, unless it is already installed:$ sudo apt-get install fakeroot

5. Build the kernel source:$ cd linux-3.13.0

$ export $(env CC=aarch64-linux-gnu-gcc dpkg-architecture -aarm64)

$ env CROSS_COMPILE=aarch64-linux-gnu- fakeroot debian/rules clean

$ env CROSS_COMPILE=aarch64-linux-gnu- fakeroot debian/rules binary-headers binary-generic

6. Patch skbuff.h:

Modify the Linux header file include/linux/skbuff.h on line 2255 to cast the expression compared to “off” to an int, that is.:

(int)(frag->page_offset + skb_frag_size(frag))

Modify skbuff.h in the following directories:

/opt/ubuntu/linux-3.13.0/include/linux/skbuff.h /opt/ubuntu/linux-3.13.0/debian/linux-headers-3.13.0-36/usr/src/linux-he

aders-3.13.0-36/include/linux/skbuff.h7. Install libxml2-dev to get xml2-config command:

$ sudo apt-get install libxml2-dev

8. Copy ARM64 versions of libxml2, libz, libm and liblzma libraries from an ARM64 system (located under /usr/lib/aarch64-linux-gnu/ and /lib/aarch64-linux-gnu) to the cross-compile system under /opt/usr/lib/aarch64/.

3.2.4 InstallingTo install the Linux ocs_fc_ramd driver, use the following command syntax:

modprobe scsi_transport_fc # midlayer uses scsi_transport_fc

insmod ocs_fc_ramd.ko <module parameter(s)> # add parameters

Module parameters are detailed in the following section.

3.2.4.1 Module ParametersTable 3-2 Linux ocs_fc_ramd Module Parameters

<module parameter> Description

auto_xfer_rdy_size This setting determines when the firmware automatically sends an FCP transfer ready (FCP_XFER_RDY) to the initiator.

It is the maximum size, in bytes, of a write command received from the initiator. For any write command less than or equal to <auto_xfer_rdy_size>, an FCP_XFER_RDY frame is automatically generated by the firmware.

Possible values are 2048 to 65536 (64K). Default is 0 (disabled).



25

ctrlmask Flag control bits to set automatic responses and specific modes. Default value is 10 (bits 1 and 3 are set to 1 by default).• bit[0] - Set to 1 to disable automatic response on target reads. Default

is 0. • bit[1] - Set to 1 to disable automatic response on target writes. Default

is 1.• bit[2] – Reserved. Setting is ignored.• bit[3] - Set to 1 to enable the target registered state change

notification (RSCN). Default is 1.• bit[4] - Set to 1 to enable the driver to always check the GUARD (CRC)

field of the protection information (even if the APPLICATION TAG is 0xFFFF or the REFERENCE TAG is 0xFFFF_FFFF). Default is 0.

• bit[5] - Set to 1 to enable the driver to fill in the REFERENCE TAG and GUARD (CRC) fields of each block when the drive is formatted. Default is 0.

• bit[6] - Reserved. Must be set to 0.bit[7] - Reserved. Must be set to 0.

• bit[8] - Set to 1 to enable Fabric Emulationa mode. Default is 0.• bit[9] - When the driver is loaded as an initiator, set to 1 to force the SLI

Port into a non-initiator mode. Default is 0.• bit[10] - Set to 1 to stop driver initialization after the firmware is reset.

Default is 0.

Example: To disable an automatic response on target reads (bit 0) and target writes (bit 1), and enable Fabric Emulation mode (bit 8), ctrlmask=0x103 (or 259).



This parameter specifies the memory model for T10-PI operations:• For the XE201: When set to 1, the T10 DIF separate model is used. When

set to 0, the inline model is used (default).• For the XE100-series: Only the separate model is supported and


Table 3-2 Linux ocs_fc_ramd Module Parameters (Continued)




26

enable_hlm Enables high login mode (HLM) when set to 1. Default is 0 (disabled). When HLM is disabled, then one remote port indicator (RPI) per login is allocated.

Enabling HLM may be necessary when an application may exceed the SLI-4/adapter RPI constraints. When HLM is enabled, the number of logins is limited only by driver resources (memory).

When HLM is enabled, remote connections that share the same service parameters can share an RPI. After an RPI and service parameters are registered with the adapter, subsequent requests (target send/receive/response requests, initiator send command requests) will specify the destination FC_ID in the request.

If enabled, then RPIs are allocated as follows:• A unique RPI is allocated for each of the first hlm_group_size logins

from remote nodes with compatible service parameters.• Subsequent login requests in excess of the hlm_group_size will use

(share) one of the previously allocated RPIs on a round robin basis.

explicit_buffer_list Host dynamically assigns SGLs as needed when set to 1. Driver pre-registers fixed-value SGLs when set to 0 (default).

external_loopback Enables external loopback mode when set to 1. This mode allows a port to discover and log in to itself when the Tx and Rx signals are connected through a loopback hood. The default is 0 (external loopback mode disabled).


hal_bounceb Note: This parameter is only applicable for the XE201 controller.

A queuing and threading parameter that specifies whether HAL bounce is enabled:• 0 – No bounce (default)• 1 – HAL bounce




hlm_group_size Sets the high login mode group size. Possible values are 1–8192. Default is 8. This low default value is chosen to facilitate testing of the feature; in a typical scenario, it is expected that a higher value would be specified.

holdoff_link_online Keeps the link offline until all PCI devices have been enumerated when set to 1. Default is 0 (disabled).

hw_cmpl_context Indicates where the driver processes hardware completions. When set to 1, completions are handled in a kernel thread (default). When set to 0, completions are handled in a tasklet context.

initiator Enables initiator functionality when set to 1. Default is 0 (disabled).





27

logdest Indicates the location of the logging destination:• bit[0] - Set to 1 to enable system logging (default).• bit[1] - Set to 1 to enable RAM logging. Default is 0 (disabled). Also see

the ramlog_size parameter.

loglevel Indicates the type and amount of log output. Each successive loglevel includes the information of the lower levels. For example, loglevel=3 (default) includes LOG_INFO, LOG_WARN, LOG_ERR, and LOG_CRIT information. Valid values are 0 - 5:• 0 - LOG_CRIT: Logs critical information. The hardware is not usable. The

chip is unresponsive or in a Unrecoverable Error (UE) state.• 1 - LOG_ERR: Logs error information that may prevent normal







logmask Flag bits to enable logging:• bit[0] - Set to 1 to enable node state machine traces.• bit[1] - Set to 1 to enable ELS traces.• bit[2] - Set to 1 to enable SCSI command traces.• bit[3] - Set to 1 to enable SCSI back-end target traces.• bit[4] - Set to 1 to enable domain and sport state machine traces.• bit[5] - Set to 1 to enable the logging of messages with a status of SCSI

Task Set Full (Queue Full) or Busy.

The default value for each of the bits is 0 (that is, logging is disabled by default). Values are also defined in the ocs_debug.h file.










28

num_vports Specifies the number of NPIV ports to create. The default is 0.

The NPIV port WWNN is a type 2 address derived from a factory-supplied MAC address. The NPIV WWPN is derived by OR’ing the NPIV port ID (also known as VPI) into bits 59:48 of the WWNN.


For details on T10 DIF, see Section 4.20, Protection Information Model, in the latest SCSI Block Commands -3 (SBC-3) standard.


queue_topologyb Note: This parameter is only applicable for the XE201 controller.

A queue topology string in the format as described in section 2.4.3.1, Queue Topology Configuration String, in the OneCore Storage Driver Design Manual. The default is eq cq rq cq mq $nulp($nwq(cq wq:ulp=$rpt1)) cq wq:len=256:class=1

ramd_threadingb Note: This parameter is only applicable for the XE201 controller.

A queuing and threading parameter that specifies whether to start RAMD threads:

0 – Do not start RAMD threads

1 – Start RAMD threads (default)



ramlog_size Indicates the size of the RAM logging buffer, an integer in bytes. Default is 1048576 (1M). Also see the logdest parameter.

rq_threads Indicates the number of Receive Queue (RQ) processing threads that the driver creates. Default is 0.

RQ threads are used to distribute the work of incoming SCSI command frames across CPUs. On slow CPU machines (for example, 2.1 GHz), using multiple RQ threads (for example, rq_threads=2) may increase CPU performance. On fast CPU machines (for example, 3.3 GHz), using multiple RQ threads has little to no effect on performance.

speed Sets the link speed. Value is in megabits per second. Possible values include: • 0 – Auto-speed negotiation (default)• 4000 (4GFC)• 8000 (8GFC)• 10000 (10GbE)• 16000 (16GFC)





29




target Enables target functionality when set to 1 (default). Set to 0 to disable.

target_io_timer Note: This parameter is only applicable when the target parameter is 1.

The timeout value, in seconds, for target commands. The default is 0 (target_io_timer disabled).

thread_cmdsb Note: This parameter is only applicable for the XE201 controller.

A queuing and threading parameter that specifies whether to thread RAMD commands:• 0 – Do not thread RAMD commands (default)• 1 – Thread RAMD commands

topology Specifies FC topology:• 0 – Auto-select; first tries N_Port, then loop. (default)• 1 – N_Port (point-to-point)• 2 – Loop

wwn_bump Local port’s WWNs are XOR’d with this value prior to registering service parameters with the adapter. This is used to alter the WWN values for the purpose of testing point-to-point negotiations. The value can be hexadecimal (using “0x”) or decimal.

a. When enabling Fabric Emulation, you must also set topology=1 (N_Port). For details on Fabric Emulation, see section 2.3, Fabric Emulation, in the OneCore Storage Driver Design Manual.

b. For details on queuing and threading, see section 2.4, Advanced Queuing and Threading, in the OneCore Storage Driver Design Manual.





30

3.2.4.2 Module Parameter ExamplesFor a target installation, to create two 100MB LUNs with the default block size (512):

insmod ocs_fc_ramd.ko ramdisc_size=100m num_luns=2

For an initiator installation, to load the Linux ocs_fc_ramd driver with initiator-only support:

insmod ocs_fc_ramd.ko target=0 initiator=1

For Fabric Emulation mode, set the ctrlmask module parameter bit[8] to 1 (enables Fabric Emulation) and topology=1 (N_Port), for example:

insmod ocs_fc_ramd.ko ctrlmask=0x100 topology=1

Note: ocs_fc_ramd initiator integration into the Linux SCSI mid-layer allows the OneCore Storage driver to be used in conjunction with various SCSI initiator tools in Linux (such as sg3_utils and fio).

3.2.5 Creating Device Special FilesIf you need to create device special files for some utilities, they must be created manually after the driver has loaded. A shell script may be used for this purpose. For details, see Section 6.1.1, Device Special Files, on page 54.

3.2.6 Performance OptimizationFor performance optimization, you may want to run the performance tuning script, ocs_config.py, after loading the driver. For details, see Section 6, Linux-Specific Scripts, on page 54.


3 Building and Installing the RAMD Drivers3.3 Linux iSCSI I+T RAMD Driver (Linux ocs_iscsi_ramd)

31

3.3 Linux iSCSI I+T RAMD Driver (Linux ocs_iscsi_ramd)

The Linux ocs_iscsi_ramd driver is an iSCSI reference driver that uses the Emulex RAMD target. For its target support, the driver implements a non-persistent RAM disk backing store. It includes support for several SCSI commands, sufficient to allow most iSCSI initiators to discover the RAM disk, log in, and perform I/Os. The Linux ocs_iscsi_ramd driver also provides initiator support to the Linux SCSI mid-layer through its initiator API and a separate, loadable Open-iSCSI interface (ocs_openiscsi) module (see Section 3.3.4, Open-iSCSI Interface (ocs_openiscsi) Module, on page 36).

3.3.1 OS RequirementsUse the RHEL 7.1 distribution.

3.3.2 Using the iSCSI Target and iSCSI Initiator

3.3.2.1 Using the iSCSI TargetTo use the iSCSI target, load the Linux ocs_iscsi_ramd driver with the appropriate module parameters and ensure the target module parameter is set to 1. See Section 3.3.3, Building and Installing the Linux ocs_iscsi_ramd Driver, on page 32.

3.3.2.2 Using the iSCSI InitiatorTo use the iSCSI initiator, perform the following:

1. Load the libiscsi.ko module.2. Load the Linux ocs_iscsi_ramd driver with the appropriate module parameters and

ensure the initiator module parameter is set to 1 (see Section 3.3.3, Building and Installing the Linux ocs_iscsi_ramd Driver, on page 32):insmod ocs_iscsi_ramd.ko initiator=1

Note: You can load the Linux ocs_iscsi_ramd driver prior to the libiscsi.ko module, however both the libiscsi.ko module and Linux ocs_iscsi_ramd driver must be loaded before the ocs_openiscsi module.

3. Load the ocs_openiscsi module (see Section 3.3.4, Open-iSCSI Interface (ocs_openiscsi) Module, on page 36).

4. Verify that the “ocs” interface is found by Open-iSCSI with the following command:

iscsiadm --mode iface --op show

Example Output:

ocs.00:00:c9:d5:6a:a5.ipv4.0 ocs,00:00:c9:d5:6a:a5,<empty>,<empty>,<empty>

ocs.00:00:c9:d5:6a:a3.ipv4.0 ocs,00:00:c9:d5:6a:a3,<empty>,<empty>,<empty>



32

3.3.3 Building and Installing the Linux ocs_iscsi_ramd Driver

3.3.3.1 BuildingTo build the Linux ocs_iscsi_ramd driver:

1. In a RHEL 7.1 system that is capable of performing Linux driver builds, extract the ocs_iscsi_ramd files into a work directory.

2. Change to the ocs_iscsi_ramd directory and build the ocs_iscsi_ramd driver:make

The make utility builds the ocs_iscsi_ramd.ko loadable kernel object module.

3.3.3.2 InstallingTo install the Linux ocs_iscsi_ramd driver, use the following command syntax:

insmod ocs_iscsi_ramd.ko <module parameter(s)>

The following example creates two 100mb LUNs with the default block size (512):

insmod ocs_iscsi_ramd.ko ramdisc_size=100m num_luns=2

3.3.3.2.1 Module Parameters

Table 3-3 Linux ocs_iscsi_ramd Module Parameters


ctrlmask Flag control bits to set automatic responses and specific modes. Default value is 2 (only bit 1 is set to 1).• bits[0:3] - Reserved. Setting is ignored. • bit[4] - Set to 1 to enable the driver to always check the GUARD

(CRC) field of the protection information (even if the APPLICATION TAG is 0xFFFF or the REFERENCE TAG is 0xFFFF_FFFF). Default is 0.

• bit[5] - Set to 1 to enable the driver to fill in the REFERENCE TAG and GUARD (CRC) fields of each block when the drive is formatted. Default is 0.

• bit[6] - Reserved. Setting is ignored. Default is 0.• bit[7] - Set to 1 to enable iSNS discovery and registration. Default

is 0.




33


This parameter specifies the memory model for T10-PI operations:• For the XE201: When set to 1, the T10 DIF separate model is used.

When set to 0, the inline model is used (default).• For the XE100-series: Only the separate model is supported and


enable_data_digest Enables data digest support when set to 1 (default). Disabled when set to 0.

enable_hdr_digest Enables header digest support when set to 1 (default). Disabled when set to 0.

enable_immediate_data Enables immediate_data functionality when set to 1 (default). Disabled when set to 0.

enable_jumbo_frame Enables jumbo frame support when set to 1 (default). Disabled when set to 0.

enable_sgl_chaining Enables scatter/gather list (SGL) chaining functionality when set to 1 (default). Disabled when set to 0.

gateway When the local_ip parameter is set to a specific static address, this parameter value is the IP address of the gateway. The default is an empty string (no gateway).

This parameter is ignored if local_ip is set to “DHCP” or “AutoV6”.





hw_cmpl_context Indicates where the driver processes hardware completions. When set to 1, completions are handled in a kernel thread (default). When set to 0, completions are handled in a tasklet context.

initiator Enables initiator functionality when set to 1. Default is 0 (disabled).

Table 3-3 Linux ocs_iscsi_ramd Module Parameters (Continued)




34

local_ip Possible values include:

• <x.x.x.x> – the IP address of the device. Default is 20.0.0.106. For static IP addresses, the IP address is incremented for each adapter that is found.

• DHCP – enables the Dynamic Host Configuration Protocol (DHCP), which acquires an IP address automatically for the device.

• <x:x:x:x:x:x:x:x> – an Internet Protocol Version 6 (IPv6) static address of the device. For static IP addresses, the IP address is incremented for each adapter that is found.

• AutoV6 – enables IPv6 Stateless address auto-configuration, which acquires an IP address automatically for the device.

local_subnet Subnetwork for a statically assigned address. Default is 255.255.255.0.

logdest Indicates the location of the logging destination:• bit[0] - Set to 1 to enable system logging (default).• bit[1] - Set to 1 to enable RAM logging. Default is 0 (disabled). Also

see the ramlog_size parameter.

loglevel Indicates the type and amount of log output. Each successive loglevel includes the information of the lower levels. For example, loglevel=3 (default) includes LOG_INFO, LOG_WARN, LOG_ERR, and LOG_CRIT information. Valid values are 0 - 5:• 0 - LOG_CRIT: Logs critical information. The hardware is not usable.

The chip is unresponsive or in a Unrecoverable Error (UE) state.• 1 - LOG_ERR: Logs error information that may prevent normal







max_sge_size The maximum size (in bytes) of the scatter/gather entry. Default is 65536. The value must be in the range of 1024 to 65536.

Used for testing purposes only.






35

num_ramdisc The number of ramdiscs on a PCI device. Default is 1.• 1 - a single global ramdisc is created that is shared between all PCI

devices. In this case, the target name is “iqn.tgt0”.• “n” - where “n” is an integer greater than 1. This means “n” ramdiscs

are created on each PCI device. This is a testing feature to allow more connections from the same number of initiators. The target names are “iqn.tgt[0:(n-1)]”. For example, if n=3, then the target names are iqn.tgt0, iqn.tgt1, and iqn.tgt2.





num_tgts Indicates the number of targets per port. Default is 1. This parameter supersedes the num_ramdisc module parameter (which has been removed).


For details on T10 DIF, see Section “4.20 Protection information model” in the latest SCSI Block Commands -3 (SBC-3) standard.


post_sgl_config Determines the number and version of the POST_SGL ioctl. The following values are supported:• 0 – Use the controller’s specific default value (default).

For example, if post_sgl_config=0 and the Emulex XE100 controller is used, then the setting is 256 SGEs per iSCSI command descriptor (ICD) Version 0.

• 1 – 32 SGEs per iSCSI ICD Version 0.• 2 – 256 SGEs per ICD Version 0• 3 – 32 SGEs per ICD Version 1. Default for the Emulex XE100-series

controller.• 4 – 256 SGEs per ICD Version 1







36

3.3.4 Open-iSCSI Interface (ocs_openiscsi) Module

3.3.4.1 OverviewThe Emulex Open-iSCSI interface (ocs_openiscsi) module is a separate loadable module included in the OneCore Storage SDK package to be used for initiator support with the Linux ocs_iscsi_ramd driver. The files for this module are located in the “driver/linux/ocs_openiscsi/” directory.

The ocs_openiscsi module acts as an interface adapter between the driver’s Initiator API and the Open-iSCSI kernel module (which includes iSCSI transport and libiscsi.ko). The ocs_openiscsi module registers with the Linux ocs_iscsi_ramd driver as an initiator, and then registers with Open-iSCSI as a transport. Afterwards, the Emulex SLI-4 adapters are presented to the SCSI Mid-layer as available SCSI host devices.

3.3.4.2 Building

Note: The Linux ocs_iscsi_ramd driver must be built before building the ocs_openiscsi module.

To unpack and build the ocs_openiscsi module:

1. In a RHEL 7.1 system that is capable of performing Linux driver builds, create a build directory.

mkdir build_dir

cd build_dir

ramlog_size Indicates the size of the RAM logging buffer, in bytes. Default is 1M. Also see the logdest parameter.




target Enables target functionality when set to 1 (default).

tcp_window_scale The TCP window scaling value to use when establishing new connections. Possible values include 0 (disabled) and 4 (default).

tcp_window_size The TCP Window size, in bytes. Default is 65536.

Note: This value is shifted based on the tcp_window_scale value.





37

2. Extract the tar file:tar xfv <tarfile>

3. Move to the directory containing the Makefile and build it:cd driver/linux/ocs_openiscsi

make

3.3.5 Creating Device Special FilesIf you need to create device special files for some utilities, they must be created manually after the driver has loaded. A shell script may be used for this purpose.

For details, see Section 6.1.1, Device Special Files, on page 54.

3.3.6 Performance OptimizationFor performance optimization, you may want to run the performance tuning script, ocs_config.py, after loading the driver. For details, see Section 6, Linux-Specific Scripts, on page 54.

3.3.7 Using CHAPThe ocs_iscsi_ramd driver is currently configured with two hard-coded Challenge-Handshake Authentication Protocol (CHAP) secrets. Only the Message-Digest 5 (MD5) CHAP routine is supported. Table 3-4 shows the CHAP secrets:

3.3.7.1 Enabling CHAP on LinuxTo enable CHAP on Linux using Open-iSCSI, use the following procedure:

1. Open the /etc/iscsi/iscsid.conf file and change the following lines:# *************

# CHAP Settings

# *************

# To enable CHAP authentication set node.session.auth.authmethod

# to CHAP. The default is None.

node.session.auth.authmethod = CHAP

# To set a CHAP username and password for initiator

Table 3-4 CHAP Secrets

username password

EmulexTgt ElxTgtPassword

EmulexIni ElxIniPassword



38

# authentication by the target(s), uncomment the following lines:

node.session.auth.username = EmulexIni

node.session.auth.password = ElxIniPassword

# To set a CHAP username and password for target(s)

# authentication by the initiator, uncomment the following lines:

node.session.auth.username_in = EmulexTgt

node.session.auth.password_in = ElxTgtPassword

2. Restart the iSCSI service.3. Delete any existing connections to targets using the Open-iSCSI administration

utility, iscsiadm:iscsiadm -m node -o delete

4. Log in to the targets. 5. To validate CHAP negotiation on the initiator system, use the following command

to request detailed session information after a successful login:iscsiadm -m session -P2

This command provides print level 2 session information regarding the session, as shown in the following example:

[root@ruger ~]# iscsiadm -m session -P2

Target: iqn.1990-07.com.emulex:tgt2.ramdisc4

Current Portal: 10.0.0.138:3260,1

Persistent Portal: 10.0.0.138:3260,1

**********

Interface:

**********

Iface Name: default

Iface Transport: tcp

Iface Initiatorname: iqn.1994-05.com.redhat:e2d82bc6b124

Iface IPaddress: 10.0.0.114

Iface HWaddress: <empty>

Iface Netdev: <empty>

SID: 1

iSCSI Connection State: LOGGED IN

iSCSI Session State: LOGGED_IN

Internal iscsid Session State: NO CHANGE

*********

Timeouts:

*********

Recovery Timeout: 120

Target Reset Timeout: 30

LUN Reset Timeout: 30

Abort Timeout: 15

*****

CHAP:



39

*****

username: EmulexIni

password: ********

username_in: EmulexTgt

password_in: ********

************************

Negotiated iSCSI params:

************************

HeaderDigest: None

DataDigest: None

MaxRecvDataSegmentLength: 262144

MaxXmitDataSegmentLength: 65536

FirstBurstLength: 8192

MaxBurstLength: 65536

ImmediateData: Yes

InitialR2T: Yes

MaxOutstandingR2T: 1

3.3.7.2 Enabling CHAP on WindowsUse the Microsoft iSCSI Initiator application to configure one-way or mutual CHAP authentication. In one-way CHAP, the target authenticates the initiator. In mutual CHAP, the target authenticates the initiator and the initiator authenticates the target.

Before enabling CHAP, load the ocs_iscsi_ramd driver and have an available target portal.

For one-way CHAP, skip the following section and proceed to Section 3.3.7.2.2, Enabling CHAP for Discovery Sessions, on page 40.

3.3.7.2.1 Mutual CHAP Set-up

For mutual CHAP, you must set the initiator CHAP secret (refer to Figure 3-1) before establishing sessions.

1. Open the Microsoft iSCSI Initiator application and click on the “Configuration” tab.2. Click on “CHAP..” to open the “iSCSI Initiator Mutual CHAP Secret” window.



40

3. Fill in the “Initiator Chap Secret” field with the Emulex hard-coded password “ElxTgtPassword”, and click “OK”.

Figure 3-1 Windows CHAP: Initiator CHAP Secret Field Uses ElxTgtPassword

3.3.7.2.2 Enabling CHAP for Discovery Sessions

Some targets require CHAP to be enabled during discovery. To discover the target portal and enable one-way or mutual CHAP, use the following steps while referring to Figure 3-2.

1. Click on the “Discovery” tab. 2. Click the “Discover Portal” button to open the “Discover Target Portal” window.3. Fill in the IP address or DNS name (“10.0.0.156” in this example).4. Click the “Advanced..” button.5. In the “Advanced Settings” window, select the appropriate “Local Adapter:” and

“Initiator IP:” values. (in this example, Microsoft SCSI Initiator and 10.0.0.110, respectively).

6. Select “Enable CHAP log on”7. Under “CHAP Log on information”, fill in the “Name:” with the Emulex username

“EmulexIni” and the “Target secret:” with the hard-coded password “ElxIniPassword”.



41

8. For mutual CHAP, select “Perform mutual authentication” (as shown in this example). For one-way CHAP, clear the “Perform mutual authentication” box.

Figure 3-2 Windows CHAP: Target Portal Discovery and Enabling Mutual CHAP

9. Click “OK” in the “Advanced Settings” window, and then click “OK” in the “Discover Target Portal” window.

3.3.7.2.3 Enabling CHAP for Normal Sessions

To connect a target and enable one-way or mutual CHAP, use the following steps while referring to Figure 3-3.

1. Click on the “Targets” tab. 2. Select an inactive target (“iqn.1990-07.com.emulex:tgt0.ramdisc0” in this example)

and click “Connect”.3. In the “Connect To Target” window, click “Advanced..”.4. In the “Advanced Settings” window, select the appropriate “Local Adapter:”,

“Initiator IP:”, and “Target port IP:” values (in this example, Microsoft SCSI Initiator, 10.0.0.110, and 10.0.0.156 / 3260”, respectively).



42

5. Select “Enable CHAP log on”6. Under “CHAP Log on information”, fill in the “Name:” with the Emulex username

“EmulexIni” and the “Target secret:” with the hard-coded password “ElxIniPassword”.

7. For mutual CHAP, select “Perform mutual authentication” (as shown in this example). For one-way CHAP, clear the “Perform mutual authentication” box.

Figure 3-3 Windows CHAP: Connecting to Target and Enabling Mutual CHAP

8. Click “OK” in the “Advanced Settings” window, and then click “OK” in the “Connect to Target” window.

9. Verify the CHAP settings for the target (refer to Figure 3-4). a. On the “Targets” tab, select the target that was just connected

(“iqn.1990-07.com.emulex:tgt0.ramdisc0” in this example).b. Click “Properties..”.



43

c. In the “Properties” window, on the “Sessions” tab, check the “Authentication:” status. In this example, “Mutual CHAP” has been enabled. For one-way CHAP, the “Authentication” status is simply “CHAP”.

Figure 3-4 Windows CHAP: Verifying Mutual CHAP

10. Repeat the steps for enabling CHAP in Section 3.3.7.2.2, Enabling CHAP for Discovery Sessions, on page 40 and this section, as needed, for all targets requiring CHAP authentication.


3 Building and Installing the RAMD Drivers3.4 User Space FC/FCoE Target RAMD Driver (Uspace ocs_fc_ramd)

44

3.4 User Space FC/FCoE Target RAMD Driver (Uspace ocs_fc_ramd)

The user space ocs_fc_ramd driver uses the Emulex RAMD target back-end. It implements a non-persistent RAM disk backing store, including support for several SCSI commands, sufficient to allow most FC/FCoE initiators to discover the RAM disk, log in, and perform I/Os.

The user space ocs_fc_ramd driver consists of a user space application (located in the “driver/uspace/ocs_fc_ramd” directory) and a small kernel module (ocs_uspace.ko). The kernel module provides access to PCI resources, as well as managing DMA memory buffers.

There is no initiator functionality provided with this driver.


3.4.2 Unpacking and BuildingTo unpack and build the user space ocs_fc_ramd driver, use the following steps.

1. Create a build directory:mkdir build_dir

cd build_dir


3. Compile the kernel module.a. Move to a directory containing the Makefile:

cd driver/linux/ocs_uspace

b. Build it:make

4. Compile the user space application.a. Move to a directory containing the Makefile:

cd driver/uspace/ocs_fc_ramd

b. Build it:make

This command builds the ocs_fc_ramd application.



45

3.4.3 Installing the Kernel ModuleTo install the kernel module, use the following commands:


insmod ocs_uspace.ko

3.4.4 Running the User Space ApplicationPrior to running the user space application, create the device special files:

cd tools/bin

./ocs_config.py

To run the user space application, use the following commands:

cd driver/uspace/ocs_fc_ramd

./ocs_fc_ramd [parameters]

The driver’s available module parameters are found in the following section or by using the following command:

./ocs_fc_ramd --help

3.4.4.1 Module ParametersThe driver’s module parameters are set using the following command line:

./ocs_fc_ramd --<parameter>=<value>

Note: The module parameters for the user space ocs_fc_ramd driver are the same as for the kernel space Linux ocs_fc_ramd driver, except that the hw_cmpl_context parameter is not supported for the user space ocs_fc_ramd driver.

For user space ocs_fc_ramd module parameter details, see Table 3-2, Linux ocs_fc_ramd Module Parameters, on page 24.

3.4.4.2 Module Parameter ExampleFor a target installation, to create two 100mb LUNs with the default block size (512):

./ocs_fc_ramd --ramdisc_size=100m --num_luns=2

3.4.5 CPU Affinity and Performance OptimizationFor user space drivers, if the uspacemask bit[1]=0 (threads block on events), it is recommended to distribute interrupt handlers across multiple CPU cores.

To insure OCS interrupt handlers are distributed across CPU cores and to optimize IOPS performance, you may want to run the performance tuning script, ocs_config.py, after loading the driver. For details, see Section 6, Linux-Specific Scripts, on page 54.



46

3.4.6 Terminating the User Space ApplicationAn interrupt or kill signal (Ctrl-c) terminates the user space driver. When the kernel module senses that the user space application is terminating, it will reset the ports and free DMA buffers.


3 Building and Installing the RAMD Drivers3.5 User Space iSCSI Target RAMD Driver (Uspace ocs_iscsi_ramd)

47

3.5 User Space iSCSI Target RAMD Driver (Uspace ocs_iscsi_ramd)

The Uspace ocs_iscsi_ramd driver is a Linux iSCSI driver that uses the Emulex RAMD target back-end. It implements a non-persistent RAM disk backing store, including support for several SCSI commands, sufficient to allow most iSCSI initiators to discover the RAM disk, log in, and perform I/Os.

The Uspace ocs_iscsi_ramd driver consists of a user space application (located in the “driver/uspace/ocs_iscsi_ramd” directory) and a small kernel module (ocs_uspace.ko). The kernel module provides access to PCI resources, as well as managing DMA memory buffers.

There is no initiator functionality provided with this driver.


3.5.2 Unpacking and BuildingTo unpack and build the Uspace ocs_iscsi_ramd driver, use the following steps.

1. Create a build directory:mkdir build_dir

cd build_dir


3. Compile the kernel module.a. Move to a directory containing the Makefile:


b. Build it:make

4. Compile the user space application.a. Move to a directory containing the Makefile:

cd driver/uspace/ocs_iscsi_ramd

b. Build it:make

The make utility builds the ocs_iscsi_ramd application.



48

3.5.3 Installing the Kernel ModuleTo install the kernel module, use the following commands:


insmod ocs_uspace.ko

3.5.4 Running the User Space ApplicationPrior to running the user space application, create the device special files:

cd tools/bin

./ocs_config.py

To run the user space application, use the following commands:

# as root

cd driver/uspace/ocs_iscsi_ramd

./ocs_iscsi_ramd [parameters]

The driver’s available module parameters are found in the following section or by using the following command:

./ocs_fc_ramd --help

3.5.4.1 Module ParametersThe driver’s module parameters are set using the following command line:

./ocs_iscsi_ramd --<parameter>=<value>

Note: The module parameters for the user space ocs_iscsi_ramd driver are the same as for the kernel space Linux ocs_iscsi_ramd driver, except that the hw_cmpl_context parameter is not supported for the user space ocs_iscsi_ramd driver.

For user space ocs_iscsi_ramd module parameter details, see Table 3-3, Linux ocs_iscsi_ramd Module Parameters, on page 32.

3.5.4.2 Module Parameter ExampleFor a target installation, to create two 100mb LUNs with the default block size (512):

./ocs_iscsi_ramd --ramdisc_size=100m --num_luns=2



49

3.5.5 CPU Affinity and Performance OptimizationFor user space drivers, if the uspacemask bit[1]=0 (threads block on events), it is recommended to distribute interrupt handlers across multiple CPU cores.

To insure OCS interrupt handlers are distributed across CPU cores and to optimize IOPS performance, you may want to run the latest performance tuning script, ocs_config.py, after loading the driver. For details, see Section 6, Linux-Specific Scripts, on page 54.

3.5.6 Terminating the User Space ApplicationAn interrupt or kill signal (Ctrl-c) terminates the user space driver. When the kernel module senses that the user space application is terminating, it will reset the ports and free DMA buffers.

3.5.7 Using CHAPThe Uspace ocs_iscsi_ramd driver is currently configured with two hard-coded CHAP secrets.

For more details, such as enabling CHAP, see Section 3.3.7, Using CHAP, on page 37.


4 User Space ioctl() RPC Interface4.1 Overview

50

4 User Space ioctl() RPC Interface

4.1 OverviewIn a OneCore Storage kernel space driver, external programs may interact using the ioctl() filesystem interface and API. This ioctl() filesystem interface provides calls into the kernel space driver for executing various management and other functions (such as driver status, firmware download, and reading dump files). Since the user space driver runs as a user process, by default, there is no built-in interface similar to the filesystem ioctl() interface.

To provide a similar interface as part of the user space driver, with minimal impact on the programs that previously used the ioctl() interface (for example, the elxsdkutil utility), a sockets-based remote procedure call (RPC) interface is implemented.

The user space ioctl() RPC interface functionality includes the following:

As each port (PCI function) is initialized, a server thread (ocs_ioctl_server) is created to wait on a socket for external ioctl() requests. The socket is initialized to listen on port [9000 + the device instance index].

An external program establishes a connection to the server, formats an RPC request, and transmits an RPC request. The request format encapsulates an ioctl() call, using a command request value, associated structure data, and any caller supplied outbound data buffers. The server executes the command on behalf of the caller, and returns the command request with status, along with any server originated data buffers.

Note: The system that issues the request does not need to be the same system that the user space driver is running on.

4.2 elxsdkutil --host OptionNormally, the elxsdkutil utility opens the local device special file /dev/ocs_<n> to access the ioctl() filesystem interface. For user space drivers, the elxsdkutil utility includes a Host option. The Host option is used on the command line to specify the host name or IP address of the system on which the user space driver is running. This option is required on all commands which access a device. The Host option command line syntax is:

--host <hostname>

4.2.1 ExamplesList all devices on the user space driver on the local machine:

elxsdkutil list –-host localhost


4 User Space ioctl() RPC Interface4.2 elxsdkutil --host Option

51

Download the firmware to device 1 of the user space driver running on machine 192.168.1.153:

exlsdkutil write –-host 192.168.1.153 –d 1 –f fwfile

Download the firmware to the user space driver running on the machine facebook.com, port base 8000 instead of 9000:

elxsdkutil write –-host facebook.com:8000 –f fwfile


5 Building and Installing the elxsdkutil Utility5.1 Prerequisites

52

5 Building and Installing the elxsdkutil UtilityThe Emulex OneCore Storage SDK utility (elxsdkutil) and source code provide a working example of common management operations, including firmware download, diagnostic dumps, device queries, and so on.

The elxsdkutil utility can be built and used in conjunction with the FreeBSD and Linux drivers described in this document. As such, it can be built and used on the operating system distribution specified with the selected driver.

For more details on the elxsdkutil and its commands, see the OneCore Storage Utilities Manual.

5.1 PrerequisitesFor licensed RHEL 7.1 systems, you must manually install additional packages prior to building the elxsdkutil utility. Run the following command to add the necessary XML library packages:

yum install libxml2 libxml2-devel

For non-licensed RHEL 7.1 systems, you must also manually install any required dependent packages.

5.2 BuildingTo build the elxsdkutil utility:

1. Extract the elxsdkutil files into a work directory.2. Change to the elxsdkutil directory, and build the utility:

make

This command builds the elxsdkutil user space application.

5.3 InstallingNo installation is needed for the elxsdkutil utility.

To get a list of available commands and examples, use the following command:

elxsdkutil help

To get the usage for a particular command, type

elxsdkutil help <command>

For example:

elxsdkutil help connection-info

Output:

Command: connection-info [-d <device>] [--host <hostname>] [--connection-handle <handle>]


5 Building and Installing the elxsdkutil Utility5.3 Installing

53

Usage: If a connection handle is provided, reports information on that connection. If no connection handle is provided, displays a list of open connections.

5.3.1 Device Special FilesThe elxsdkutil utility file creates device special files if needed.


6 Linux-Specific Scripts6.1 ocs_config.py Script

54

6 Linux-Specific ScriptsNote: This section is only applicable to OneCore Storage Linux drivers only.

6.1 ocs_config.py ScriptThe ocs_config.py script, located in the tools/bin directory performs the following:

Creates device special files. Employs performance tuning settings in multi-core environments:

Specifies interrupt and thread CPU affinity settings. Sets the CPU frequency scaling governor to performance mode, if

appropriate. Uses OCS driver port to non-uniform memory access (NUMA) node

mapping to minimize latency by running OCS driver processes on local memory (versus remote memory).

Note: The ocs_config.py script should be run after the driver is loaded.

See the following sections for further details.

6.1.1 Device Special FilesThe Linux drivers for the SDK 3.4 release, and later, no longer have the capability to create device special files. If you need to create device special files for some utilities or other user space applications, they must be created manually after the driver has loaded. A shell script may be used for this purpose.

6.1.1.1 Creating Device Special Files ManuallyThe Linux drivers create a procfs entry (that is, “/proc/ocs_*” for kernel space drivers and “/proc/ocsu” for user space drivers), which produces lines of comma-separated values when read. The first line is a pair, where the first value is the major device number and second value is the number of instances. The second and subsequent lines contain three comma-separated values: the first is the devices instance, the second is the interrupt vector, and the third is the thread process ID (if threading is enabled, otherwise, it is “-1”). Using this information, the device special files can be created by issuing appropriately-formatted mknod commands.

The following example shows the values for a Linux kernel space driver, four ports:

247,4

0,113,9429

1,114,9430

2,115,9431

3,116,9432


6 Linux-Specific Scripts6.1 ocs_config.py Script

55

In this example, the device major file is 247, for four device instances. The following command could be used to create the device special files:

mknod /dev/ocs_0 c 247 0




The ocs_config.py script can be used to create device special files for OneCore Storage Linux drivers.

6.1.2 Performance TuningPerformance tuning settings for drivers used in multi-core environments have been incorporated in the ocs_config.py or ocs_config.sh script. For details, see the OneCore Storage Performance Tuning Application Note (April 2015, or later version), included as in the OneCore Storage SDK 10.4 Release (or later package).


Appendix A. Module Parameter ExtrasA.1 Module Parameters Per Driver Comparison

56

Table A

Modul_iscsicst

auto_x

ctrlma

ddump_

dif_se

enable

enable

enable

enable

enable

enable

explic

extern

gatewa

global

hal_bo

hal_wa

hlm_gr

hw_cmp

holdof

initia

local_

local_

logdes

loglev

Appendix A. Module Parameter Extras

A.1 Module Parameters Per Driver ComparisonTable A-1 shows the comparison of parameters between the various OneCore Storage drivers.

-1 Module Parameter Comparison Between OneCore Storage Drivers

e Parameter

BSD Drivers Linux Drivers

ocs_fc_

camocs_fc_

ramdocs_fc_ramd

ocs_iscsi_ramd

Uspaceocs_fc_

ramd

Uspaceocs_iscsi_

ramd

ocs_fc_

lioaocs_fc_

scstocs

_s

fer_rdy_size

sk

saved_size

parate

_data_digest

_hdr_digest

_hlm

_immediate_data

_jumbo_frame

_sgl_chaining

it_buffer_list

al_loopback

y

_ramdisc

unceb

r_version

oup_size

l_context

f_link_online

tor

ip

subnet

t

el


Appendix A. Module Parameter ExtrasA.1 Module Parameters Per Driver Comparison

57

logmas

max_sg

num_lu

num_sc

num_tg

num_vp

p_type

post_s

queue_

ramd_t

ramdis

ramdis

ramlog

rq_thr

speed

stub_r

target

target

tcp_wi

tcp_wi

thread

topolo

uspace

wwn_bu

a. Theb. Thi

Table A

Modul_iscsicst

k

e_size

ns

si_ios

ts

orts

gl_config

topologyb

hreadingb

c_blocksize

c_size

_size

eads

es6_rel6

_io_timer

ndow_scale

ndow_size

_cmdsb

gy

mask

mp

ocs_fc_lio driver is in technical preview.s parameter is only applicable for the XE201 controller.

-1 Module Parameter Comparison Between OneCore Storage Drivers (Continued)

e Parameter

BSD Drivers Linux Drivers

ocs_fc_

camocs_fc_

ramdocs_fc_ramd

ocs_iscsi_ramd

Uspaceocs_fc_

ramd

Uspaceocs_iscsi_

ramd

ocs_fc_

lioaocs_fc_

scstocs

_s



Appendix B. Viewing Parameter Values

58

Appendix B. Viewing Parameter ValuesTo view the values of the driver’s module parameters at driver load, use the “driver-info” command with the elxsdkutil utility. This command displays information about a driver when it is initially loaded, such as the driver’s version (driver_version), system power state (pm_state), and initial module parameter values. For more details, see the latest OneCore Storage Utilities Manual.

Example

Retrieve the information for the kernel space Linux ocs_fc_ramd driver that was loaded as a target:

elxsdkutil driver-info

Output:

driver_version: 0.0.9999.0 pm_state: S0 auto_xfer_rdy_size: 0 ctrlmask: 0xa ddump_saved_size: 0 dif_separate: false enable_hlm: false explicit_buffer_list: false external_loopback: false global_ramdisc: 1 hal_war_version: (null) hlm_group_size: 0 holdoff_link_online: false hw_cmpl_context: 1 initiator: false logdest: 1 loglevel: 3 logmask: 0x0 num_luns: 1 num_scsi_ios: 8192 p_type: 0 ramdisc_blocksize: 512 ramdisc_size: 50M ramlog_size: 1048576 rq_threads: 0speed: 0 stub_res6_rel6: false target: true target_io_timer: 0 topology: 0 wwn_bump: 0

onecore™ storage sdk quick start guide - broadcom · linux ocs_fc_ramd linux fc/fcoe initiator...

Documents