CEE-SECR 2010October 13-15. Moscow, Russia

Language-based Implementation of Multi-Vendor Support in an AdvancedTCA Shelf ManagerSergey Zhukov

2

CEE-SECR 2010

Introduction

Auriga/Pigeon Point Systems (PPS) collaboration

Dedicated engineering team for more than 10 years

Specialized in management for telecommunications systems based on open

standards

Current project: management for AdvancedTCA systems

AdvancedTCA (ATCA) – an open standard for multi-board systems (shelves) used in

telecommunications, developed by PICMG standards body in 2003

AdvancedTCA boards can be single-board computers or specialized boards (DSPs,

storage boards)

Shelf can host boards from different vendors

Management based on IPMI management specification from Intel

MicroTCA – a smaller form-factor variant of AdvancedTCA for use in other areas

(industry, medical)

3

CEE-SECR 2010

Management Architecture in AdvancedTCA

4

CEE-SECR 2010

Pigeon Point Shelf Manager

Hardware and software solution for management in AdvancedTCA

shelves

Implemented on a small mezzanine single-board computer (ShMM)

Runs embedded Linux

Shelf Management software runs as a user-mode application

Needs vendor-specific carrier board to plug into the shelf

Interface between a carrier board and a shelf is vendor-specific

Device population of a carrier board is vendor-specific

Shelf Manager operates with carrier boards and shelves from

different vendors

5

CEE-SECR 2010

An AdvancedTCA Shelf with Shelf Manager

6

CEE-SECR 2010

Shelf Manager Software Adaptation Problem

To provide multi-vendor support, Shelf Manager software needs

adaptation to a specific carrier and/or shelf

Previously: carrier- and shelf-specific pieces of code in the Shelf

Manager software

Does not scale well when number of supported vendors grows, which

happened over time

However the most practical solution for not very powerful original ShMMs

But in newer ShMM models processing power increased

A language-based solution to this problem has been introduced

7

CEE-SECR 2010

Language-Based Solution

Define a language for describing carrier and shelves

Hardware Platform Definition Language - HPDL

Each language unit (module) describes a carrier or a shelf

Created by PPS engineers or by shelf/carrier vendors

Modules compiled into binary format by a special compiler

Binary format loaded and interpreted by the Shelf Manager

software

8

CEE-SECR 2010

Storing Binary HPDL modules

Preferred location – Carrier or Shelf FRU Information storage

AdvancedTCA/IPMI standard storage

Collocated with the respective carrier or shelf

Coexists with other (standard) records that describe carrier and

shelf

Carrier and shelf become self-describing

Stored together with sensor data records (SDRs)

SDRs are standardized descriptions of IPMI sensors

Shelf Manager uses HPDL descriptions and SDRs together

9

CEE-SECR 2010

HPDL Features

Describes following objects:

Shelf Manager carrier board

Other modules on the carrier board (mezzanines, RTMs)

Non-intelligent modules in the shelf under direct control of the

Shelf Manager (power supplies, fan trays, alarm panels)

I/O devices on the carrier board and in the shelf, controlled by

the Shelf Manager (mostly I2C devices)

For each device, accessible signals and expressions over these

signals are defined

IPMI sensors on the carrier board and in the shelf, bound to I/O

devices, signals and expressions

10

CEE-SECR 2010

HPDL Object Hierarchy

1

*

FRU

Fan Tray

Device Signal

Sensor

1 *

1

*

**

Expression

1

*

11

CEE-SECR 2010

Describing FRUs

FRU as an object is defined in the AdvancedTCA specification

Hot swappable: can be replaced dynamically

Power Management: dynamically powered on and off

Supports other AdvancedTCA features – payload reset, etc.

Carrier board, non-intelligent modules, mezzanines are represented as

FRUs

In HPDL, described via “FRU” statement with subordinate clauses and

sub-statements

Unique name

Presence expression: indicates physical presence of the FRU

Hot Swap latch expression and LED signal

12

CEE-SECR 2010

Describing FRUs (2)

More clauses for “FRU” statement

FRU Information location (normally an EEPROM)

Signals for other standard and custom LEDs (besides Hot Swap)

Initial and periodic actions (local FRU management policies)

List of device descriptions for devices that belong to the FRU

List of IPMI sensor descriptions for sensors that belong to the

FRU

Amount of power consumed by the FRU (used in power budget

calculation and management)

Some other spec-defined FRU properties

13

CEE-SECR 2010

Describing Fan Trays

Fan tray is a specific case of FRU

Includes one or more fans that are used for shelf cooling

In HPDL, defined by “FAN TRAY” statement

Includes all clauses of the “FRU” statement, plus:

Fan speed properties: minimum, maximum and recommended

speed level

A signal or set of signals to set fans to a certain speed level

A signal to cause emergency shutdown (in case of fire in the

shelf)

List of fan speed (tachometer) sensors, associated with each fan

14

CEE-SECR 2010

Describing Devices

I/O devices on the Carrier board and in the shelf

Directly controlled by the Shelf Manager (without IPMI)

Mostly I2C devices, with some exceptions (direct ShMM GPIOs)

Each device is associated with some FRU

In HPDL, defined by “DEVICE” sub-statement

In the scope of the corresponding “FRU” statement

Includes the following clauses:

Device type

Device bus number and address

List of used signals

15

CEE-SECR 2010

Describing Signals

Set of signals represents the device interface

A signal can represent

An integer register

A single bit in a register

A data storage (e.g. an EEPROM)

Signal can be read-write, read-only or write-only

Internal signals a for device are defined by device type

HPDL maps internal signals to externally visible signals

Directly or using expressions

Defined by “SIGNALS” clause

In the scope of the corresponding “DEVICE” statement

16

CEE-SECR 2010

Handling Devices in the Shelf Manager

Some devices need an HPDL driver

A piece of code in the Shelf Manager software

Defines internal signals that can be exposed

Provides access to the actual device via the signals

Some I2C devices do not need a special driver

Use generic I2C driver in the Shelf Manager software

Exposes 256 byte-sized signals (mapped to registers) and 256*8

one-bit signals

Access via Read Byte/Write Byte SMB commands

17

CEE-SECR 2010

Describing Sensors

AdvancedTCA uses IPMI sensor model

Sensor properties described in Sensor Data Records (SDRs)

Analog sensors

Expose 1-byte raw value

Translated to actual value using formulas in SDR

Support thresholds and hysteresis

Discrete sensors

Up to 15 states, can be set simultaneously

Settable sensors: value/states can be set via IPMI commands

HPDL: mapping device signals to IPMI sensors

To make signals accessible via standard IPMI mechanisms

18

CEE-SECR 2010

Describing Sensors (2)

Described by “SENSORS” clause in a “FRU” statement

For each sensor, following attributes are specified

Sensor number and LUN (IPMI addressing information)

Sensor value as expression or expressions over signals

Initial state for discrete sensors

Polling interval in milliseconds

For settable sensors, signals to set when the value is set externally

SDRs are compiled separately and stored in a parallel data storage

SDR binary format defined by the IPMI specification

Compiled by already existing (reused) compiler

When implementing sensors, Shelf Manager software uses both HPDL and SDRs

SDRs provide sensor properties, HPDL provides binding to hardware

19

CEE-SECR 2010

HPDL Example

FRU Carrier_FRU {

SITE 3, 1;

...

POWER_LEVELS { 10000; };

FRU_INFORMATION MasterADM1026.EEPROM;

DEVICES {

DEVICE Multiplexer {

PCA9545: 0: 0x70;

SIGNALS { Channels:

Channels, "1", "2", "3", "4"; };

};

DEVICE MasterADM1026 {

ADM1026: 0: 0x2e;

SIGNALS {

Temp1: IntTemp;

Temp2: ExtTemp1;

Temp3: ExtTemp2;

St_3_3V: Stdby_3_3V;

Main_3_3V: Main_3_3V;

Plus_5V: Plus_5V;

Minus_12V: Minus_12V;

Vbat: Vbat;

# Tachometer signals 0 to 7

Tachometer0: Tachometer0, "8";

...

Tachometer7: Tachometer7, "8";

# GPIOs: input, active low

PwrGoodBus: GPIO12, in, low;

PwrGoodAcb: GPIO13, in, low;

FanControl: PWM;

FanLocalControl: PWM_AFC;

EEPROM: EEPROM;

};

};

}; # End of DEVICES statement

20

CEE-SECR 2010

HPDL Example (2)

... SENSORS {

0x10,0,2: 3000: MasterADM1026.Temp1;

0x10,0,3: 3000: MasterADM1026.St_3_3V;

0x10,0,6: 3000: MasterADM1026.Vbat;

0x10,0,15: 3000: MasterADM1026.PwrGoodBus;

0x10,0,16: 3000: MasterADM1026.PwrGoodAcb;

# Expression used for the next sensor

0x10,0,17: 3000: !MasterADM1026.PwrGoodBus || MasterADM1026.PwrGoodAcb;

}; # End of SENSORS statement

}; # End of FRU statement

21

CEE-SECR 2010

ASN.1 as Binary HPDL Format

ASN.1 BER (optionally compressed) as format for binary data

Produced by HPDL compiler, parsed by the Shelf Manager software

Motivations

Backward compatibility (new Shelf Manager interprets old HPDL modules)

Forward compatibility (old Shelf Manager interprets new HPDL modules)

Both can be achieved using ASN.1 tagged encoding

Compactness of representation (EEPROMs are small)

Decoding can be done efficiently (limited resources on ShMM)

Availability of libraries that encode and decode BER

22

CEE-SECR 2010

HPDL Compiler Implementation

Command-line program in two versions: for

Windows and for Linux

Implemented in C++

HPDL defined by a context-free grammar

Bison used as a parser generator

Can be compiled by GCC for Linux, by MSVC for Windows

Special tools to convert output to IPMI FRU Information

format

23

CEE-SECR 2010

HPDL Introduction: Case Study: Client A

Shelf and Carrier Board vendor, collaborates with PPS for a long time

Carrier-specific code has been used to support their carriers and shelves

Initialization parameters were used to specify actual carrier and shelf model

Gradual transition to HPDL for this client

HPDL now used to support all new shelf models and new generation Carrier boards

Shelves became self-describing, no initialization parameters needed

Carrier-specific code still exists to support legacy hardware but is not actively

maintained recently

HPDL modules are developed by PPS engineers, Client engineers have a working

knowledge of them and sometimes make changes (in coordination with PPS)

Many change requests from the client affect HPDL modules only and do not require

a new release of the Shelf Manager

24

CEE-SECR 2010

HPDL Introduction: Case Study: Client B

Another shelf and Carrier board vendor, recently came to

AdvancedTCA market

HPDL was already used in PPS

No carrier-specific code for this client, only HPDL

HPDL modules developed by engineers on the client side

Made accessible to PPS to review and store in the common

repository of HPDL modules

Client is self-sufficient with respect to support of their

hardware with PPS management solutions

25

CEE-SECR 2010

Conclusion. Advantages of Using HPDL

No need of special Shelf Manager release to

support new carrier boards/shelves

Dramatically reduced effort to implement support

for new carrier boards/shelves in PPS

Carrier and shelf vendors can themselves use

HPDL for their products and take support activity

in their own hands

26

CEE-SECR 2010

Resources

Auriga Inc. http://www.auriga.com

Pigeon Point Systems http://www.pigeonpoint.com

PICMG http://www.picmg.org

IPMI http://www.intel.com/design/servers/ipmi

27

CEE-SECR 2010

Q&A; Auriga At a Glance

World leader in R&D outsourcing

Global Services 100 and Global Outsourcing 100 company

Top 10 Offshore Eastern and Central Europe In business since 1990—first in Russia Incorporated in the U.S. in 1993 5 engineering centers in Russia—Moscow (3), Nizhny

Novgorod (1), Rostov-on-Don (1) Engineering center in EU (Vilnius, Lithuania) Close relationship with the leading Chinese vendors 250+ FTEs with low attrition & rotation CMMI Level 4 company SPICE (ISO 15504) assessed

Life critical software compliant Focus on software R&D and product engineering for high-

tech clients Leader in system-level & embedded development

services Proficient in Web & enterprise applications Mobile technologies & Knowledge management

solutions as outposts of further expertise development