cisco connected assets 1.0 design guide · i asset vision 1.0 design guide design guide preface the...

Cisco Connected Assets 1.0 Design GuideSeptember 2015

Building Architectures to Solve Business Problems

ii

About Cisco Validated Design (CVD) Program

The CVD program consists of systems and solutions designed, tested, and documented to facilitate faster,

more reliable, and more predictable customer deployments. For more information visit http://

www.cisco.com/go/designzone.

ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY,

"DESIGNS") IN THIS MANUAL ARE PRESENTED "AS IS," WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL

WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTIC-

ULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRAC-

TICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR

INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING

OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF

THE POSSIBILITY OF SUCH DAMAGES.

THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR

APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL

ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS

BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of

California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved.

Copyright © 1981, Regents of the University of California.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other

countries. To view a list of Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks. Third-party trade-

marks mentioned are the property of their respective owners. The use of the word partner does not imply a partner-

ship relationship between Cisco and any other company. (1110R).

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual

addresses and phone numbers. Any examples, command display output, network topology diagrams, and other fig-

ures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone num-

bers in illustrative content is unintentional and coincidental.

Cisco Connected Assets 1.0 Design Guide

© 2015 Cisco Systems, Inc. All rights reserved.

Asset Vision 1.0 Design Guide

http://www.cisco.com/go/trademarks

http://www.cisco.com/go/designzone

iv

Contents

C H A P T E R 1 System Overview 1-1

Solution Use Cases 1-2

Partner Information 1-3

Cisco Solution USP 1-3

C H A P T E R 2 System Architecture (or System Design) 2-1

Major Architectural Requirements 2-1

End-to-End System Architecture 2-2

End-to-End System Functional Blocks and Message Flow 2-2

Cell Site Architecture 2-3

Cell Site Functional Blocks 2-3

Sensors Interface 2-4

RS485 2-4

Modbus 2-4

Sensors and Modbus Adapters 2-5

IP Cameras 2-5

Real and Virtual Sensors 2-5

Sensor Gateway and Fog Computing System 2-5

Fog Computing Considerations 2-6

Limitations of IR910 2-6

Interface between Cell Site Components and NOC 2-6

Sensor Interface 2-6

Camera Interface 2-7

Cell Site Network Architecture 2-7

IP Address and VLAN Considerations at the Cell Site 2-7

VLAN Configuration 2-7

IP Address 2-7

Cell Site Deployment Models 2-8

Deployment Model-1 2-8

Deployment Model-2 2-9

WAN Connectivity 2-11

WAN Edge Router 2-11

WAN Connectivity Failure Detection 2-11

NOC Cloud Architecture 2-11

Connected Assets NOC Functional Blocks 2-12

NBI 2-12

Security Architecture 2-13

Network Security 2-13

1Asset Vision 1.0 Design Guide

Design Guide

Contents

Application Security 2-14

Cell Site Security 2-14

Other Security Considerations 2-14

QoS Architecture 2-14

Network QoS 2-14

Application QoS 2-15

Message Delivery Priority 2-15

Guaranteed Message Delivery (MQTT QoS) 2-15

Protecting the Client from Storage Overflow during Server Link Down 2-16

Connected Assets Asset Manager 2-16

Browser Requirements 2-17

Notification E-Mails 2-17

CAM Functionality - Reports and Analytics 2-18

Representative Set of Reports 2-18

Connected Assets System Requirements 2-19

Azeti Control Panel Functionality 2-19

C H A P T E R 3 System Components 3-1

Cisco Products 3-1

Third-Party Products 3-2

Modbus and Non-Modbus Sensors and Applications 3-2

C H A P T E R 4 System High Availability and Scalability 4-1

System Redundancy and High Availability 4-1

Redundancy at the Edge Components 4-1

Redundancy for Communication Link 4-1

One-Box-Model 4-1

Two-Box-Model 4-1

Redundancy for Server Applications in the Cloud 4-1

Scalability and Bandwidth Requirements 4-2

Server Sizing Details 4-2

Server Sizing and Bandwidth Recommendations 4-3

2Asset Vision 1.0 Design Guide

Design Guide

Preface

The Cisco Connected Assets solution has been developed to easily connect different types of sensors, and to leverage the capabilities of edge and cloud-based processing to provide business intelligence insights to help the customers optimize operations and improve productivity efficiency. This Connected Assets CVD 1.0 includes asset management of telecom cell towers and managing assets of other similar remote sites. This solution will help operators monitor the status of assets in remote sites from a central location and also control many aspects; thus improving service response time and reducing OPEX.

NavigatorThis document is set up in the following chapters:

• Chapter 1, “System Overview,” introduces the Cisco Connected Assets Validated Design (CVD) 1.0 solution and its use cases.

• Chapter 2, “System Architecture (or System Design)” describes the end-to-end system architecture, giving design considerations for various aspects such as network connectivity, security, requirement, and the Network Operating Center (NOC).

• Chapter 3, “System Components” lists Cisco and third-party components at the cell site and the NOC.

• Chapter 4, “System High Availability and Scalability” provides a system functional description.

Document Objective and ScopeThis CVD provides a comprehensive explanation of Cisco Connected Assets 1.0 system design. It includes information about the system's architecture, possible deployment models, and guidelines for implementation and configuration. It recommends best practices and potential issues when deploying the reference architecture.

Abbreviations NOC: Network Operation Center

CAM: Cisco Asset Manager

ACP: azeti Control Panel

SG: Sensor Gateway

iAsset Vision 1.0 Design Guide

Design Guide

ii

Design Guide

C H A P T E R 1
System Overview
This CVD addresses the design and implementation details of Cisco Connected Assets. Connected Assets solutions can be used for various applications with remote site assets. Typical examples include the following:

• Cell towers, Cell on Light Truck, Cell on Wheel (COLT/COW)

• Utility substations

• Oil and gas pump sites

• Remote branch offices

The Connected Assets solution for asset management applied to cell towers is detailed here as an example.

The Connected Assets service can help service providers secure, operate, and optimize their cell tower infrastructure. The wealth-creating ability of mobile telecommunications operators and tower companies, for example, depends on the cost, effectiveness, and availability of their networks that are composed of thousands of remote and unattended base transceiver stations (BTS). The success of many operations in telecommunications and other industries relies on the remote monitoring of the performance of these costly assets.

BTS sites are exposed to hazards such as theft, destruction, harsh weather conditions, and malfunctions of equipment. Having intelligence on the tower sites helps to keep them up and running and financially viable. BTSs often have a suboptimal performance level due to inefficient resource allocation. Enhanced remote management capabilities are the basis for substantial cost savings as well as improvements in efficiency and utilization of a given site infrastructure.

The Connected Assets services consist of a secure and manageable hardware and software solution. The solution offers energy reporting and management, remote monitoring and management, systematic actions from well-defined events, pre-defined NOC interfaces, and an easy-to-use configuration to accelerate deployment.

When successfully deployed, this solution provides a single view of all assets located at the Site and detailed information on telemetry data. The solution is comprised of a combination of the functionality listed below:

• Site Security

• Advanced Analytics

• Environmental Monitoring

• Detailed Reporting

• HVAC Management

• Security Management

1-1Asset Vision 1.0 Design Guide

Chapter 1 System Overview Solution Use Cases

• Access Management

• Power Monitoring

• Multi-tenant Power Monitoring

• Energy Usage and Optimization by Asset-type

• Fuel Monitoring

• Data Visualization

• Other IoT sensors data with the capability of analytics, reporting and visualizations

Solution Use CasesThis CVD 1.0 addresses the following use cases:

• Environmental Monitoring (Operational Continuity)

– A number of environmental sensors such as temperature and humidity sensors as well as water intrusion, smoke detectors, and hazardous gas sensors are connected on the site. These sensors send real-time alarms to the NOC when thresholds are crossed.

– The remote environmental monitoring helps to gather detailed data on environmental parameters for further analysis.

– Determination of optimal environmental settings.

• Security Management (Operational Continuity)

– Surveillance of the whole tower site with the help of cameras and motion detection sensors, triggering alarms in case of unauthorized access.

– Detection of theft of earthing (grounding) systems; thus increasing the security of the tower site in case of a lightning strike.

• Access Management (Operational Efficiency)

– Keypads that allow keyless entry and logging access to the site.

– Work time-stamp control.

– Remotely-granted access to sites from the NOC.

• Power Monitoring (Operational Continuity)

– Monitors the battery, the generator, and the rectifier and provides their parameters.

– Predefined thresholds that generate alarms to the NOC.

– Data and energy logging that allows for detailed breakdown of energy consumption for comprehensive business analytics.

• Multi-tenant Power Monitoring (Operational Efficiency)

– Multi-tenant site operations, meaning that different Telco operators use one tower site and are paying only for the energy they have consumed.

– Efficient use of current tower site infrastructure.

– Detailed power consumption per tenant per tower site; export to billing systems, and generating performance analysis reports.

• Fuel Monitoring (Operational Continuity)


Design Guide

Chapter 1 System Overview Partner Information

– Fuel level sensors that determine when the next refill is due as well as to detect theft by tapping, therefore helping to prevent generator damage.

– Constant tapping of minor amounts of fuel can be revealed using flow sensors in the fuel pipe to the generator as well as door contacts on the fuel cap showing if it has been opened besides scheduled refilling visits.

Partner Informationazeti Networks is a global provider of machine-to-machine (M2M) technology for a variety of verticals. Cisco has partnered with azeti to provide an end-to-end solution for Connected Assets for cell towers. azeti has pioneered in building sensor gateway (SG) and control products that interface with Modbus-based sensors. In the current project, azeti sensor gateway software (Site Controller) is used at the cell site and azeti Control Panel software is used in the NOC and is hosted in the cloud.

Cisco Solution USPCisco Solution USP offers the following benefits:

• Highly scalable architecture can support asset management for thousands of remote sites.

• Secured WAN connectivity with advanced VPN options.

• Fog computing at the edge ensures the cell is operational even when the WAN link to the NOC is down.

• Faster response time to critical situations by executing actions locally at the remote site.

• Limited WAN bandwidth requirement by data compression and summarization at the remote site.

• Multiple WAN backhaul support with primary and backup options.

• Wide range and easily extendable sensors supported.

• Supports remote site surveillance with camera.

• Real time event notification to the NOC.

• Highly customizable reporting and analytics.

• Support for multi-tenant operations.


Design Guide

Chapter 1 System Overview Cisco Solution USP


Design Guide

Design Guide

C H A P T E R 2
System Architecture (or System Design)
Major Architectural Requirements This section describes major architectural requirements of Connected Assets CVD 1.0:

• The Connected Assets CVD 1.0 system needs to have a highly scalable and distributed architecture with thousands of cell sites geographically distributed across a wide region.

• Fog computing software residing at the cell site on the SG, which is responsible for local computing, needs to have a reliable communication channel to interact with the central operations and analytics software hosted in NOC.

• Intermittent low speed network connectivity between the remote cell sites and the central operations center is a considerable challenge, which needs to be addressed by the network architecture.

• The architecture must leverage the available customer wired network while providing a backup on a wireless connection. The communication on both these media needs to be secured as it can pass through public Internet.

• The cell site architecture needs to support interfacing with Modbus serial sensors and IP cameras. The solution needs to collect various system and environmental parameters from the sensors as well as support snapshots based surveillance and future video surveillance

• The Connected Assets CVD 1. 0 NOC should be hosted in a public or private cloud.

• Overall solution needs to have a unified Connected Assets asset management system to manage entire operations of the network.


Chapter 2 System Architecture (or System Design) End-to-End System Architecture

End-to-End System ArchitectureFigure 2-1 End-to-End High-Level Network Block Diagram

Figure 2-1 shows major functional blocks in the Connected Assets end-to-end system architecture. Multiple cell sites distributed geographically connect to the central Connected Assets NOC hosted in a public or private cloud. As shown in Figure 2-1, the connectivity between the cell sites and the NOC can pass through the Internet. In the cell site, multiple sensors are connected to the SG over the Modbus serial interface. The functional blocks at the cell site are described in section Cell Site Architecture and functional blocks at the NOC are described in section NOC cloud architecture.

End-to-End System Functional Blocks and Message FlowThe end-to-end message flow architecture is shown in Figure 2-2.

Figure 2-2 End-to-End Message Flow

3750

67

Public Internet

Cell Site

3G/4G Backhaul

SAM Asset ManagerWeb client

ModbusSensor

Analog Sensor

DigitalSensor

IPCamera

Sensor GW

Fog Computing

Modbu sAdaptor

DryContact Sensor

Edge Router Customer

Network

Ethernet backhaul

Asset Vision DC

Ethernet

3rd Party NMS / BI tools

Site Controller

Modbus

Modbus

Sensor Manager

Web, MQTT, SNMP, VPN, SSH Server

Single Plane of Glass

Reports/Analytics

Fault /Performance

Provisioning/Configuration

Video Sensor

Video Manager

AnalyticsServer

Web client

3750

68

DataStorage

CAM

DataStorage

Reports/Analytics

Asset Vision DC

API/Pub-

Sub

https

ActiveMQ

SAM Asset ManagerWeb client

REST API

IPSec VPN for all

communications

InternetCell Site

SAM Asset Manager Web client

Modbus Daemon

Analog Sensor

DigitalSensor

SNMP

Mosquito Internal Modules

HTTP client

DryContact Sensor

Cloud Connector

History Exporter

IP Camera

MQTT connections

IPSec VPN for all

communications

3rd Party NMS / BI tools

Site Controller

VPN with encryption Https

SNMPv3MQTT

azeti ACP


Design Guide

Chapter 2 System Architecture (or System Design) Cell Site Architecture

In the cloud, the azeti Control Panel (ACP) and Cisco Asset Manager (CAM) are the front-end application servers. ACP provisions all sites and a sensor, while CAM queries and obtains these configuration details from ACP using Rest API. ActiveMQ is the MQ Telemetry Transport (MQTT) broker at the cloud end. PostgreSQL is the database used by ACP.

ACP and CAM connect to ActiveMQ and subscribe required MQTT topics. The cloud connector at the SiteController (IR910) is the MQTT client, which connects to ActiveMQ in the cloud. When required, the cell site-to-cloud connection is secured with a VPN tunnel. SiteController is the SG application at the cell site that communicates with the sensors, processes locally, and publishes the results to the cloud. Mosquito is the MQTT broker and SQLite is the database used at the cell site. Various application processes at the cell site communicate with each other by exchanging MQTT messages via the Mosquito broker. Third-party NMS systems, Web clients, and others that are hosted outside the NOC communicate using one of the secured connections such as HTTPS and SNMPv3. Tomcat is the webserver used in the cloud.

At the cell site, the Modbus daemon polls different sensors at regular intervals in a round robin fashion. The polling interval for each sensor can be configured. The collected sensor data is processed at the edge to check threshold crossovers. The threshold crossover events are published in real time to the cloud. Specific actions are taken locally based on different sensor data. The collected history records are published to the cloud at regular intervals.

IP cameras at the cell site are configured with static IP address. HTTPS is enabled on camera and session ID configuration is disabled. Communication to the camera is always initiated from the SiteController over HTTPS. Only photos are taken; no live video streaming is supported. The trigger to take a photo is always initiated from the SiteController to the camera with an HTTP request. On getting a trigger, The camera takes a photo and stores it in its local memory. SiteController fetches the stored photo from camera with and HTTP request. At any given point in time, the camera needs to store only the latest photo. The collected photos are published to the cloud.

Cell site connects to the cloud using either a 3G/4G connection or Ethernet provided by the cell site operator. Appropriate routing, IP addressing, and VLANs are configured for the communication to be possible (details are covered in the following sections).

Cell Site Architecture

Cell Site Functional BlocksThe functional blocks in the cell site include the following:

• Sensors - Transducers that sense or detect some characteristics of the environment and provide corresponding output. Different kinds of sensors supported by Connected Assets project are Modbus and non-Modbus sensors.

• Sensor Gateway - Acts as gateway between sensor nodes and the end users as they typically forward data from the end sensors to a central server. Typically they are deployed on routers/switches that interface with the sensors.

• IP Cameras - Deployed at the cell site for surveillance.

• Internet connectivity - Physical connectivity to the cloud/Internet.


Design Guide


Sensors InterfaceThis section describes the physical interface and communication protocols between the sensors and SG, which are supported in CVD 1.0.

RS485

RS485, which is the physical interface between the SG and the sensors, is a serial interface. The key advantage of RS485 is its simplicity and use as a multi-drop communication link to connect multiple sensors. It offers data transmission speeds of 35 Mbps up to 10meters and 100 Kbps at 1200 meters. A rule of thumb is that the speed in Mbps multiplied by the length in meters should not exceed 108. Thus, a 50 meters cable should not signal faster than 2 Mbps. RS485 electrical interface permit connecting up to 32 nodes. This number can be extended with repeaters.

It is preferable to have all sensors connected over RS485 to have the same serial communication parameters such as baud rate, parity, start, and stop bits. However, different communication parameters for different sensors are allowed.

Modbus

Modbus is the communication protocol between the SG and the sensors. Modbus comes with the three following modes:

• Modbus ASCII

• RTU

• Modbus TCP

In case of communication over a serial interface such as RS485, the possible Modbus modes are ASCII and RTU. As the name suggests, Modbus ASCII packet carries data in ASCII format and RTU carries data in binary format.

Modbus serial can address up to 247 devices and has one broadcast address. Each slave can have a number of registers that master can read/write. The stored data can be simple discrete or Boolean values or can be of 16 bit unsigned value.

Modbus is a bus connection; all communications are broadcast messages on the bus. A Modbus RTU network has one master and one or more slaves. The master always initiates the conversation. It sends a query addressing a slave and only the addressed slave responds to the query. All other slaves receive the request, but they do not respond.

In Connected Assets 1.0, the Modbus RTU interface is supported. Modbus RTU is a prevalent sensor interface in the IOT industry. Modbus RTU sensors are inexpensive and are easily available in the local markets. The sensors can have Modbus natively or can interface via the Modbus adapter. For more information, please refer to the following URLs:

http://en.wikipedia.org/wiki/Modbus

http://www.lammertbies.nl/comm/info/modbus.html


Design Guide

http://en.wikipedia.org/wiki/Modbus

http://www.lammertbies.nl/comm/info/modbus.html


Sensors and Modbus AdaptersAs discussed in the previous section, Modbus RTU is the interface supported in CVD1.0. However, sensors with different interfaces (analog sensors, digital sensors and dry contact sensors, and Modbus sensors) can be connected to the sensor gateway. The sensors having Modbus RTU natively can be connected directly to the sensor gateways, whereas other sensors can be connected via a Modbus Adapter.

The following Modbus adapters interface with different kind of sensors:

• Analog Modbus adapters

These are adapters interfacing between Modbus and analog sensors (for example, ADAM-4117-AE).

• Digital Modus adapters

These are adapters interfacing between Modbus and digital sensors (for example, ADAM 4150-AE).

• Dry contact Modbus adapters

These are adapters interfacing between Modbus and sensors providing Boolean data (for example, ADAM 4150-AE.

IP Cameras

IP cameras are connected to the SG (IR910) over Ethernet. The sensor gateway communicates with the IP camera using HTTPS and controls its operations.

Real and Virtual Sensors

A real sensor is a physical sensor that measures some condition in the real world such as temperature, humidity, door open, current, voltage, and so on. Virtual sensors are computed software derivation based on inputs from one or more real sensors. For example, a virtual power sensor can measure instantaneous power based on the physical inputs received from current and voltage sensors. Virtual sensors can be present at the cell site or even at the cloud side.

Sensor Gateway and Fog Computing SystemSensors gateways are middle devices between the end sensors and the NOC applications. Typically they are deployed on routers/switches that interface with the sensors. The capabilities of sensor gateways include types of interfaces supported to communicate with the sensors, local processing capability, capability to host third party applications, and cloud interfaces supported including data security. As the sensor gateway is close to the data collection point and it interfaces with often slow and intermittent links with the central cloud servers, data summarization at the SG is extremely useful and is an important feature. This capability is termed as edge computing of fog computing.

Fog computing extends the cloud computing and services paradigm to the edge of the network. Similar to cloud, Fog provides data, compute, storage, and application services to end-users. A distinguishing Fog characteristic is its proximity to the data collection point. Services are hosted at the network edge or even on end devices such as edge routers or access points. By doing so, Fog reduces service latency, and improves QoS, resulting in superior user-experience. Fog supports densely distributed data collection points.


Design Guide

Chapter 2 System Architecture (or System Design) Interface between Cell Site Components and NOC

In Connected Assets 1.0, the IR910 router is the sensor gateway and Fog computing device. The specification for IR910 router models can be found at the following URL:

http://www.cisco.com/c/en/us/products/collateral/routers/900-series-industrial-routers/datasheet-c78-732129.html

Fog Computing Considerations

• All events crossing the threshold should be forwarded in real time.

• Set different water mark levels for thresholds. Different severity levels (Critical, Minor, and Clear) for different services. Refer to azeti documentation for configuring, thresholds, and severity levels.

• History files compress before transmitting. History to be pushed at every 5 minutes, when connection is up. Preserve Events and History data when connection is down.

• All actions configured and executed locally; no connection to server needed.

• Polling interval - 500msec for low and normal priority services and 100msec for critical priority services (based on the baud rate and response delay from the sensor). Refer to sensor data sheet.

• No change is needed in the polling interval or list of sensors polled based on the connectivity state to NOC.

• Purge/Summarize history data - Low priority.

Limitations of IR910

• Single Ethernet port; used for WAN connectivity. Cannot connect IP sensors and cameras.

• Lack of Multiple L3 SVI interface. Does not support inter VLAN routing.

• Supports only two default routes. Host route not supported.

• IPsec site-to-site tunnel is supported only on 3G uplink. Remote Access (RA) tunnel supported on Ethernet.

• IPSLA not supported. Auto switchover not supported when WAN connectivity fails.

• Easy VPN (EZVPN) not supported.

• No built-in auto dialer and auto switchover capability.

Interface between Cell Site Components and NOC

Sensor InterfaceThe sensor interface is azeti Site Controller. The sensor gateway software running on IR910 is the single interface for all sensor communications between the cell site and the NOC. MQTT is the messaging protocol used for this. Traps form the IP sensors (including cameras) that are translated to MQTT messages and these MQTT events are sent to the cloud based on configuration. To detect link failure between the SG and NOC, MQTT keep-alive can be set to 5 minutes. With this MQTT, the ping request is sent once in every 5 minutes (activity based) and link failure can be detected.


Design Guide

http://www.cisco.com/c/en/us/products/collateral/routers/900-series-industrial-routers/datasheet-c78-732129.html

Chapter 2 System Architecture (or System Design) Cell Site Network Architecture

Camera InterfaceIn the current version of Connected Assets, no direct communication occurs between camera and NOC.

Cell Site Network Architecture

IP Address and VLAN Considerations at the Cell SiteIR910 Ethernet interface, 899G LAN interface and all cameras are to be on the same local subnet. IPsec tunnel is set up from 899G to IPsec VPN server in the cloud. In case of 3G/4G, dynamic IP address allocated by 3G/4G will be the IPsec tunnel end point IP address. In case of WAN, uplink the IP address configured at the 899G will be the IPsec tunnel end point IP address. The IP in case of WAN can be public or private IP. However, reachability from cell site to the VPN server needs to be ensured. IPsec tunnel is always initiated from the cell site.

VLAN Configuration

Table 2-1 shows the VLAN specifications at the cell site.

IP Address

Table 2-2 shows the IP address specifications at the cell site.

Figure 2-3 shows the VLAN and IP address.

Table 2-1 VLAN Specifications at the Cell Site

Device VLAN

Camera VLAN 3

WAN VLAN 1

Table 2-2 IP Address Specification at the Cell Site

Device/Host IP Subnet

IP910 Ethernet interface, Camera and 899G LAN interface

192.64.168.0/24

899G Router WAN interface As supplied by the operator (public/private)

3G/4G Dynamic IP allocated by the SP


Design Guide


Figure 2-3 VLAN and IP Address

Cell Site Deployment ModelsTwo different deployment models are recommended to suit the following different cell site needs:

• Cell site having only serial sensors

• Cell site having serial sensors and cameras

• Ethernet is the only backhaul connectivity

• 3G is the only backhaul connectivity

• Ethernet is the primary backhaul connectivity with 3G as backup

• Cell site located in the region of:

– US, China

– Regions outside of the former two

The two deployment models (a single box solution and a two-box solution) are discussed in the following sections.

Deployment Model-1

In this document, we refer to this model as a one-box model (see Figure 2-4). The IR910 3G model is recommended for Deployment Model-1. This has two possible WAN links: one 3G and one Ethernet.

IR910 899G

Modbus adapter

Modbus Sensor-1(Temp)

On Top of Door

Modbus Sensor-2(Power)

Pointing to generator

Ethernet uplink

AnalogSensor

DigitalSensor

Dry Contact

Modbus over RS485

Ethernet

Ethernet POE

Analog/Digital/Binary

VLAN InterfaceVLAN-100192.168.1.1

VLAN-100192.168.1.x

3750

72


Design Guide


Figure 2-4 Deployment Model - 1: One-Box-Model

Table 2-3 shows the device roles in Deployment Model - 1.

Figure 2-5 illustrates the system architectural layers of Deployment Model - 1.

Figure 2-5 Deployment Model - 1: System Architectural Layers

Deployment Model-2

In this document, we to this model as two-box model (see Figure 2-6). IR910 GE is recommended for Deployment Model-2. IR910 GE does not have 3G cellular capability; its remaining specifications are identical to IR910 3G model.

3750

73

3GWAN

IR910

Table 2-3 Device Roles: Deployment Model - 1

Device Role

IR910 3G Sensor gateway

IR910 3G Transport router

3750

74

Wireless3G

WiredCustomer Ethernet

WAN SecurityIPsec

IR910 Sensor Gateway

Environmental Energy GeneratorSecure Site

Access

RS485/Modbus

Cisco Cloud Services Data ManagementCisco Asset Manager/Azeti ACP


Design Guide


Figure 2-6 Deployment Model - 2: Two-Box-Model

Table 2-4 shows the device roles of Deployment Model - 2.

The deployment model selection criteria are summarized in Table 2-5.

Figure 2-7 shows the system architectural layers of Deployment Model - 2.

Figure 2-7 Deployment Model - 2: System Architectural Layers

3750

75

899G

3G/4G WAN

IR910IR910

IR910

Table 2-4 Device Roles: Deployment Model - 2

Device Role

IR910 GE Sensor gateway

899G Transport router

Table 2-5 Deployment Model Selection Matrix

Deployment in US and China Deployment Globally

Modbus sensors only One-box model Two-box model

Modbus sensors and cameras Two-box model Two-box model

3750

76

Wireless3G/4G

WiredCustomer Ethernet

WAN SecurityIPsec

IR910 Sensor Gateway

RS485/Modbus

890G

Environment Energy Generator Site Access

Cameras (up to 4)

Ethernet (PoE support)

Cisco Cloud Services Data ManagementCisco Asset Manager/Azeti ACP


Design Guide

Chapter 2 System Architecture (or System Design) NOC Cloud Architecture

Benefits of Two-Box Solution Deployment Model-2

With the two-box solution, the customer gets a number of rewards for the additional investment as shown in Table 2-6.

WAN Connectivity

WAN Edge Router

WAN edge router is the cell site router/gateway that connects the cell network to the service provider network and cloud. This could be sensor gateway itself or an external router. Different connectivity options include 3G/4G and Ethernet. In case of Ethernet, the connection provided by the service provider at the cell site could be either L2 or L3.

WAN Connectivity Failure Detection

WAN connectivity failure is detected in the following ways:

• MQTT keep-alive detects connectivity failure between the sensor gateway and the MQTT server in the cloud.

• In the two-box-model, transport router detects connectivity failure with the help of IPSLA.

NOC Cloud Architecture The Connected Assets Network Operation Center (CA-NOC) is hosted in a public or private cloud. This securely communicates with all sensor gateways at the cell sites. The functionality of NOC includes site provisioning, sensor data collection, monitoring the status of various parameters, generating and escalating alerts, and presenting various fault/performance reports across sites to the operator. It is also responsible for providing North Bound Interface (NBI). The different servers responsible for this functionality include azeti Control Panel (ACP) and Cisco Asset Manager (CAM).

Table 2-6 Benefits of the Two-Box Solution

Features One-Box Two-Box

Modbus sensors Yes Yes

3G backhaul connectivity Yes Yes

IPSec VPN Yes Yes

IP sensors support No Yes

4G backhaul connectivity No Yes

IP camera support No Yes

POE for IP devices and cameras No Yes

Auto switchover between primary Ethernet and backup 3G/4G

No Yes

Advanced VPN such as Easy-VPN and FlexVPN No Yes

Compliance to 3G/4G spectrum/standard in wider regions across world

U.S. and China Globally


Design Guide

Chapter 2 System Architecture (or System Design) NOC Cloud Architecture

Figure 2-8 shows components of AV-NOC. All applications in the cloud are hosted in VM. All servers have a fixed IP address. In this version of Connected Assets, no redundancy support is considered for any components in the cloud. The solution is tested using the Cisco cloud.

Figure 2-8 Architecture of the Network Operation Center Hosted in the Cloud

Connected Assets NOC Functional BlocksThe following are the Connected Assets NOC functional blocks:

• Communication Servers: Different interfaces are possible to communicate with the NOC servers. These interfaces include Web (HTTPS), SNMP, VPN, SSH, and MQTT.

• NOC hosted in Cloud: NOC for the Connected Assets can be hosted in cloud. Various NOC applications such as Sensor Manager, Video Manager, and Analytics Manager are part of the NOC hosted in cloud.

• Northbound Interface (NBI) system: The Connected Assets NOC applications can interface with customer work-flow/BI systems and NMS systems. Different NBI could be SNMP, MQTT, XML and so on.

• Asset Manager Web Client: This is a user-friendly management console, where the operator can monitor and manage the entire network from a web client. The client presents information from all kinds of sensors and camera. It also visualizes various live fault, performance data, and trend charts from historical data.

NBICustomer could have different third-party Business Management System (BMS) systems and NMS systems. The AV-NOC needs to interface with these third-party systems. The possible NBI supported by the AV-NOC includes SNMP, MQTT, and HTTP.

3750

77

Internet

Asset Manager Web client

azeti ACPData

Storage

CAM

Data Storage

Reports/Analytics

Asset Vision NOC

API/Pub-Sub

https

ActiveMQ

Asset ManagerWeb client

REST API

Cisco CSR1000V


Design Guide

Chapter 2 System Architecture (or System Design) Security Architecture

Security ArchitectureSecurity is one of the most important aspects as the assets are spread over geography and are connected via the public network to the central management station. However, if the cell sites and NOC are connected via a secured private network owned by the customer, then additional security measures could be required.

Network SecurityWhen required, communication channels between the cell site and the servers in the cloud are protected by IPSec tunnel. For Deployment Model - 1, IR910 plays the role of both sensor gateway and transport router. The tunnel is set up between IR910 at the cell site and a VPN server in the cloud. The VPN server in the cloud could be the Cisco CSR1000V router. Thus, all communications are authenticated and encrypted. When the Ethernet fails and connection is switched to 3G/4G, a new tunnel is set up via 3G/4G. Tunnel creation is always initiated from the cell site by IR910 and a standard IPSec tunnel is set up.

Figure 2-9 shows the IPSec tunned for Deployment Model - 1.

Figure 2-9 Deployment Model - 1: IPSec Tunnel

Deployment model - 2 is similar to model - 1, except that the role of IR910 is limited to a sensor gateway and 899G a transport router. In this case, the IPSec tunnel is set up between 899G at the cell site and VPN server at the cloud. Similar to model - 1, the VPN server could be a Cisco CSR1000V router.

All communications are authenticated and encrypted. When the Ethernet connection fails, 899G detects with the help of IPSLA and switches the interested traffic via the IPSec tunnel over 3G/4G. On restoration of Ethernet connectivity to the cloud, the traffic is switched back to Ethernet.

Similar to model - 1, tunnel creation is always initiated from the cell site transport router. In this case, it is 899G.

In the deployment model 2, FlexVPN is the recommended VPN model. FlexVPN is based on open standards-based IKEv2 as the security technology. It also provides many Cisco-specific enhancements. FlexVPN is the latest VPN model from Cisco. Some key advantages of FlexVPN are that it is designed to simplify the deployment of VPNs and it encompasses all features of various VPN models such as EZVPN, CryptoMap, and DMVPN. Cisco IOS-based routers such as ISR, ASR, and CSR support FlexVPN.

Figure 2-10 shows the IPSec tunned for Deployment Model - 1.

3750

78azeti ACPData

Storage

CAM

Data Storage

Reports/Analytics

NOC in cloud

API/Pub-Sub

https

ActiveMQ


REST API

CSR1000V

IR910

3G

Modbus adapter



AnalogSensor

DigitalSensor

Dry Contact

Modbus over RS485

Ethernet


SPNetwork

Internet

IPSec tunnel


Design Guide

Chapter 2 System Architecture (or System Design) QoS Architecture

Figure 2-10 Deployment Model - 2: IPSec Tunnel

Application Security

Cell Site Security

All communications to the cell site are protected with a VPN. Access to devices at the cell site is authenticated with username and password. Direct access is only via SSH.

Other Security ConsiderationsFigure 2-10 shows other security considerations.

QoS Architecture

Network QoSAs the Connected Assets traffic passes through the Internet, no QoS parameters such as latency, jitter, or packet loss are guaranteed for the traffic. The desired network bandwidth is computed and that is to be provisioned to ensure smooth system operations.

3750

79azeti ACPData

Storage

CAM

DataStorage

Reports/Analytics

NOC in cloud

API/Pub-Sub

https

ActiveMQ


REST API

CSR1000V

IR910 899G

Modbus adapter


On Top of Door


Pointing to generatorAnalog

SensorDigitalSensor

Dry Contact

Modbus over RS485

Ethernet

Ethernet


SPNetwork

Internet

VLAN InterfaceVLAN-100192.168.1.1

VLAN - 100192.168.1.x

IPSec tunnel

Table 2-7 Other Security Considerations

Server LocationMaximum Number of Client Connections

Subscription to Topics azeti Support / Compliance

Mosquito device MQTT broker

20 All topics Comply

ActiveMQ cloud MQTT broker

1200 All topics Comply

Limit using the ActiveMQ configuration file.


Design Guide


Application QoSMQTT Keep Alive:

The keep-alive is a 16-bit value measured in seconds. It is the maximum idle time permitted between control packets sent by the client, including PINGreq (ping request). The keep-alive value is defined during the connection setup. Server disconnects the client connection upon non-receipt of any control packet for 1.5 times the keep-alive time. Client can send PINGreq at any time to determine the up status of network and the server. A keep-alive value of zero turns off the keep-alive mechanism. In this case, the server is not required to disconnect the client on the grounds of inactivity. Typically the keep-alive time is kept to a few minutes. The maximum value is 18 hours 12 minutes and 15 seconds.

Message Delivery Priority

No delivery priority can be guaranteed for different message types as the delivery is across the Internet. Events and History are published on separate connection. No out-of-band channel is supported by MQTT. All events are scheduled in FIFO.

The reference link to MQTT can be found at the following URL: http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html

Guaranteed Message Delivery (MQTT QoS)

QoS 0: At Most Once Delivery (Best Effort)

The message is delivered according to the capabilities of the underlying network. The receiver sends no response and no retry is performed by the sender. The message arrives at the receiver either once or not at all.

QoS 1: At Least Once Delivery (Guaranteed)

This quality of service ensures that the message arrives at the receiver at least once.

QoS 2: Exactly Once Delivery (Guaranteed and No Duplicates)

This is the highest quality of service, for use when neither loss nor duplication of messages is acceptable. An increased overhead is associated with this quality of service.

Table 2-8 shows the guaranteed MQTT delivery levels.

Table 2-8 MQTT Guaranteed Delivery Levels Configuration

Message Type / Topic CVD Recommendation MQTT QoS azeti Support / Compliance

Ping requests Best effort QoS 0 Comply

Events • Guaranteed delivery

• No duplications

• Encryption

QoS 1 (At least once)

• azeti is using QoS1 for events.

• Duplicates are removed at the server.


Design Guide

http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html


Protecting the Client from Storage Overflow during Server Link DownEvents older than a week are to be purged at the device. History data file is zipped and published at configurable interval when the connection is up. The interval is configured as 5 minutes. When the connection is down, the files are kept on a local disk/RAM uncompressed and data is never purged. History file is log rotated on daily basis and a maximum of 5 copies are kept. Table 2-9 summarizes the log-rotation policy for various log files.

Connected Assets Asset ManagerCAM and ACP together provide all network management functionality. At an abstract level, all configuration and provisioning operations are supported by ACP and all fault, performance-related operations are supported by CAM. Table 2-10 lists the different management roles and corresponding user-interface that supports it.

History data • Guaranteed delivery

• No duplications

• Zipped file

• Encryption


• QoS 1 used. Each history file zipped and separate message

• File/record duplication checked at cloud.

XML upload (from cloud to device)

• Guaranteed delivery

• No duplications

• File Integrity check

• Data correctness check


• QoS 1 used. Each XML file separate message

• File duplication checked at device.

• Integrity of the file is checked with md5.

• File data validated by a process.

Table 2-8 MQTT Guaranteed Delivery Levels Configuration

Message Type / Topic CVD Recommendation MQTT QoS azeti Support / Compliance

Table 2-9 Log-Rotation Policy

Log File NameRotation Frequency

Maximum Log File Size(Based on disk space requirement computation)

Number of Copies

History Daily 1 MB 5

Debug files Daily 1 MB 5

Table 2-10 Management Roles for CAM and ACP

Management Role Supported by (CAM/ACP)

Configuration of all sensors and all sensor-related parameters (for example, communication parameters, polling intervals, priority)

ACP

Configuration of actions on device (actions-based events) ACP

Viewing cell site system logs and debugging ACP


Design Guide


Browser Requirements• Cisco Connected Assets Services have a browser-based management console for configuration,

operation, and reporting and it uses very extensively JavaScript. For the best user experience, it is recommended that an up-to-date browser with fast and efficient JavaScript support, such as the latest version of Google Chrome or Mozilla Firefox be used.

• Avoid any versions of Internet Explorer, especially IE 8 and before.

• Regardless of which browser is used, JavaScript needsRemote Access

• The Cisco Connected Assets Services server is administered through a web interface. By default, both port 80 (HTTP) and 443 (HTTPS) are supported. For security reasons, it is highly recommend to open port 443 (HTTPS) only for remote administration and client connections.

Notification E-MailsCisco Connected Assets Services can generate notification emails for fatal system errors, low resources or if the license expires. To enable email notifications, the email account details to send emails and the email addresses of the appropriate recipients is to be configured. These settings are also used to send scheduled favorite reports or other sources via email.

Cisco Connected Assets Services will email a notification if one the following events occur:

• The amount of devices exceeds the licensed number.

• The product is not licensed.

• A resource drops below its threshold.

• The service terminates unexpectedly (for example, crash, out of memory/disk space).

• A new update is available.

• An update is being installed.

• An update was successfully installed.

• An update failed.

In addition to those events this mail configuration is also used to send emails during the following:

• Sending system notifications, for example, in policies or alerts

Live alarm monitoring (Events/alarms monitoring/alarm panel) CAM

Live performance monitoring (graphical and tabular performance data) CAM

Performance reports (analytics, summary charts, trend charts, correlation reports)

CAM

Performance and fault reports across cell sites CAM

Performance and fault reports across sites and tenets (multi-site and multi-tenant)

CAM

Camera photo viewing (recent and history) CAM

North Bound Interface (alarms) - SNMP CAM

Table 2-10 Management Roles for CAM and ACP

Management Role Supported by (CAM/ACP)


Design Guide


• Sending scheduled favorite reports

CAM Functionality - Reports and AnalyticsRefer to CAM documentation for report generation capability and their dimensions (time and count based reports, statistics based reports, reports across cell sites and so on). Define reports based on use cases. It should be possible to view performance reports in graphical form, show previously stored reports, schedule report generation for standard and customized reports and view critical performance metrics.

Cisco CAM Server is hosted in public or private cloud infrastructure and does not need the Customer to provide any hardware. CAM software is packaged with PostgreSQL database to collect energy information about all devices.

Representative Set of Reports

As mentioned CAM is a generic reporting tool and customers can generate multiple reports suiting their specific needs. To show the flexibility a sample set of reports that can be generated by CAM are listed below:

• Monitor site security to be enabled.

– How many times the door was opened during a month?

– How long the site was in un-armed state?

– How many failed attempts to open the door?

– How many brute force attacks to open the door?

– How many times door was opened with keypad and opened from remote?

– Report list of snaps taken by a cell site for a period

– How many man-down events noted in a year?

• Environmental Monitoring and Advanced analytics

– What are the average, min and max temperature noted in a month?

– What is the duration external temperature was outside the operating range?

– Humidity monitoring, average, peak and minimum over a day/week.

– How long AC was running?

– Correlate environmental temperature and AC power consumption.

– Correlate power consumption with external temperature, AC run time and system up time.

– Define AC maintenance schedule based on running hours.

• Power and Generator Monitoring

– HVAC power source power quality report: voltage fluctuations, frequency fluctuations, power cut pattern and duration.

– Generator run time during a month/week/day?

– Generator fuel consumption report per day/week/month. Fuel refill alerts.

– What is the rate of consumption of fuel per run time of the generator?

– Number of times fuel leak/theft detected?


Design Guide

Chapter 2 System Architecture (or System Design) Connected Assets System Requirements

– Generator power quality reports: Voltage, frequency.

– Generator power consumption reports: Units.

– Generator startup battery status report.

– Generator maintenance schedule based on running hours.

– Power consumption per asset type such as Air Conditioner and per equipment rack.

– HVAC three phase load distribution.

– Power theft detection.

• Multi-tenant reporting

– Individual power consumption reports for each tenant in a site.

– Individual power requirement and prediction reports.

• Statistics reporting

– Number of threshold cross notifications.

– Number of critical, major events reported in a month.

– Network bandwidth reports, peak and average bandwidth.

– Average storage requirement over a month.

– Cell link down events and duration report.

Case study: How you can use the reports.

Connected Assets System RequirementsThe Connected Assets system is composed of two main parts: Cisco Connected Assets Services Central Server hosted in a public or private cloud and Cisco Connected Assets fog computing system site controller hosted at the cell site in the sensor gateway, hardware and sensors installed locally at the site/base station.

Azeti Control Panel Functionality Azeti Control Panel functionality should include configuration of all sensor and communication related parameters: control door access froma remote location with and without a time limit, detect events such as man-down, and so on.


Design Guide

Chapter 2 System Architecture (or System Design) Azeti Control Panel Functionality


Design Guide

Design Guide

C H A P T E R 3
System Components
Cisco ProductsTable 3-1 shows the Cisco components.

Table 3-1 Cisco Components

Cisco Product Software Release Description

Cisco IR910

Models

IR910 GE

IR910 3G

1.2.1rb2 Sensor gateway and fog computing

International deployment

C899G-LTE-GA-K9

U.S. deployment

C899G-LTE-NA-K9

C899G-LTE-VZ-K9

C899G-LTE-ST-K9

14.4.3.M3 Transport router and switch to connect cameras

CIVS-IPC-6400E

CIVS-IPC-7030E

CIVS-IPC-6xxx-V2.5.0-10.bin

CIVS-IPC-7xxx-V2.5.0-10.bin

Camera

CAM 5.0 Sensor network monitoring, analytics and reporting tool. NOC software


Chapter 3 System Components Third-Party Products

Third-Party ProductsTable 3-2 lists the third-party products.

Modbus and Non-Modbus Sensors and ApplicationsThe list of Modbus and Non-Modbus sensors that are included in the CVD 1.0 are given in Table 3-3. Modbus sensors can be serially daisy chained and connected to the serial ports. Non-Modbus sensors are connected to an adapter that translates the received packets from analog/digital/dry-contact to Modbus over serial packets.

Table 3-2 Third-Party Products

Vendor Product Release Description

Sensors Refer to Table 2-7 Modbus and non-Modbus sensors.

Site Controller 1.1.0 Sensor gateway software.

azeti Control Panel

0.3.0 Sensor configuration and provisioning tool. NOC software.

Table 3-3 List of Sensors

Vendor Product Description Remarks

Advantech ADAM-4117-AE Eight analogue inputs, analog to digital converter

RS485 MODBUS

For example, quantization of voltage output of pressure sensor

Advantech ADAM 4150-AE Seven digital inputs, eight outputs

RS485 MODBUS

For example, sense switch on and off state/dry contact output

Elkor i-Snail-VC AC current sensor Analog input

For example, monitor electrical load

Elkor WattsOn-1100-MS160-120A

AC power meter

Can monitor up to three single phase loads or one three-phase load

RS485 MODBUS output

Elkor i-Snail-S Current switched sensor, monitoring on/off state of a device

Digital input

For example, detect electrical load on/off state


Design Guide


Elkor MS160 Current transducer, for POC only needed one phase.

Current measuring with clamped upper limit.

Connected to WattsOn

CE-Transducers AD11B-34GS4-1.0/0-50A*0-65V

DC multi-parameter monitoring (U,I,P,KWh)

RS485 MODBUS

CE-Transducers CE-AU11-34MS3-0.2/0-65V

1-element DC Voltage Meter

RS485 MODBUS

Comet Magnetic door contact SA-200-A

Door contact, digital output

Digital input

Schneider Electric

NSYS3D6640P Enclosure

Comet T3411 Temperature/Humidity Sensor

RS485 MODBUS

Infranet WTSC-485 Wiegand to RS-485 Converter (for keypad)

Converter for the keypad RS485 MODBUS

AST Sensors AST 4510 - Pressure Sensor

Fuel level sensor Analog input

HIDGlobal Keypad Proxpro including Converter

HIDGlobal

5355 WTSC-485

Security keypad Weigand

Table 3-3 List of Sensors

Vendor Product Description Remarks


Design Guide



Design Guide

Design Guide

C H A P T E R 4
System High Availability and Scalability
System Redundancy and High Availability

Redundancy at the Edge ComponentsSensors: Critical sensors can be duplicated and connected to different RS485 ports on IR910.

Cameras: Critical IP cameras can be duplicated and connected to different ports of 899G switch.

However, in each of these cases IR910, 899G at the cell site remain single point of failure.

Redundancy for Communication Link

One-Box-Model

In case of the one-box model, transport capability is provided by IR910. This box supports 3G and Ethernet backhaul, however, auto switchover is not supported. IR910 detects L2 failure (connectivity to Ethernet gateway) and does auto-switchover to the 3G connection. However, when the L2 connection is intact, but the L3 connection connectivity to the cloud via Ethernet fails, manual switchover needs to be done to switch the connection via 3G. Similarly, reverting from 3G to Ethernet can happen either manually or automatically when 3G-connectivity fails.

Two-Box-Model

It is preferable to have 3G/4G as a backup for Ethernet with auto switchover capability in all conditions. In such as case, 899G achieve this by implementing IPSLA. In case the connectivity to NOC via Ethernet fails, the system switches the connection to 3G/4G and reverts back when the connection via Ethernet is restored.

Redundancy for Server Applications in the CloudIn the current version of CVD 1.0, redundancy/load balancing for the applications running in the cloud (such as ACP, CAM, ActiveMQ, DB servers, CSR1000V and so on) are not considered.


Chapter 4 System High Availability and Scalability System Redundancy and High Availability

Scalability and Bandwidth RequirementsThe system supports polling of up to 50 data points (services) from each serial port. A polling interval up to 500msec is supported without any event loss. Up to 10 sensors with a polling interval of 100msec is possible. The CPU utilization should be below 90% and load average should be below 5. The system supports the network bandwidth and storage requirements shown inTable 4-1, without any impact on the performance and stability.

Based on the computation given in Table 4-1, the following bandwidth and storage requirements are recommended:

• Storage requirement at the cloud per month for 1000 cell sites: 200GB.

• Storage requirement at the device to store data for one day when connectivity link is down: 7MB. This does not include debug and log files.

• Downlink bandwidth requirement is minimal as very minimal messages sent in the downlink direction.

• Bandwidth requirement between SG and Cloud with 5sec allowed message latency is 46Kbps.

Server Sizing DetailsStorage space requirement for 1000 cell sites for one-month data storage is 200 GB.

Up to 500 web-clients connect in parallel to the webserver and MQTT.

Server recommendations for different size network deployment classified as small, medium and large are as follows:

• Small instance: 8 GB RAM, 2 cores 100 GB

Table 4-1 Network Bandwidth and Storage Requirements

Number of events per day (services crossing threshold)

10 Maximum tolerable event latency in seconds

5 Photo interval in seconds

300

History update interval in seconds

300

Types of messages

Peak Hour Messages per Second

Message Size in Bytes

Average BW Requirement (Kbps)

Number of Messages Stored in a Day

Storage per Day in MB

Storage per Month in GB for 1000 Cell Sites

ping 1 64 0 0 0 0

Events (threshold cross)

10 500 8 10 0 0

History update 1 3846 6 32

Photo from camera

1 20000 32 288 5.5 166

Total 46 7 198


Design Guide


• Medium instance: 16 GB RAM, 4 cores 500 GB

• Large instance: 32 GB RAM, 8 cores 2 TB

Server Sizing and Bandwidth Recommendations

Based on the number of cell sites the network can be classified into three categories: small deployments up to 100 cell sites, medium deployments up to 500 cell sites and large deployments up to 1000 cell sites. A non-cloud based NOC is recommended for deployments larger than 1000 cell sites.

The server sizing in the cloud for three deployment categories are discussed in the following sections.

Small Deployments

Server sizing in the cloud up to 100 cell sites, with a maximum of 10000 sensors total (see Table 4-2).

Medium Deployments


Table 4-2 Server Specification for Small Deployments

Server sizing for ACP Two medium size servers

(Medium size server: 16 GB RAM, 4 cores 500 GB)

Server sizing for CAM VM-1 hosting one application server and one controller

(1 Quad Core, 8 GB RAM, 250 GB disk space)

Table 4-3 Server Specification for Medium Deployments

Server sizing for ACP Three medium size servers

(Medium size server: 16GB RAM, 4 cores 500 GB)

Server sizing for CAM VM-1: hosting one app server


VM-2: hosting one controller



Design Guide


Large Deployments


Table 4-4 Server Specification for Large Deployments

Server sizing for ACP Three Medium size servers

(Medium size server: 16 GB RAM, 4 cores 500 GB)

Server sizing for CAM VM-1: hosting one application server

(2 Quad Core, 24 GB RAM, 1 TB disk space)


(1 Quad Core, 16 GB RAM, 100 GB disk space each)


(1 Quad Core, 16 GB RAM, 100 GB disk space each)


Design Guide

cisco connected assets 1.0 design guide · i asset vision 1.0 design guide design guide preface the...

Documents