Download - !Chaos Control System
C. Bisegni
Control system based on aHighlyAbstracted andOpenStructure
school 2012
http://chaos.infn.it
1venerdì 14 dicembre 12
the scene...
imagine to have this...
2venerdì 14 dicembre 12
the scene...
with many distributed
controls rooms
3venerdì 14 dicembre 12
the scene...
with many distributeddata center
4venerdì 14 dicembre 12
the scene...
with only ONEControl System with
complex automated task
5venerdì 14 dicembre 12
the scene...
somewhere in the future perhaps will have this
6venerdì 14 dicembre 12
the scene...
with only ONE Control System with even more complex automated task
7venerdì 14 dicembre 12
the scene...
securely, there will be a lot of
chaos
8venerdì 14 dicembre 12
the scene...
but only ONE!CHOASinstance
9venerdì 14 dicembre 12
10
Objec>ve
What is needed to control the CHAOTIC
complexity of big apparatus?
10venerdì 14 dicembre 12
11
Objec>veWhat is needed to control the CHAOTIC
complexity of big apparatus?
• a design that permits to scale to achieve high speed in the data
acquisi@on and elabora@on and for fault tolerant;
• possibility to aBach and detach an en@re subsystem;
• possibility to “configure” a specified instrument without halt the full system;
• possibility to hot insert an instrument or a subsystem;
11venerdì 14 dicembre 12
12
Objec>veWhat is needed to control the CHAOTIC
complexity of big apparatus?
• give the flexibility to implements control algorithm on abstracted
instrument class;
• work on data abstrac@on, without fixed schema;
• give a data system that permit to monitor the instrument channels and query the history instruments data;
• the data can be managed into domains
• every domain can be implemented with different technology
12venerdì 14 dicembre 12
13
Objec>ve
13venerdì 14 dicembre 12
14
What is!CHAOS (Control system based on Highly Abstracted and Open Structure), is an experiment of CSN 5(technological research experiments) at the LNF and Roma-‐TV sec@on.
The project has been appreciated by educa@onal world and the University of Tor Vergata and Cagliari have joined the development team.
UniversitàTorVergata LNF - Roma2
Universitàdi Cagliari
14venerdì 14 dicembre 12
15
Objec>ve
!CHAOS has been designed to widely scale and for manage large
apparatus with different needs in term of storage technology and speed
for data acquisi@on
15venerdì 14 dicembre 12
16
Objec>ve
• new topology in the CS that permit to integrate the triggered DAQ;
• define a new way to access real@me data;
• data acquisi@on, storage and indexing fully scalable;
• high scalability in every cri@cal point.
16venerdì 14 dicembre 12
17
Objec>ve
• 99% of adapta@on @me of instrument to !CHOAS done by framework
• simplify the crea@on of the user interfaces
• simplify the crea@on of the control algorithms basing it on an abstrac@on
of the I/O channels or full dataset not on physical device
• All service for data management and monitoring already done and fully
customizable
A CPP Framework and services that permit to have:
17venerdì 14 dicembre 12
18
!CHAOS Simplify Vision
18venerdì 14 dicembre 12
Implementation
Impl
emen
tatio
nImplem
entation
Control'sAlgorithm
User InterfaceInstrument's Driver
Implementation
Data Cloud
Live Data
History Data
network
DirectoryServerCloud
CHAOS Boundary
User specialized Boundary
Simplify Vision
Abstract RPC Driver
AbstractEvent Drivernetwork
AbstractDirect I/O
Implementation
Data Cloud
Live Data
History Data
network
Implementation
Data Cloud
Live Data
History Data
19venerdì 14 dicembre 12
20
!CHAOS Real Model
20venerdì 14 dicembre 12
Live / History domain
Live / History domain
Data Boundary
Common Toolkit
User Interface Toolkit
InformaIon Boundary
ChaosDirectory
Nodes
Common Toolkit
Control Unit Toolkit
Co
mm
on
To
olk
it
Exe
cuti
on
Un
it T
oo
lkit
RPC - Event - Direct I/O
RPC - Event - Direct I/O
RP
C -
Eve
nt
- D
ire
ct I
/O
HT
TP
- A
PI
Node Information
Node Activity
Security
CQL Proxy Nodes
Indexer Nodes
Management Nodes
AbstracIon Boundary
Driver for sensor and instruments
User Interface for monitoring and controls
ControlsAlgorithm
User CustomizaIon Boundary
21venerdì 14 dicembre 12
22
Toolkit and Services
!CHAOS is a set of toolkit and services that will permit the realizaAon of an control system
22venerdì 14 dicembre 12
23
!CHAOS Toolkit and Services
Common Toolkit
CU Toolkit
UI Toolkit
EU Toolkit
DirectoryServices
CQL Proxy
Indexer
Managment
Live and History Data
23venerdì 14 dicembre 12
24
!CHAOS Toolkit
implements the fundamental soZware's layer to RPC, I/O, Event and other’s u@lity code and for logic
Common Toolkit
abstracts the !CHAOS resources to the device drivers developers.
abstracts the CHAOS resources to the developers that need to develop control and compu@ng algorithms
abstracts the CHAOS resources to the developers that need to develop control and monitoring user interface
CU Toolkit
UI Toolkit
EU Toolkit
24venerdì 14 dicembre 12
!CHAOS Services
DirectoryServices
gives services via RPC (or with other higher transport layers) for get device structured informa@on, system topology etc...
CQL Proxy
Indexer
Management
give the access to insert and query opera@on on live and history data to other CHOAS node using CHOAS Query Language (CQL)
performs the indexing of the new inserted data
performs the management of the datas chunk saved on the storage (erase, move, etc.)
25venerdì 14 dicembre 12
26
!CHOAS Scalability and Fault-‐Tolerant
Some node needs to be scaled for performance reason or for fault-‐tolerant:
• Directory Nodes• Live and History nodes
26venerdì 14 dicembre 12
27
!CHOAS Scalability and Fault-‐Tolerant
Server P1
Server P2
Server PN
Device 1Driver
Framework
Server P1
Server P1
Server P2
Server PN
Client ReaderDriver
Framework
Server P2
X
XX
Clients that need to talk to scaled services, has more than one ip address for
every service
Every ip address has a priority, the client, as first address, uses the one with
highest prioritywhen a server goes down, another ip will be used
27venerdì 14 dicembre 12
28
!CHOAS Scalability and Fault-‐Tolerant
the re-‐balance occurs when a server with a highest priority, previously goes
down or stopped for maintenance, becomes up and running
Server P1
Server P2
Server PN
Device 1Driver
Framework
Server P1
Server P1
Server P2
Server PN
Client ReaderDriver
Framework
Server P2
use at TS1
use atTS1
ok
the new server will be used starIng from a pre-‐determinated Ime stamp (TS1)all the operaIons and the calculaIon of TS1 are coordinated by one of the
directory nodes
28venerdì 14 dicembre 12
11/dic/2011 29
!CHAOS Abstrac>on of the Data
29venerdì 14 dicembre 12
30
!CHAOS use a structured serializa@on for data descrip@on and serializa@on
• Data is managed by CDataWrapper cpp object that wrap the serializa@on;
• BSON (hBp://bsonspec.org/) has been chosen, very similar to JSON but it is binary;
• serializa@on is wrapped into a CPP class so it can be changed in future;
Abstrac>on
30venerdì 14 dicembre 12
31
it is used in (for now):
•Hardware dataset descrip@on's and acquired data packaging (“channel desc” = value)
•RPC message
•Chaos Query Language• Directory Server communicaIon
Abstrac>on
31venerdì 14 dicembre 12
32
• Hardware aBributes and I/O channels are described into the DATASET• Each aBribute is defined by
• Name• Descrip@on• Type (type ex: INT32, INT64, DOUBLE, STRING, BINARY, STRUCT)• Cardinality (single or array)• Flow (Input, Output, Bidirec@onal)• Range• default value• other will came
Abstrac>on for Hardware Dataset
32venerdì 14 dicembre 12
11/dic/2011 33
VOLT_CHANNEL_1
RESET
DATASETname:VOLT_CHANNEL_1type:VOLT32flow:outputrange:1-‐20card: 1
name:VOLT_CHANNEL_2type:VOLT32flow:outputrange:1-‐20card: 1
name: RESETtype: BYTEflow: inputcard: 1
VOLT_CHANNEL_2
Hardware Dataset
33venerdì 14 dicembre 12
11/dic/2011 34
Hardware DATASET
The JSON visualiza>on of BSON DS representa>on
{“device_id”: DEVICE_ID
“device_ds”: { "name" : "VOLT_CHANNEL_1", "desc" : "Output volt...", "attr_dir" : 1, "type" : 4, "cardinality" : 1 ! }
....
}
VOLT_CHANNEL_1
RESET
VOLT_CHANNEL_2
34venerdì 14 dicembre 12
11/dic/2011 35
!CHAOS Communica>on Systems
35venerdì 14 dicembre 12
36
!CHOAS has three different communicaIon system:
RPC, used for command (messages that need to be securely received and
recorded and for message that need a response [ asynchronous ])
Events, used for inform to a wide range of nodes, that something has
happened (device heartbeat, alert on device channel, received command,
etc..)
Direct I/O, used for streams to and from the live, historical data or inter
node data exchange
Communica>on Systems
36venerdì 14 dicembre 12
37
Every !CHAOS node has at last one
client and server instance of any of
the three system
Communica>on Systems
37venerdì 14 dicembre 12
38
•RPC layer is implemented as a plug-‐ins System, it's is abstracted to internal CHAOS framework so we con change it
• used to send important messages that need to be sure of it's delivering status
• can be used to receive asynchronous responses• the set command for input channel on control unit is done with RPC
RPC System
38venerdì 14 dicembre 12
39
RPC Message Flow• request message is constructed with BSON, serialized and sent to RPC end point• if the requester want a response it put answer-‐code and sender-‐address in BSON pack,and automa@cally RPC System wait for the answer• the request and response involves the forwarding of the ack message, from the receiver to the sender in synchronous mode
RPC System
Node A Node B
Messagge forwarding
ack [sync]Node A Node B
Response forwarding
ack [sync]
Message Answer [opIonal]
39venerdì 14 dicembre 12
40
!CHAOS rpc system
• Two kind of data forwarding:•Message, the informaBon are sent to remote host
•Request, the informaBon are sent to remote host, and an answer is
awaited
•In either types of data forwarding, a result of submissions is sent to the
client when a server has received the data
40venerdì 14 dicembre 12
41
RPC ac>on message
•In either types of data forwarding, the data informaBon are stored into a
k/v message:
•NODE_DST, internal MessageBroker address[string]
•ACT_ALIAS_DST, alias for the call acBon on the node[string]•ACT_MSG, k/v object used as param for the acBon[BSON]
“NODE_DST”: “3b95d9b5”“ACT_ALIAS_DST”: “alias”“ACT_MSG”: message
BSON Document for the message
41venerdì 14 dicembre 12
42
RPC ac>on request
the request, add some other informaBon to the message
BSON Document for the request
“NODE_DST”: “3b95d9b5”“ACT_ALIAS_DST”: “alias”“ACT_MSG”: messagem
essa
gela
yer
requ
est
laye
r
“ANSW_ID”: 1“ANSW_NODE”: “44342364”“ANSW_ACT”: “alias”“ANSW_IP”: “ip:port”
42venerdì 14 dicembre 12
43
!CHAOS submessage
• a message can bring a sub message
• the sub message is dispatched aLer the main message has been
successfully executed
43venerdì 14 dicembre 12
44
!CHAOS submission result•the message or request submission is a synchronous operaBon
•aLer received the server send back to the client the result, wrapped into a BSON object with these key:
•SUBMISSION_ERROR_CODE, integer number that represents the
error code[0 mean no error]
•SUBMISSION_ERROR_MESSAGE, string that represents what is
happened
•SUBMISSION_ERROR_DOMAIN, string that represents the locaBon
where the error is occurred
“SUBMISSION_ERROR_CODE”: 1“SUBMISSION_ERROR_MESSAGE”: “error string”“SUBMISSION_ERROR_DOMAIN”: “class::method”su
bmis
sion
resu
lt
44venerdì 14 dicembre 12
45
!CHAOS answer to a request
•the answer to a request, is composed by:
• the error fields shown in previous slides, in case of error there will be all tree
• the BSON object that represents the answer, in case of success
“SUBMISSION_ERROR_CODE”: 1
“ACT_MSG”: Result BSON Docansw
erre
sult
45venerdì 14 dicembre 12
09/dic/2011 46
• Event is implemented as a plug-‐ins System it's is abstracted to internal CHAOS framework so we con change it
• there are four type of event:• Alert (highest priority)• Instrument
• Command
• Custom (user specific event type)
• all informaAon are packed into 1024 byte and transported by protocols that support mulAcast (UDP, PGM[hVp://en.wikipedia.org/wiki/PragmaAc_General_MulAcast])
Event System
46venerdì 14 dicembre 12
09/dic/2011 47
Some numbers about event data
• Every event type has 2^16 sub type• Every sub type can have up to 2^16 priority level• Every subtype can have up to 255 byte of informaAon
Event System
47venerdì 14 dicembre 12
09/dic/2011 48
• Direct I/O is implemented as a plug-‐ins system
• used to make insert or select into the live and the history data
• can be used as stream for receive result data of the query or read the stored raw data file of the devices
• fastest way to transfer data between node
Direct I/O System
48venerdì 14 dicembre 12
49
Communica>on Systems
NetworkBrokerRPC Event Direct I/O
Client Server Client Server Client Server
Network
CommonToolkit, implementa a so_ware switch that permit, to all other class, to access all three protocoll.
Node NodeNode
Actio
n
Actio
n
RPCChannel
Answer Actio
n
EventChannel
Handler
49venerdì 14 dicembre 12
50
NetworkBroker For RPC and Event
DeclarAction
EventAction
CPP Class
NetworkBroker
RegisterAction
addActionDescritionInstance
virtual void handleEvent(const event::EventDescriptor * const event) = 0;
RegisterEventAction
50venerdì 14 dicembre 12
51
NetworkBroker For RPC and Event
the dispatcherfind des>na>on queue
Domain Queue is filled
ac>on
NetworkBrokerSERVER
Client
Communication Implementation
data
data
data
data
data
51venerdì 14 dicembre 12
52
NetworkBroker For RPC and Event
MessageChannel
NetworkBroker
Get Message Channel API
Get Event Channel APIEventChannel
DeviceMessageChannel
MDSMessageChannel
AlertEVTChannel
DeviceEVTChannel
CmdEVTChannel
CustomEVTChannel
52venerdì 14 dicembre 12
53
NetworkBroker For RPC and Event
MessageChannel
EventChannel
security check for
who can send what(TODO)
GetMessageChannel
GetEventChannel
NetworkBroker
53venerdì 14 dicembre 12
11/dic/2011
!CHAOS ControlUnit toolkit
54venerdì 14 dicembre 12
•CUToolkit help the Control Unit development
•A ControlUnit is and “Instrument” driver that aBach an hardware to !CHAOS
•Developer need to extend only one class to create a CHAOS driver, the “AbstractControlUnit”
•AbstractControlUnit expose all the APIs needed for interac@ng with !CHOAS
CU Toolkit
55venerdì 14 dicembre 12
•defineAc@onAndDataset•init•run•stop•deinit
AbstractControlUnit
Write a Control Unit
AbstractControlUnit is an abstract cpp class, that force developer to implement some method
Common Toolkit
Control Unit Toolkit
developer need to extends this class to create a !CHOAS device driver
56venerdì 14 dicembre 12
Write a Control UnitdefineAcIonAndDataset, is the method where the developer needs, using api, describe the aBached device.
void setDefaultScheduleDelay(int32_t);
void addDeviceId(string);
void addAttributeToDataSet(const char*const deviceID, const char*const attributeName, const char*const attributeDescription, DataType::DataType attributeType, DataType::DataSetAttributeIOAttribute attributeDirection);
Add a new device iden@fica@on to the control unit
Add a global default scheduling rate for run method
Add a new aBribute to the device dataset
57venerdì 14 dicembre 12
Write a Control Unit
template<typename T>void addInputXXXXAttributeToDataSet(const char*const deviceID, const char*const attributeName, const char*const attributeDescription, T* objectPointer, typename TDSObjectHandler<T, XXXX>::TDSHandler objectHandler)
Add a new input aBribute of the type XXXX to the device dataset giving ha handler for manage it
typedef void (T::*TDSHandler)(const std::string & , const XXXX&);
The handler is every method of an class that respect this prototype:
58venerdì 14 dicembre 12
Write a Control Unit
A device driver can also define custom ac@ontemplate<typename T>AbstActionDescShrPtr addActionDescritionInstance(T* actonObjectPointer, typename ActionDescriptor<T>::ActionPointerDef actionHandler, const char*const actionAliasName, const char*const actionDescription);
typedef CDataWrapper* (T::*ActionPointerDef)(CDataWrapper*, bool&);
An ac@on is every method of a class that respect this prototype:
59venerdì 14 dicembre 12
Write a Control Unit
init, is the method where the developer needs to ini@alize his hardware.
The local dataset is filled, from controlling client, with the current default value, for every channel or custom property. The developer can read these value and setup some useful things.
60venerdì 14 dicembre 12
Write a Control Unit
Send a data buffer to the output buffer for a determinated device idvoid pushDataSetForKey(const char *deviceID, CDataWrapper*);
CDataWrapper *getNewDataWrapperForKey(const char*); Return a new CDataWrapper prepared to be push on data output queue
All the data acquired in the driver, that are associated to the output channels, need to be insert into a CDataWrappedObject to be published
The data is filled with the default chaos key for: @mestamp, deviceID, triggerCode
run, is the method where the developer needs, using api, check the state of the device and acquire the value of output channel, or make control logic
61venerdì 14 dicembre 12
Write a Control Unit
stop, inform the developer that has been stopped the device
deinit, inform the developer that has been requested the deini@aliza@on of the device. So all resource of the hardware need to be deallocated.
62venerdì 14 dicembre 12
Write a Control Unit
• A CU is a C++ class.
• CUToolkit help to create a deamon, and take care to:
• talk with MDS for registra@on;
• make periodical check for network(s@ll in development) availability;
• take care to configure all driver with all the ip for the server that will manage the acquired data;
• other secret things...
summarizing
63venerdì 14 dicembre 12
Write a Control UnitA typical CUToolkit main for a CU, is the code below:
int main (int argc, char* argv[] ) {
// Initialize the CUToolkitChaosCUToolkit::getInstance()->init(argc, argv);
// Add a new control unit identified by the class WorkerCUChaosCUToolkit::getInstance()->addControlUnit(new WorkerCU());
// Start the all the server and client for RPC and execute all// work to startup and register the instance of WorkerCU class// and wait an shutdown command via RPC or SIGINT signalChaosCUToolkit::getInstance()->start();
return 0;
}
the method ChaosCUToolkit::getInstance()->addControlUnit(...) can be called for other instance of WorkerCU Class or other control unit
64venerdì 14 dicembre 12
11/dic/2011 65
!CHAOS UI Toollkit
65venerdì 14 dicembre 12
UIToolkit helps the user interface development
Gives API that help to develop: monitoring processes, device controller etc...
Gives an high performance queue using "locking free" design. The queue can manage quan@za@ons , over @me, for the dataset data acquisi@on
UI Toolkit
UIToolkit
Live Acquisition Thread Pool
Absr
actIO
Drive
r
Absr
actIO
Drive
r
DeviceControl DeviceControl
InterThread Cache Logic
Network
66venerdì 14 dicembre 12
At the moment the whole package of UI is in study but some things has been done to help developer to control hardware.
The DeviceController class can help developer to connect to a specific device using the ID
UI Toolkit
67venerdì 14 dicembre 12
UI Toolkit
DeviceController
MDSChannel
DeviceChannel
EventHandler
Directory Server
DeviceID
Live Data DriverMemcached Instances(for now)
68venerdì 14 dicembre 12
UI Toolkit
A “c” interface is provide to access UIToolkit into sofware that don’t permit to use C++ as developer language.
This permit to realize an user interface or controller algorithm into another soZware as plugin, as for example LabView.
The “C” proxy has a minimal set of interface to simplify the device controller alloca@on, dealloca@on and usage.
69venerdì 14 dicembre 12
11/dic/2011 70
!CHAOS EU Toolkit
70venerdì 14 dicembre 12
helps the development of distributed algorithms (Execu@on Unit) that can be used to compute or automate the devices control
The EU can have, in input the device channels or all the Dataset descrip@on, so it work on a "Class" of device not directly on one device.
The EU can be also used to perform calcula@on instead control algoritm;
on EU can be preset in many instance distributed on different fiscal server
EU Toolkit
71venerdì 14 dicembre 12
EU Toolkit
Input Parameter Output Parameter
ExecutionQueue
Result Queue
Specialized Execution Unit
Abstract Execution Unit
Input Param Pack Output Result Pack
scheme of flow of input or output of an EU
72venerdì 14 dicembre 12
EU Toolkitexcep@on could be concatenated crea@ng a sequence of any combina@on of:
• CompuIng algorithm
• Control algorithm
Input Parameter Output Parameter
ExecutionQueue
Result Queue
Specialized Execution Unit
Abstract Execution Unit
Input Param Pack Output Result Pack
Input Parameter Output Parameter
ExecutionQueue
Result Queue
Specialized Execution Unit
Abstract Execution Unit
Input Param Pack
73venerdì 14 dicembre 12
14/dic/2012 74
Execu>on Unit Example 1Thinks about an EU that make some logic on CU data and control another CU
The EU read the needed data from the Output aBribute of the HW from the CU_1 data
The EU write the result of computa@on to a CU_2 for set the Input aBribute of the HW
CU_1 CU_2
74venerdì 14 dicembre 12
14/dic/2012 75
Execu>on Unit Example 1this is the real data flow
live dataCU_1 DATA BLOCK
The Input ABribute on CU_2is set with RPC command
Output CU_1 aBributedata is read from live
Output aBribute data is pushed
on live memory blockfrom CU_1
some logic is done
CU_1 CU_2
Ouput aBribute data isread from device
The ABribute is set on HWby CU_2
75venerdì 14 dicembre 12
14/dic/2012 76
Execu>on Unit Example 2
Now think to an execu@on unit that send output to another execu@on unit and so on...
the idea is to create a simil-‐language for describe a “program” that can be executed by a “sequence” of EU;
76venerdì 14 dicembre 12
11/dic/2011 77
!CHAOS Data Management
77venerdì 14 dicembre 12
78
Data Management
!CHOAS has two different management system for the data apparatus
controlled environment data
Live DataHistory Data
78venerdì 14 dicembre 12
11/dic/2011 79
!CHAOS Live Datalive
cachedev_1
push data
ui_1
ui_2
eu_1
79venerdì 14 dicembre 12
80
Live Data
The system of the Live Data, is ideated for the sharing of the real @me
state of the instruments (CU) to other nodes
80venerdì 14 dicembre 12
81
Live Data
1. every node can read data from the central cache at any Rme[polling]
2. the nodes can register to central cache for receive update[push]
livecachedev_1
push data
ui_1
ui_2
eu_1
Every instrument pushes the data of his I/O channels into the shared
cached through the controller CU
81venerdì 14 dicembre 12
82
Live Data
livecachedev_1
push data
ui_1
ui_2
eu_1
livecachedev_2
push data
scalability & reliability
All data of all instruments are
distributed across many cache server
82venerdì 14 dicembre 12
83
Live Data
livecachedev_1
push data
ui_1
ui_2
eu_1
livecachedev_2
push data
X
scalability & reliability
with the algorithm previously shown,
when one server goes down, all client
that push's, or read, data on that
server, use another server, with highest priority
83venerdì 14 dicembre 12
84
Live Data
The usage of the two modes permits to manage, more efficiently, the
bandwidth of the flows of the real @me data (read and write)
not all nodes need real@me data at the same speed:
• the mode (1) is for monitoring and control
• the mode (2) is for control algorithm that can’t lose packages
84venerdì 14 dicembre 12
11/dic/2011 85
!CHAOS History Data
85venerdì 14 dicembre 12
86
Live & History Data
Storage Driver
Index Driver
Com
mon Layer
Abstract History Logic
Proxy CQL Interpreter
Cache D
river
Cache Logic
ChaosQL Proxy
the live data management is merged into history data management into
the ChaosQL node with the Cache Driver
86venerdì 14 dicembre 12
87
History Data
The System for History Data has been designed to efficiently
store the serialized data pack, manage different type of index on
demand, and managing data file
87venerdì 14 dicembre 12
History Data
Storage Driver
Index Driver
Common LayerAbst
ract
His
tory
Log
ic
CQ
L In
terp
rete
r
Storage Driver
Index Driver
Common LayerAbst
ract
His
tory
Log
ic
Man
agem
ent I
ndex
Log
ic
Storage Driver
Index Driver
Common LayerAbst
ract
His
tory
Log
ic
Stor
age
Man
agem
ent L
ogic
It is realized by three different services and two driver
ChaosQL Proxy Indexer Node Management Node
Storage Driver Index Driver
Storage Driver abstrac@on Index Driver abstrac@on
88venerdì 14 dicembre 12
89
Chaos Query Language
ChaosQL implement the following operaBon on data
• push history/live data per device• create index on device::aXribute::{rule}• delete index on device::aXribute• retrieve data with logic operaBon on aXribute and index
89venerdì 14 dicembre 12
90
History Data
Storage Driver
Index Driver
Common LayerAbst
ract
His
tory
Log
ic
CQ
L In
terp
rete
r
ChaosQL Proxy
Manages the insert and select opera@ons
by parsing the Chaos Query Language
Insert -‐ as fast as possible stores the data into the storage system
select -‐ decodes the query, asks at the indexer the posi@on in the storage of the
data that match the criteria
90venerdì 14 dicembre 12
91
History Data
All nodes, using the ChaosQL Driver (that
uses direct I/O) can query the history and
live data.
the results are received in asynchronous
mode
Storage Driver
Index Driver
Comm
on Layer
Abstract History Logic
Proxy CQL Interpreter
UI or CU or EU
UI-CU-EU Toolkit
Chaos Common
Query Re
sult
CQL I/O Driver
Direct I/O Driver
91venerdì 14 dicembre 12
11/dic/2011
Index Database
Storage
CommonTK
CUTK
CQL Proxy
CQL Proxy
Push Data CQL
Prox
y
Indexer Cluster
Store Data
Launch Index
get data pack
Store Index
Management Cluster
92venerdì 14 dicembre 12
11/dic/2011
Index Database
Storage
CommonTK
CUTK
CQL Proxy
CQL Proxy
CommonTK
EUTK
UITK
CommonTK
CQL
Prox
y
Query Live Data
Query History Data
Indexer Cluster
send search on idx
execute search on idx DB
get pack at locaIon
Receive History Data
Management Cluster
93venerdì 14 dicembre 12
94
History Data
the query can be spliXed in more than one Bme
period from the receiving proxy.
The spliXed query are sent to other proxy that
will process the sub query
Storage Driver
Index Driver
Comm
on Layer
Abstract History Logic
Proxy CQL Interpreter
UI or CU or EU
UI-CU-EU Toolkit
Query
Resu
lt
Storage Driver
Index Driver
Comm
on Layer
Abstract History Logic
Proxy CQL Interpreter
Storage Driver
Index Driver
Comm
on Layer
Abstract History Logic
Proxy CQL Interpreter
Result
CQLPackOriginal
CQLPack1
CQLPack2
CQLPack3
Sub Query
Result
QueryChannel
ChaosCommon
CQL I/O Driver
Direct I/O Driver
94venerdì 14 dicembre 12
95
History Data
Executes the necessary opera@ons to index
the data saved on the storage system
two type of index:
• TMI -‐ Time Machine index
• DSI -‐ Dataset Index
Storage Driver
Index Driver
Common LayerAbst
ract
His
tory
Log
ic
Man
agem
ent I
ndex
Log
ic
Indexer Node
95venerdì 14 dicembre 12
96
History Data
TMI -‐ Time Machine index
•This index permits to catalog every data pack for a device for the
@mestamp.
• all the data pack are recorded and associated to their @mestamp
• this index is managed by system
96venerdì 14 dicembre 12
97
History Data
DSI -‐ Dataset Index
•This index permits to catalog aBributes of the dataset for a
determinated instrument
• there will be possibility of customize the index algorithm
• this index is created on demand and on specific @me periods
97venerdì 14 dicembre 12
98
History Data
Executes management opera@ons on files
that store the data packs
some data pack could be subject to ageing
the ageing state will determine the erase
or long term storage of a data pack
Storage Driver
Index Driver
Common LayerAbst
ract
His
tory
Log
ic
Stor
age
Man
agem
ent L
ogic
Management Node
98venerdì 14 dicembre 12
23/02/12INFN-‐Vitrociset mee@ng
Directory server
CU Toolkit
CU
modules (init)
CU
modules (config)
CU
modules (set)
CU
modules (de-init)
Control UnitC
U m
odules (stop)
infrastructure
Control Unit
CU
modules (init)
CU
modules (config)
CU
modules (set)
CU
modules (de-init)
CU
modules (stop)
CU-LV interface
distributedHistory
Data service
distributedLive Dataservice
I/O m
odules
I/O Unit Exec Unit
I/O m
odules
I/O m
odules
EU m
odules
EU m
odules
EU m
odules
Common Toolkit
C User Interface LV User Interface MatLab UI
User Interface Toolkit 20%
Common Toolkit
Project Status
99venerdì 14 dicembre 12
23/02/12INFN-‐Vitrociset mee@ng
infrastructure
developments’ details
LV User Interface: prototype in preparation (shlib for LV)
Live Data: single-instance (no-distributed) prototype ready
RPC client/server: prototype ready (msgpack and zmq impl)
History Data service: performance test (mongodb) ongoing (UniTV)
Directory services: DS simulator ready; DS under developmentscalability & availability: test facility in preparation
UI Toolkit: prototype ready
Common Toolkit: prototype ready
CU Toolkit: prototype ready
CU modules: prototype ready (C++)
CU-LV Interface prototypes& modules: almost ready
QT User Interface: prototype ready (Qt)
Event client/server: prototype ready (boost asio impl)Direct I/O c/s: todo
C wrapper UI Toolkit: prototype ready
100venerdì 14 dicembre 12
101
...closing
in the next year all development force will be focused to realize the
system for the history data and for the directory service, a prototype for
each system will be available for 2013 2Q
101venerdì 14 dicembre 12
102
thanks for the >me
CHOAS GroupClaudio Bisegni, Luciano Catani,
Giampiero Di Pirro, Luca FoggeBa,
Mazziteli Giovanni, MaBeo Mara, Antonello Paolew, Stecchi Alessandro,
Federico Zani
thesis'sFlaminio Antonucci (tor Vergata
Ingengeria), Andrea Capozzi (Tor Vergata Ingengeria), Francesco Iesu (Cagliari
Infroma@ca),
102venerdì 14 dicembre 12