![Page 1: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/1.jpg)
José Augusto Suruagy (UFPE)
Joberto S. B. Martins (UNIFACS)
2nd perfSONAR WorkshopArlington – February 20, 2014
Monitoring in Federated Future Internet Testbeds: the FIBRE case
![Page 2: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/2.jpg)
The FIBRE Project
The main objective of this project is to create a common space between the EU and Brazil for Future Internet (FI) experimental research into network infrastructure and distributed applications, by building and operating a federated EU-Brazil Future internet experimental facility
The project is designing, implementing and validating a shared Future Internet research facility between Brazil and Europe, supporting joint Future Internet
experimentation of European and Brazilian researchers
2
![Page 3: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/3.jpg)
FIBRE Partners
3
![Page 4: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/4.jpg)
FIBRE Testbeds
4
![Page 5: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/5.jpg)
Monitoring & Federation Context•Monitoring Scope:
• Infrastructure:• Objective: support NOC/administrators in FIBRE facility management and
experimenters in finding out alternatives in terms of available and usable resources (links, machines, switches, …)
• Experiment (slice):• Objective: support the experiment itself monitoring parameters like,
throughput, loss, … in a specific slice (experiment context)
•Monitoring Types:• Active Measurements• Passive Measurements
•Measurement tools per CMF:• OCF: none native• OMF: OML (-> GIMI)• ProtoGENI: LAMP (pS) + INSTOOLS (-> GEMINI)
5
![Page 6: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/6.jpg)
Monitoring & Federation Context•Portal/Monitoring GUI and gateway(s) global view:
6
OCF OMF ProtoGENI
Portal/ Tool
GWMDIP, TopHat GW, …
GW
FIBRE-BR I&M Portal,TopHat, …
GW
Format
CMFs (BR & EU;communities)
![Page 7: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/7.jpg)
Monitoring & Federation Context (Related but Independent Solutions)
![Page 8: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/8.jpg)
Monitoring Federation Use CasesMeasurement Configuration and Types
• Measurement Configuration:• Intra-CMF• Inter-CMF
• Measurement Use Cases• Infrastructure measurements:
• UC 1: Active measurements• UC 2: Passive measurements
• Experiment/Slice measurements• UC 3: Intra-CMF• UC 4: Inter-CMF
8
![Page 9: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/9.jpg)
Monitoring Federation Use CasesMeasurement Configuration Alternatives
• “Intra” and “Inter”- CMF measurements
9
![Page 10: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/10.jpg)
Monitoring Federation Use CasesInfrastructure Measurements
• UC 1: Infrastructure active measurements
10
![Page 11: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/11.jpg)
Monitoring Federation Use CasesInfrastructure Measurements
UC 1: Infrastructure active measurements• Proposal
– Install perfSONAR PS Toolkit nodes at each FIBRE island– Provide a Dashboard at NOC with global view of the
testbed’s performance– Develop a gateway to translate perfSONAR data to TopHat
standard (in progress)
11
![Page 12: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/12.jpg)
Monitoring Federation Use CasesInfrastructure Measurements
• perfSONAR Dashboard prototype implementation
12
![Page 13: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/13.jpg)
Monitoring Federation Use CasesInfrastructure Measurements
• UC 2: Infrastructure passive measurements – Monitoring a variety of metrics (CPU, memory, NIC
utilization, etc.)
13
![Page 14: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/14.jpg)
Monitoring Federation Use CasesUC 3: Intra-CMF measurements
14
Portals
MI Service
Data PlaneControl Plane
4
ii2
MDA
iv
PortalsGUIs
A multi-CMF Slice
OCF
intra-CMF measurement Experimenter
GatewayAPI
1
3
i
iii
ProtoGENI
GatewayAPI
![Page 15: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/15.jpg)
• UC 3: Intra-CMF measurements• Proposal:
– CMF’s native I&M software (e.g.: GEMINI on ProtoGENI) is responsible for the measurement management and configuration (the experimenter will use the CMF’s I&M Portal)
– Gateways (such as MDIPs) are responsible for exposing the measurement data collected by the CMF’s I&M software in a standardized format
15
Monitoring Federation Use CasesExperiment/Slice Measurements
![Page 16: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/16.jpg)
16
Monitoring Federation Use CasesUC3: ProtoGENI MDIP (Implemented)
![Page 17: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/17.jpg)
17
Monitoring Federation Use CasesUC3: ProtoGENI MDIP (Proposed)
![Page 18: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/18.jpg)
Monitoring Federation Use CasesUC3: OCF MDIP
![Page 19: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/19.jpg)
19
Monitoring Federation Use CasesUC 4: Inter-CMF measurements
Portals
MI Service
Data PlaneControl Plane
MDA
PortalsGUIs
A multi-CMF Slice
Experimenter
Orch.Service
1
24
OCF
ProtoGENI
GatewayAPI
GatewayAPI
M. Tool
M. Tool
3 inter-CMF measurement
![Page 20: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/20.jpg)
• UC 4: Inter-CMF measurements• Proposal:
– Each node, regardless of “its CMF”, comes with a common measurement tool pre-installed. This way, measurements become possible between any node pair
– A gateway abstracts the configuration and data exposal complexities of the common tool through a standard API, so:
– The experimenter can use ANY Portal to configure measurements and retrieve data
• Alternatives:– perfSONAR BUOY– OML
20
Monitoring Federation Use CasesExperiment/Slice Measurements
![Page 21: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/21.jpg)
21
Monitoring Federation Use CasesUC4: Instrumentation Phase
ProtoGENI slice
Orchestration Service
PortalsPortalsPortals
REST
AP
I
registerSlice()addNode()
instrumentize2
1
![Page 22: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/22.jpg)
22
Monitoring Federation Use CasesUC4: Measurement Configuration Phase
ProtoGENI slice
Orchestration Service
PortalsPortalsPortals
1
Agent
GW A P IBUOY
BUOY
measure!
Agent
GW A P I
auto-update REST
AP
I
addTest()
2
3
3 fetch data
![Page 23: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/23.jpg)
Goal:• Turn OML into a perfSONAR-enabled solution
• PS-like query for experiment data:• NMWG measurement data format
• All communication will be made through Web Services to receive and retrieve data
23
Monitoring Federation Use CasesUC4: OML MDIP
![Page 24: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/24.jpg)
Monitoring Federation Use CasesUC4: OML MDIP
24
![Page 25: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/25.jpg)
• Testbeds Federation x Multi-Domain – Specially for Infrastructure measurements (each testbed uses its own
tools)
• Need for a common data measurement report format and protocol– Webservices, NMWG (XML x JSON), NML, extended with other
resources (servers, VMs, etc.) measurement report formats
• Need for measurement tool gateway in case of similar but heterogeneous tools– E.g., Owamp x ETOMIC
• Extend SFA with a sort of MSpec (Measurement Specification)– To retrieve, reserve (if necessary), schedule tests, and fetch data from
available infrastructure measurement resources
25
Monitoring Federation in FI TestbedsChallenges/Issues
![Page 26: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/26.jpg)
• Universal Measurement Information?• Measurement software tools deployed as experimenter own
software:– Hopefully without requiring that the experimenter be a measurement expert– Avoids heterogeneity problems
• Specialized Hardware Measurement Tools:– E.g., DAG cards, wireless band interference monitors, etc.– Availability and heterogeneity issues
• Persistent data archiving (format and location issues)• Experiment repeatability issues:
– Specialized resources may not be always available
26
Monitoring Federation in FI TestbedsChallenges/Issues
![Page 27: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/27.jpg)
• Constraint based resource orchestration:– Orchestration agent for each CMF and one summary agent for the island
27
Monitoring Federation in FI TestbedsChallenges/Issues
![Page 28: Monitoring in Federated Future Internet Testbeds: the FIBRE case](https://reader033.vdocuments.us/reader033/viewer/2022051818/54c2340f4a7959fd618b45b1/html5/thumbnails/28.jpg)
Thank you José Augusto Suruagy/[email protected]
Joberto S. B. Martins/[email protected]
twitter.com/FIBRE_project
www.facebook.com/fibre.project
www.fibre-ict.eu
28