otn d2.5 architecture blueprint and hub technical...
TRANSCRIPT
This project has received funding from the European Union’s Competitiveness and Innovation Framework Programme under grant agreement no. 620533.
DELIVERABLE Project Acronym: OTN Grant Agreement number: 620533 Project Full Title: OpenTransportNet – Spatially Referenced Data Hubs for
Innovation in the Transport Section
D2.5OTN PROJECT ARCHITECTURE BLUEPRINT AND HUB TECHNICAL
SPECIFICATION
Version: 1.0 Authors: Babis Ipektsidis (INTRA) Evangelos Argyzoudis (INTRA) Leonidas Kallipolitis (ATC) Dmitrii Kozhukh (HSRS)
Pieter Colpaert (iMINDS) Karel Charvat (HSRS) Karel Jedlicka (UWB)
Internal Reviewers: Lieven Raes (CORVE) Karel Charvat (HSRS) Karel Jedlicka (UWB) Indulis Makens (EX)
Eva Jaho (ATC) Pieter Colpaert (iMinds) Jan Martolos (EDIP)
Dissemination Level
P Public X
C Confidential, only for members of the consortium and the Commission Services
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 2
Table of Contents Table of Contents ................................................................................................................ 2
List of Tables ..................................................................................................................... 4
List of Figures .................................................................................................................... 4
Revision History .................................................................................................................. 5
1 Introduction .................................................................................................................. 6
1.1 Purpose and scope ..................................................................................................... 6
1.2 Structure of the deliverable .......................................................................................... 6
2 System Specifications ...................................................................................................... 7
2.1 Functional Requirements ............................................................................................. 7
2.2 Non-functional requirements ....................................................................................... 10
2.3 Technical Specifications ............................................................................................ 12
3 OTN Hub Architecture .................................................................................................... 17
3.1 Architectural Design Methodology ................................................................................. 17
3.2 High level Overview ................................................................................................. 19
3.2.1 RESTful Web Services ........................................................................................... 21
3.3 Platform Technology Stack ......................................................................................... 21
3.3.1 Java EE 7 .......................................................................................................... 21
3.3.2 Linux Ubuntu Operating System ............................................................................... 21
3.3.3 Liferay Portal .................................................................................................... 22
3.4 System Workflows .................................................................................................... 26
3.4.1 Link an existing dataset to the OTN Hub .................................................................... 26
3.4.2 Create a mashup ................................................................................................ 28
3.4.3 Upload geospatial data to Hub’s internal storage .......................................................... 30
3.4.4 Use a mobile device to upload VGI to the Hub ............................................................. 32
3.4.5 Receive notifications about a traffic accident .............................................................. 34
4 OTN Hub Components .................................................................................................... 36
4.1 Front End User Interface ............................................................................................ 36
4.1.1 Configuration Management .................................................................................... 37
4.1.2 Plan Integrator ................................................................................................... 37
4.2 Back-End Data Aggregator and Open APIs ........................................................................ 37
4.2.1 Data Harverster .................................................................................................. 37
4.2.2 Data Harmonisation ............................................................................................. 38
4.2.3 OTN Data Catalogue ............................................................................................ 39
4.2.4 Open APIs ......................................................................................................... 39
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 3
4.3 Geospatial Metadata Catalogue .................................................................................... 40
4.3.1 MICKA ............................................................................................................. 40
4.4 Visualisation Components ........................................................................................... 43
4.4.1 Thematic Map Viewer ........................................................................................... 43
4.4.2 Location Evaluator .............................................................................................. 44
4.4.3 Map Creator ...................................................................................................... 44
4.4.4 Embed Map ....................................................................................................... 45
4.4.5 Mobile Applications ............................................................................................. 45
4.5 Geospatial Middleware .............................................................................................. 47
4.5.1 GeoServer ........................................................................................................ 47
4.5.2 Analysis Engine .................................................................................................. 50
4.5.3 Integration Engine ............................................................................................... 51
4.5.4 Traffic Models .................................................................................................... 51
4.5.5 SensLog ........................................................................................................... 53
4.5.6 Maplog ............................................................................................................ 60
4.5.7 Mapserver ......................................................................................................... 63
4.6 Service Marketplace ................................................................................................. 65
4.7 OTN Community Forum ............................................................................................. 65
4.8 Access Control and Identity Management System ............................................................... 66
4.8.1 Security Risks and Mitigation .................................................................................. 66
4.9 Storage ................................................................................................................. 68
4.9.1 Primary data Storage ........................................................................................... 68
4.9.2 Secondary Data Storage ........................................................................................ 69
4.9.3 Semantic Indexing Infrastructure ............................................................................. 69
4.9.4 External Storage Connector ................................................................................... 70
4.9.5 CMS and Datatank Storage ..................................................................................... 71
5 Conclusion .................................................................................................................. 72
ANNEX: Technical Questionnaire ............................................................................................ 73
INTRODUCTION .............................................................................................................. 73
QUESTIONS ................................................................................................................... 73
1. Questions about your component ................................................................................... 73
2. Development Needs ................................................................................................... 75
3. Interoperability Requirements ....................................................................................... 76
4. Security principles ..................................................................................................... 77
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 4
List of Tables Table 1 Functional Requirement ............................................................................................ 10 Table 2 Non-Functional Requirements ..................................................................................... 12 Table 3 Technical Specifications ............................................................................................ 16
List of Figures Figure 1: Layered Architecture .............................................................................................. 18 Figure 2: OTN Platform High Level Architecture ......................................................................... 20 Figure 3: Link an existing dataset to the OTN Hub sequence diagram ................................................ 27 Figure 4: Create a mashup sequence diagram ............................................................................ 29 Figure 5: Upload geospatial data to the Hub’s storage sequence diagram ........................................... 31 Figure 6: Use a mobile device to upload a VGI sequence diagram ..................................................... 33 Figure 7: Receive notifications about a traffic accident sequence diagram ......................................... 35 Figure 8 OTN Hub Components .............................................................................................. 36 Figure 9 System of Security Patterns realizing Access Control ......................................................... 68
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 5
Revision History
Version Date Author Organization Description
0.1 04/06/2014 Babis Ipektsidis INTRA ToC
0.2 11/07/2014
Babis Ipektsidis,Evangelos Argyzoudis, Leonidas
Kallipolitis,
INTRA, ATC First Draft
0.3 07/08/2014
Babis Ipektsidis, Evangelos Argyzoudis, Leonidas Kallipolitis,
Dmitrii Kozhukh
INTRA, ATC, HSRS Second Draft
0.4 19/08/2014 Evangelos Argyzoudis INTRA Updated document with input from iMinds, HSRS
and UWS
0.5 27/08/2014 Evangelos Argyzoudis, Babis Ipektsidis INTRA
Amendments based on comments from Internal
Reviewers
1.0 29/08/2014 Evangelos Argyzoudis, Babis Ipektsidis INTRA
Final version after consortium review for
submission
Statement of originality:
This deliverable contains original unpublished work except where clearly indicated otherwise. Acknowledgement of previously published material and of the work of others has been made through appropriate citation, quotation or both.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 6
1 Introduction
1.1 Purpose and scope The objective of this document is to present the overall architecture of the OTN platform. The current document outlines the platform architecture including a general description of the components layout, their internal architecture as well as the interactions among them. The flows and architecture of the OTN platform is defined in terms of the supported functionalities, the respective processes and the components that realise them.
This will serve as a reference point for the development work that will take place in the technical work packages. The decisions presented in this deliverable are subject to refinements and modifications, based on the progress of the technical work packages, as well as the validation and evaluation phases.
1.2 Structure of the deliverable The following sections in this deliverable are:
Section 2, where the OTN system specifications are presented, based on the user requirements elicitation process and resulting technical requirements.
Section 3 presents an analysis of the OTN Hub Architecture, which includes the constituent components and required technologies as well as the user workflows covered by the system.
Section 4 presents in more detail the software components that make up the OTN Hubs separated according to the high-level architectural block they belong to.
Section 5 concludes the deliverable
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 7
2 System Specifications
2.1 Functional Requirements The requirements of the system are defined based on the user needs in order to clearly identify what are the functionalities and attributes of the final product. To this end, in the context of WP2, a number of workshop meetings were conducted to facilitate the interaction between stakeholders, stimulate focused discussions and eventually lead to the elicitation of user requirements. A summary of major functional requirements is presented in the following table, in a numbered list so as to have an easy reference to the requirements during the development stages, and to facilitate the evaluation of the Hubs during the internal testing and pilot testing.
No Name Description Priority Pilot
FR01 White-Label Front End Hubs hosted as a standalone site. Must have Birmingham
FR02 Hub Access Registration using social media credentials should be offered. Nice to have All
FR03 User Profile Photo and a short profile will help build trust between users (e.g. Twitter). Should have All
FR04 Landing Page Key Hub features should be accessible directly from the landing page. Should have All
FR05 Visible stakeholder paths
Hubs to be structured to guide users through activity flows based on what they want to do – stakeholder categories: Discoverer, Sponsor, Innovator.
Should have All
FR06 Data Search
Data needs to be easily filtered and categorized to make it easy for a user to find relevant data in as few steps as possible (based on INSPIRE and DCAT metadata fields)
Filtering / data discovery
- thematic - location based
Must have All
FR07 Data Download
“Easy download” / “link to the dataset” options to be available for developers wanting to use open data outside of the Hub.
Must have All
FR08 Data Upload Data needs to be uploaded / linked / connected in a way that will immediately work with visualisation tools.
Must have All
FR09 Volunteer Data Users to be able to sign up and link their phone to Hubs to be a data volunteer
TBC, must have at least
all
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 8
No Name Description Priority Pilot
Different types of VGI:
- users verify/correct stuff (updating of data)
- users report issues (and geo-locate) – famous potholes
- live position tracking used for learnings
- OSM
(include trust management)
Levels:
- OSM - User generated VGI (from
smartphones etc.)
Distinguish data from verified source (from “stakeholders”) and non-verified user-volunteered data.
some types of VGI
FR10 External APIs Need to integrate APIs for non-transport related information such as weather. Must have Liberec
FR11 Data Notification
Option to ‘tell a friend’ or notify groups when a new data set is uploaded
Sign-up for notification of new datasets in selected categories.
Nice to have all
FR12 Data Licensing
Data licenses should be clear to all users
Different types / purpose of data
- analysed data used for additional services accessible only via user interface (e.g. Liberec crisis data)
- analysed data open for re-use under open license
- open data available for re-use (open license)
Must have all
FR13 Map Visualisations
Map visualisations should be large, clear and offer choice of POI locations or isochron mapping. Isochrones could be used in two ways. Firstly, isochron maps can be created in order to calculate time needed from different locations to reach hospital or some other important objects of infrastructure. Secondly, it could be offered as an API service that will calculate isochrones from a certain point by some time intervals entered as parameters.
Must have all
FR14 Map Notifications Map notification service to allow VGI Nice to have ANT
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 9
No Name Description Priority Pilot
contributions to a mapping service to send a notification to the map owner.
FR15 Map Use Ability to easily embed map visualization in own website, as well as print and/or save for future use.
Must have All
FR16 Optimal Routing Service
Routing service consuming different types of data sources (city-specific).
Include traffic volume calculation as a (pan-European) data source for routing.
Link to routing API that will allow the user to find optimal routes based on updates and traffic information.
Must have Liberec
ANT
FR17 Other Visualizations Clear, easy to read graphs and charts functionality.
To be detailed
FR18 Social Game Feature Users should be awarded points or badges (e.g. FourSquare) for activity on the Hubs.
Must have ANT, BCC, ISSY
FR19 Visualisations Catalogue
Area where you can see and access recent visualizations and what they were used for (to stimulate use of OTN tools)
Public and private setting needed (default is private).
Should have all
FR20 Skill Matching Engine
A rule-based search function linked to profile features should be put in place to allow for active matching on the basis of preferences and selected search criteria. Results should be filterable according to defined categories.
Nice to have all
FR21 Predictive Analysis Engine
Ability to access additional tools to help with data analysis (paid-for services?)
For example:
- routing service - traffic accidents prediction - traffic volume prediction - real speed prediction - Estimate time of arrival
prediction
Must have all
FR22 Map mash-up application
Application that allows users to combine data from HUB and display them on a single map. This may also be data from external sources in certain formats (WMS, GeoJson etc.)
Must have all
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 10
No Name Description Priority Pilot
FR23 Community
Group chat and one-on-one chat. Store chat sessions for later retrieval and any related information such as date, time and who was involved
- Forum - private messaging / chat
Must have (forum)
all
FR24 Business Mentoring Interface
A simple interface should be set up to allow one-on-one or small group discussion between business mentors and clients.
See above all
FR25 Payment Module
Payment module should be implemented using a major carrier or merchant to allow for transactions and for-fee services to be processed (e.g. some of the predictive services above).
Must have for
sustainability all
FR26 VGI Interface
Widget or form that allows users to quickly and simply submit a piece of information to a mapping dataset. This could either be based on a location service or on a clickable map.
Must have all
FR27 Users Notifications Users need to be notified when accidents happens along the route they have selected.
Must have all
FR28 Find all amenities in close distance
Users need to know all the available amenities in close distance. Must have all
FR29 Multi-language Multi-language functionality. The Hub interface should be multilingual. Should have At least
Liberec
FR30 Open APIs Developers need open APIs for use in their applications. Must have all
FR31 Pluggable Front End The Hub should be able to be integrated into a Council’s existing web presence Should have Issy, ANT,
Liberec
Table 1 Functional Requirement
2.2 Non-functional requirements In order to develop a robust and fully flexible system, which will cover all the needs of the OTN project, a set of non-functional requirements has to be considered. The following table summarizes these requirements and provides a short description for each one of them.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 11
No Name Description
NF01 Capacity • The Hub must be able to support 1,000 users joining and accessing over a one month period
NF02 Compliance • The Hubs must comply with and advance INSPIRE standards (and OGC and W3C standards)
NF03 Documentation • Provide full documentation so new cities can easily set up own Hub
NF04 Disaster Recovery
• Hubs to be monitored daily • Hub downtime to fix issues to be restricted to the shortest possible time
NF05 Extensibility
• The average person-time required to make a minor enhancement (including testing and documentation update) to the application between major releases shall not exceed one-person week.
• The average person-time required to make a significant enhancement (including testing and documentation update) during the development of a major new version of the application shall not exceed four person weeks
NF06 Fault Tolerance
• Hubs must be available 24/7 with an acceptable downtime of an hour early in the morning for any maintenance
NF07 Interoperability • Desirability for Hubs to work with other data platforms and routing systems to add value for businesses such as Supermarkets
NF08 Maintainability • Hub must be easy to maintain and update by the host City with a simple content management system
NF09 Privacy • Hubs must adhere to privacy requirements for personal information for each
demonstrator location • Each data set must be able to have set privacy requirements
NF10 Portability • Hubs to be optimized for mobile devices as well as the web (mobile at least
for apps and VGI tools) • Hubs to be tested across a range of browser types
NF11 Quality • 95% of users to be able to complete tasks without issues (feedback – forum
section, FAQs) • Focus on data quality management to be inbuilt into the system
NF12 Reliability
• The Hubs must respond in under 2 seconds for 95% of uses for standard requests.
• The Hubs must respond in under 20 seconds for 100% of uses, including advanced functionality, e.g. report generation.
NF13 Security • Multi-level access for city managers and end-users • Secure online payments for any advanced features
NF14 Scalability and Replicability
• Check capacity requirements with host to ensure server scalability • New Hubs must be quickly established in a cost effective manner
NF15 Usability • The OTN Hubs should have an attractive and intuitive User Interface. The interface needs to address various user groups and therefore should be easy
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 12
No Name Description
to use and give access to all system functionalities providing easy navigation through all features.
NF16 Accessibility • W3C Accessibility Guidelines should be followed to the extentpossible.
Table 2 Non-Functional Requirements
2.3 Technical Specifications The following table defines the technical specifications derived from the functional requirements, which were compiled by the pilots. Every technical specification has at least one relevant user requirement and provides a technical description of what the platform has to support. The prioritisation of the technical specifications has been made in accordance with the pilots’ views on the functionalities that must be definitely supported by the platform, the ones that should be available and a number of features that would be nice to have if their implementation does not affect the overall architecture or require significant development effort.
No Name Description User Requirement Priority
TS01 Data Harmonization
A set of SQL scripts, XML transformations and on-the-fly converters must be available in order to transform data to formats readable by the platform’s components.
FR08
(Data Upload) Must have
TS02 Data linking mechanism
The platform must have a form, which allows users to link existing online datasets.The datasets must be available as online resources with a specific URI. This form has to support a given set of metadata fields that covers geospatial data and any other form of open data. License must also be provided for every dataset.
A number of existing open data catalogues will be monitored by OTN regularly, so new published datasets to these catalogues will be automatically added to the OTN Data Catalogue.
FR08
(Data Upload), FR12 (Data Licensing)
Must have
TS03 Data model for mixed route upload
A data model to enable users to upload their preferred routes must be designed. This must support cases like, for instance, uploading a route from home to work by walking, driving and taking the metro.
FR16
(Optimal routing service)
Must have
TS04 Consume traffic Accidents API
The platform has to implement a process that will check whether traffic accidents received from certain APIs lie on the user’s
FR10
(External APIs), Must have
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 13
No Name Description User Requirement Priority
route (the route consists of different segments travelled by different transportation means).
FR27 (User Notifications)
TS05 Embeddable Map visualisation
The platform must offer map visualisations of the available data. These must be embeddable and saveable.
FR13 (Map Visualisations),
FR15 (Map Use) Must have
TS06 Timeline Data Model
Define data model of work schedule (i.e. road reparation schedule etc.).
FR17
(Other Visualizations)
Should have
TS07 Timeline Checker Create a function that will decide whether the work in the given locality is being done in accordance with planned time schedule.
FR21
(Predictive Analysis Engine)
Nice to have
TS08 Data verification (after data upload)
The platform must support a verification process. The verification depends on the role of the users uploading data. For example, after a registered user uploads a dataset, the administrator should be notified.If the dataset is verified, then the datasetwill be marked as verified when displayed to the public.
The platform must also support cases where uploaded data is not visible until its verified, e.g. editing POI information in existing datasets viewed on a map.
FR08 (Data Upload),
FR09 (Volunteer Data)
Must have
TS09 New POI/Info submission mechanism
There must be a way to submit a new piece of information for a specific point on a map or edit an existing one.
FR09 (Volunteer Data),
FR26 (VGI Interface)
Must have
TS10
Mobile application to collect data about users and send them to DB server
A mobile application that allows users to input data about their journey at different transport segments and at different times must be implemented. The data will be collected and will be used for analysis.
This app can also be extended to gather info about a number of current free parking places, at sharing points etc.
FR09
(Volunteer data)
Must have
TS11
Output from services should be in machine readable form, e.g. JSON
All services (i.e. routing, isochrones) provided through API should have output in JSON format.
FR30
(Open APIs)
Must have
TS12 Routing Different types of routing services should be FR16 Must have
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 14
No Name Description User Requirement Priority
available as part of web-applications as well as independent API services (classical routing from one point to another, routing with intermediate points, routing with blocks etc.).
(Optimal routing service)
TS13 Geocoding / Reverse geocoding API
Find closest address point by coordinates and reverse, find coordinates of the address point.
FR08 (Data Upload),
FR10 (External APIs)
Should have
TS14 Find closest amenity of type x API
Find closest amenity of type x (library, theatre, pharmacy etc.) to the given point.
FR10
(External APIs), FR28 (Find all amenities in close distance)
Must have
TS15 Metadata fields for uploaded datasets
A set of metadata fields for every uploaded dataset must be defined based on INSPIRE and DCAT.
FR06 (Data Search),
FR08 (Data Upload)
Must have
TS16 Data catalogue
A catalogue containing all the uploaded/linked datasets of the Hub. Searchingcriteria can be a geographical area or thematic category.
FR06
(Data Search) Must have
TS17 Computation of Isochrones Compute isochrones between two points.
FR13
(Map visualisation)
Nice to have
TS18 Display interactive graphs over map
Add some graphs/time sliders to interactive maps where needed (e.g. how intensity changes during day, structure of traffic in a segment etc.).
FR17
(Other Visualisations)
Nice to have
TS19 Intermodal journey planner
Support a process that checks transport network and transport schedule in order to find the fastest way to get from point a to point b.
FR16
(Optimal Routing Service)
Must have
TS20
Use Twitter API to post information about accidents
Use the traffic jams and accidents database of the Hub to update Twitter accounts. When a new accident is added to the database, post a new tweet using the Twitter API.
FR10 (External APIs),
FR16 (optimal routing service),
FR27 (Users notifications)
Nice to have
TS21 SMS notifications Support SMS notifications, for instance, by FR27 Nice to have
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 15
No Name Description User Requirement Priority
installing some SMS gateway that will send SMS to registered users.
(Users notifications)
TS22 Forum for users The platform must have a forum functionality to hold discussions and comments from the platform’s user groups.
FR23 (Community),
FR24 (Business Mentoring Interface)
Must have
TS23 Predictive Congestion Finder
This module will provide the means to create an application that will calculate how the traffic flows will redistribute in case of a road closure related to road works or other accident.
If the capacity of roads is insufficient, the application should help users find the closestpublic transport stations, P+R routes or bicycle hiring places.
FR21
(Predictive Analysis Engine)
Must have
TS24 Roadwork Timeline Mapping
Display on map parts of roads under construction together with some additional information related to reparation, such as contact person, predicted time the construction will be finished, if there is delay etc.
FR21
(Predictive Analysis Engine)
Should have
TS25 Email Notification Module
The platform should send email notifications to users for certain types of events, e.g. when new datasets are uploaded, new points have been added on a map etc.
FR11 (Data Notification),
FR14 (Map notification)
Nice to have
TS26 User Profile The platform should support user profiling with configurable profile fields and ability to connect social accounts.
FR02 (Hub Access),
FR03 (User Profile)
Should have
TS27 Role Management
The platform must provide different roles with configurable permissions to various functionalities and modules of the platform.
All functional requirements Must have
TS28 Multilingualism The interface should support multilingualism
FR29 (Multilanguage) Should have
TS29 Data Download option
The data Catalogue must provide a ‘download option’
FR07 (Data Download) Must have
TS30 Gamification features
The platform must provide gamified functionalities which award users with points, badges etc. for performing certain
FR18 (Social Game feature) Must have
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 16
No Name Description User Requirement Priority
actions inside the platform.
TS31 Shared Visualizations Catalogue
Visualisations created by users must have an option to be available in the public catalogue.
FR19 (Visualisations Catalogue)
Nice to have
TS32 Skill Matching Engine
Profiles should be linked with a defined set of categories. The Skill matching engine would then allow searching for people with certain skills.
FR20 (Skill matching engine)
Nice to have
TS33 Integration with payment module
The platform must offer some of the services through a payment option. This can be realized through an external major payment carrier.
FR25 (Payment module) Must have
TS34 The Hub should be a standalone application
A hub can be hosted as a standalone site. FR01(White-Label Front End)
Must have
TS35 The hub should be an integrable application
A hub can be integrated in an existing web platform.
FR31 (Pluggable Front End) Should have
TS36 Map mash-up application
The platform must allow the combination of different datasets on a single map interface.
FR22 (Map mash-up application)
Must have
TS37 Data uploading
The platform must support uploading and internal storing of datasets. Certain licenses might apply to this kind of data and the output of its analysis.
FR08 (Data Upload), FR12 (Data Licensing)
Must have
TS38 Role-based Components
The Hub must be structured in a way that helps different stakeholders identify the functionalities addressing their needs. Component access must be role-based and the same should apply to parts of the appearance of them.
FR04 (Landing Page), FR05 (Visible stakeholder paths)
Must have
Table 3 Technical Specifications
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 17
3 OTN Hub Architecture
3.1 Architectural Design Methodology We consider a design as successful when it covers all the following aspects:
• Usability
• Performance
• Security
• Maintainability
• Scalability
In the architectural design of the OTN platform, we are addressing all of the above in order to make a robust and fully flexible system, which will cover all the needs of OTN project. From a conceptual point of view, as most modern software architectures, it follows a layered architecture pattern. Thus, the system’s modules are divided into four distinct layers, each responsible for a specific set of functionalities. By convention, the layers interact with each other in a top-down manner, with each layer being able to access all layers below. A lower layer should never interact with layers above. This convention helps to avoid circular dependencies between layers.
The layers in a generic layered architectural design of any software platform are described as follows:
• Presentation Layer: Contains all the User Interfaces and Visualization Modules
• Services Layer: Exposes multiple APIs in the form of web services, defining a set of resources/methods as well as message structures
• Business Layer: Encapsulates all the business logic, as well as core domain entities of the system. It implements all system’s workflows and offers a simplified API (system façade) to the top layers for fulfilling the business workflows.
• Data Layer: Consists of all Data Access Objects as well as external service consumers. It is the broker to all the persistence storage and external data.
• Cross-Cutting Layer: Although it is not one of the basic four layers, it contains a set of features and modules which do not belong to a specific layer, since they are collaborating with all layers of the platform. These modules refer mainly to security and communication.
The following diagram1 depicts the layered software architecture:
1http://blogs.msdn.com/b/jmeier/archive/2008/11/24/application-architecture-diagrams-added-to-codeplex.aspx
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 18
Figure 1: Layered Architecture
The layered architectural design addresses all the aspects that we are targeting. Together with other software architectures and standards that are followed, it helps us to apply the best standards in all the design aspects we are focusing on. More specifically:
Usability: The fact that we are following a layered architecture isolates the presentation modules from the logic layer, giving the possibility to focus on good User Experience design (UX). Thus, a web designer or a usability expert can work separately on the User Interface unaffected by the backend system developers. These experts can focus distinctly on the User Interface to maximize the quality of user experience. Internally, inside the presentation layer, we are going to follow a Model View Controller design pattern for building a web application, starting from a plain, well-defined user interface that consumes the services provided by the backend. The use of cutting-edge technologies like HTML5, CSS3, jQuery and other javascript libraries will offer the best set of front-end features to provide a clean and fully functional interface. Finally, we are going to follow an agile methodology for developing the platform, based on rapid prototyping and frequent iterations. This enables more frequent evaluations close to the end-user of the platform and better result in terms of meeting the usability requirements.
Performance: For tackling performance issues, we are going to rely on two factors – caching and distribution. The system architecture logic is implemented in the backend and is exposed through an API of RESTful web services. These services offer parallelization in calls to the backend and can be deployed independently in a distributed way among several nodes of the cloud, following a Software Oriented Architecture. If needed, load balancing and caching will be applied between the presentation and the services layers. Additionally, the caching of data can take place even between the business layer and the data layer, when frequent fetching of the same data is required.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 19
Security: The security features will be applied system-wise, covering all layers of architecture. The Access Control and identity Management system will ensure that only users with appropriate permissions will be able to access the data relying in the platform’s data storage. Moreover, the external APIs (Service Interfaces) exposed by the platform can be secured by means of encryption over HTTP via SSL, which is the standard protocol for security over the Internet. Since we are going to use Liferay for OTN platform, any user credentials (for accessing the central platform) will be stored encrypted inside Liferay database, in a Database accessible only by Liferay server. Only the Services Layer and the UI is exposed publicly. Finally, we are not going to store sensitive data inside the platform (e.g. credit card details). All the payment workflow will be implemented inside the service provider’s website, therefore there is no risk for losing such kind of data.
Maintainability: For addressing maintainability, we are going to follow standards oriented and technology independent architecture. Focusing on well-defined generic standards (e.g REST) we don’t rely on specific knowledge for proprietary solutions and standards. We will use only open-source solutions written in JVM languages (Java, Groovy) that are cross-platform and fully flexible. For code collaboration tools, the use of an overall Continuous Integration solution with Hudson/Jenkins, maven, sonar and subversion for code versioning management could be established. For communication between the various developer teams, we are going to use a Redmine ticket tracking system. This way all tickets can be traceable and all code commits can be examined and explained, referring to specific tickets.
Scalability: The system is going to be able to scale up based on the volume of the requests. The architecture is free to scale up easily due to the service oriented distributed nature of the backend.
3.2 High level Overview In this section we provide a high level overview of the platform architecture. Following the user requirements analysis, an initial architecture is presented that was derived according to established design principles and in direct correspondence with the identified system technical specifications. The architecture specifies the main functional components of the envisioned system and the foreseen methods for their communication and collaboration. It must be noted that due to the complexity of the architecture and the interactions between the components, the presented architecture is subject to refinements and modifications, based on the progress of the technical work packages, as well as the validation and evaluation phases. Moreover, the consortium has created a github organisation, https://github.com/opentransportnet, where the most up-to-date developments on the OTN Hub will be available. A process of creating automation scripts is envisaged. These could be e.g. vagrant2 files which would allow everyone to test the set up on their own machine and experiment with the various components.
The proposed architecture is based on the reuse of existing software components and their adaptation to match the project-specific requirements. Therefore, a number of components already used in a working and tested services platform 3 produced by the Plan4Business4 project will form the basis of the platform’s architecture. These components will be adapted in order to fulfil the technical specifications defined in the previous section of the current document. This adaptation process is analytically described in D3.1.
The following figure depicts the overall architecture of the platform:
2https://www.vagrantup.com/ A tool for building complete development environments 3http://www.whatstheplan.eu/ This platform provides applications build on top of a database with a large amount of data related to urban planning (such as socioeconomical data, urban plans, risk zones etc.). It also provides freely accessible applications focused on visualization and interpretation of such data in the form of thematic maps and comprehensive reports that evaluate particular location from different perspectives. 4http://www.plan4business.eu/ A service platform for aggregation, processing and analysis of urban and regional planning data
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 20
Figure 2: OTN Platform High Level Architecture
As seen, the architecture follows the layered architecture principles. More specifically, the Frontend elements contain all the modules that will be available to end-users in order to use the offered functionalities. The Geospatial Middleware contains the business logic components of the platform and serves the requests coming from the Frontend. Finally, the Storage contains all the required databases.
The front end will provide a clearly structured Hub where different kinds of stakeholders will be able to view the offered functionalities and navigate to the tools that are targeted to them. The provided visualisation components will make use of various javascript libraries in order to provide advanced map representations and visualisations of data. Any necessary web forms to link or upload data to the platform as well as the Marketplace and Forum of OTN also reside in this layer. Moreover, administrative interfaces needed for the configuration of the backend components will be accessible online and therefore reside in this layer.
The business layer modules of the Hub will be responsible for the harmonization and integration of data and services in order to support the frontend elements. Components like Integration engine, Maplog and Senslog belong to this layer and are responsible to implement complex analysis and processing of geospatial data in order to support the visualisations required by users. Open APIs, in the form ofa RESTful API will be exposed as a set of resources, which will consume and proxy data and services provided by the platform or third party data and service providers. The DataTank is also part of this layer and will offer harvesting capabilities as well as a RESTful API.
Finally, the Storage layer will include a PostgreSQLdatabase server, which will be responsible for Liferay data persistence and local data maintenance. Primary and Secondary data storages will be used for storing the analysed geospatial data which will be available in the visualization components of the Frontend. A MySQL instance will be also installed in order to support DataTank’s full capabilities and functionalities.
Most of the existing components are based on Liferay Platform (analytically presented in paragraph 3.3.3) and offered as Liferay portlets. Therefore, the core logic of the platform will be implemented by means of Liferay hooks and java backend code of the portlets offering the required functionality. Components that will have to be adapted as well as all the new ones will be provided as Liferay portlets in order to ensure smooth integration and interoperability with the rest of the platform. Furthermore, components or modules that will reside outside of the platform’s main server will use a service oriented approach to communicate with the central Liferay portal. This approach allows loose coupling of services in terms of used technology and
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 21
programming language. By using common standards, web services have little or no dependency on each other, a fact that offers flexibility and scalability to the system. The following paragraph describes RESTful web services, which will be the followed approach in OTN.
3.2.1 RESTful Web Services
Communication between the components of the OTN Hub will be based on the REST web services approach. REST stands for Representational State Transfer, which means that several unique URLs are used to represent various resources. Each resource can be accessed by the use of HTTP GET or POST methods. REST is an architecture style, and while it employs standards, it does not constitute a standard itself. Main characteristics of REST web services are described below:
• Client-Server Architecture o The resources are located in a server and a client can request them through a REST web
service. • Stateless
o No session state is maintained at the server. Each request from client to server contains all of the information necessary to understand the request.
• Cache o The requested resources can be cached and therefore the client gets faster results.
• Uniform interface o REST operations use the HTTP operations GET, PUT, POST and DELETE. Thus if a client can
deal with hypermedia, it can start dealing with REST. • Named resources
o Every resource is accessed using a clear name, its URL.
3.3 Platform Technology Stack 3.3.1 Java EE 7
Java Enterprise Edition 7 is one of the most popular open source programming languages. It provides API and runtime environment for developing and running enterprise software. The main benefit of Java as compared with other languages is that it is cross-platform, therefore all applications developed in Java are fully portable and platform agnostic. Moreover, it is fast, secure, and stable and it provides a powerful memory management and garbage collection mechanism, eliminating memory leak issues.
3.3.2 Linux Ubuntu Operating System Linux Ubuntu5operating system is a Debian-based Linux operating system developed by Canonical Ltd a company based on the Isle of Man and owned by South African entrepreneur Mark Shuttleworth. In general, as any other Linux distribution, the Linux Foundation of which platinum supporters are IBM and Oracle also supports it.
Ubuntu is composed of many software packages, the majority of which are free software. Free software gives users the freedom to study, adapt/modify, and distribute it. Additional software that is not installed in the default distribution can be downloaded and installed using the Ubuntu Software Center or other APT-based package management tools6.
5http://www.ubuntu.com/ 6http://en.wikipedia.org/wiki/Linux_ubuntu
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 22
For increased security, the sudo tool is used to assign temporary privileges for performing administrative tasks, allowing the root account to remain locked, and preventing inexperienced users from inadvertently making catastrophic system changes or opening security holes.
Ubuntu also offers Ubuntu Cloud Images which are pre-installed disk images that have been customized by Ubuntu engineering to run on cloud-platforms such as Amazon EC2, Openstack, Windows, LXC and HOL.
3.3.3 Liferay Portal
In OTN, we need a platform to support user management (definition and administration of roles and privileges). We also need a solution to bring together various software modules and information in a uniform way, under a central user management mechanism. For this purpose, there is a need to use a portal (portlet container). Liferay Portal7is an open sourceenterprise portalwritten inJava that allows developers to set up
features common to websites and easily create corporate intranets and extranets. Since we are going to be based on Java programming language for the development, Liferay is the ideal solution as it is free, open source, stable, popular and rich in features. The community of Liferay provides well-documented guidelines as well as support to developers through a forum.
The current version of Liferay is 6.2. The community edition is freely available, distributed under theGNU Lesser General Public License and is maintained by the “community”, that is, a group of registered to Liferay users, while the professional is provided for a fee under a commercial license and maintained by the Liferay team, who also provide support.
Liferay portal is also a content and document management system that allows users to store, retrieve and edit documents and web content for their portals.
Although Liferay offers a sophisticated programming interface for developers, no programming skills are required for basic website installation and administration.
Some of Liferay’s key features are presented below. These features are not necessarily needed for OTN at the current stage, however, they could be useful in case we need to extend the platform.
Multi-language Support
International or multi-lingual organizations get out of the box support for 30+ languages. Users can toggle between different language settings with just one click.
Simplified UI Development
Liferay Portal simplifies the development of internal, external, and channel websites--notably those that allow users to login for personalized services or views and those that require a workflow approval process to update content and integrate or aggregate multiple existing services. Liferay Portal provides a single presentation layer for integrating all enterprise systems into a single easy to use interface for end users.
Wikis
Build up and document important or interesting information as a community with Liferay's Wiki, which competes with standalone products in the robustness of feature set.
7http://www.liferay.com/
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 23
Each community gets its own Wiki with its own set of authorizations. Anyone with editing rights can quickly contribute information to these online topical encyclopaedias.
Blogs
Liferay's Blogs provide the best features of modern blogging tools coupled with the benefits of the community-centric nature of Liferay Portal. They allow users to convey information and facilitate conversations around blogs directly in the context of a website, intranet, extranet, or social network.
Features include a user-friendly rich text editor, social bookmarking links, email notifications of blog replies, and an entry rating system. All blogs can be subscribed to via RSS and users can schedule entries to be published at specific times and dates.
Message Boards
Message Boards are a perfect solution for facilitating conversations around shared ideas within a department or project team, or for capturing and sharing the knowledge of the workgroup.
Liferay offers views of message board activity statistics and recent posts and users can both subscribe to threads via RSS and reply to threads by email. Like all other portlets, Message Boards are secured by Liferay's granular system of authorizations which grants varying levels of control to different users.
Polls
Multiple choice polls can be created with this tool that also keeps track of votes. Many separate polls can be managed and a separate portlet can be configured to display a specific poll's results.
RSS
Subscribe to frequently read RSS feeds from message boards and blogs within the portal.
User-Driven Workflow & Approval
Not only is there embedded workflow for content, Liferay Portal allows users to create their own workflow and define the number of approval paths based on their own unique business requirements and operational needs.
User Groups, Organizations and Sites
Liferay users can be intuitively grouped into a hierarchy of "organizations" or cross-organizational "user groups," providing flexibility and ease of administration.
For example, members of different geographies such as Americas and EMEA can be grouped into organizations, whereas project based or departmental teams such as a "Website redesign" that cross disciplines can be created as user groups. Liferay provides support for "sites" where both organizations and user groups can be added to a separate web property with its own set of pages, content management system, shared calendar, and authorizations. A user can belong to multiple sites and easily navigate between them.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 24
Web Content Structures and Templates
Liferay Portal allows users to easily create reusable templates for web pages and page sections. This enables users to quickly build pages and allows websites to maintain a common look and feel across the entire site by allowing new pages to be created with an approved set of templates. Users can quickly create templates within Liferay's web UI or by using external web development tools.
SOA Framework
Liferay Portal is developed using an open SOA strategy that makes it the choice of enterprises worldwide for enterprise application integration.
OpenSocial
Support for OpenSocial 1.1 creates new avenues for developers to add social capabilities and dimensions in their websites. With OpenSocial, users can manage and deploy web-based social applications built from gadgets directly to pages and sites.
Technologies
Liferay Portal is Java based and runs on anycomputing platformcapable of running theJava Runtime Environmentand an application server. A number of application servers (Tomcat, Glassfish etc.), databases (MySQL, PostgresSQL, SQLServer, etc.), technologies (AJAX, REST etc.) and open standards (JSR-286,JSR-170, JSR-314 etc.) is supported, fact that gives the opportunity to developers to combine them in a way that best suits to the needs of each project.
A full list of technologies used and supported by the current version of Liferay portal can be seen inthe Technical Specifications.
Development
With Liferay portal it is possible to develop themes that define the look n’ feel of the web application’s pages, hooks that customize the portal’s abilities and layout templates that define the way portlets are spread inside a portal page.
The main parts of a portal are functional units that are called portlets. Liferay comes with a number of built – in portlets that implement many of its features mentioned previously. It is also possible to extend Liferay’s functionalities by creating custom portlets. Technologies that can be used for this purpose (and not limited in these), are Java Server Faces (JSF), Vaadin, or Java Server Pages (JSP) and JQuery.
The development of all these units is possible through a series of plugins provided freely by Liferay for the Eclipse IDE or Liferay Developer Studio for the Enterprise Edition.
Security
Liferay Portal uses industry standard, government-grade encryption technologies, including advanced algorithms such as DES, MD5, and RSA, and was benchmarked as among the most secure portal platforms using
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 25
LogicLibrary's Logicscan suite. It offers a customizable single sign-on (SSO) that integrates with Yale CAS, JAAS, LDAP, Microsoft Exchange, and more.
What's more, Liferay Portal ships with robust user management and security features including password policies, user reminder settings, and complete log-in security procedures. Liferay also abides by OWASP guidelines to reduce the risk of security vulnerabilities. Other security features include:
• Pluggable Authentication
• Email Verification
• Session Management
Clustering
Liferay Portal is designed to serve everything from the smallest to the largest web sites. Out of the box, it's configured optimally for a single server environment. If one server isn't sufficient to serve the high traffic needs of a project, Liferay scales to the size necessary.
Liferay works well in clusters of multiple machines (horizontal cluster) or in clusters of multiple VMs on a single machine (vertical cluster), or any mixture of the two. Once a Liferay portal is installed in more than one application server node, there are several optimizations that can be performed in different aspects of the portal like database, DMS repositories, caching and searching.
• Database: Liferay supports database sharding through different portal instances, using the round robin shard selector. This is a class which serves as the default algorithm for sharding in Liferay. Using this algorithm, Liferay selects from several different portal instances and evenly distributes the data across them.
• DMS: Liferay’s Documents and Media Library is capable of mounting several repositories at a time and presenting a unified interface to the user. By default, users can make use of the Liferay repository, which is already mounted. This repository is built into Liferay Portal and can use as its back-end one of several different store implementations. In addition to this, many different kinds of third party repositories can be mounted and all nodes of the cluster will point to this repository. If one of the Liferay’s repositories is used, more configurations are available to achieve higher performance in a clustered environment.
• Search: Liferay uses as its default search engine Lucene. One way of achieving search is to further configure Lucene so indexes replicate across the individual file systems of the nodes in the cluster. Liferay also supports the usage of Solr which is an enterprise search engine and has more configuration options.
• Caching: Liferay uses Ehcache, which has robust distributed caching support. This means that the cache can be distributed across multiple Liferay nodes running concurrently. Enabling this cache can increase performance significally.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 26
3.4 System Workflows The following paragraphs describe how the system will behave in a number of workflows that derive from the user defined functional requirements.
3.4.1 Link an existing dataset to the OTN Hub
1. User navigates to the OTN portal and is presented with the login page
2. User enter his username and password and submits them
3. The portal UI communicates with the user manager passing the credentials
4. User manager communicates with the portal’s database to check the credentials
5. The database manager performs a query
6. And returns the user data in case of successful check
7. User manager notifies the portal UI
8. The portal present a confirmation message to the user for his successful login
9. The user selects the dataset registration page
10. The portal renders the page
11. The user provides the url of the dataset
12. The page checks the validity of the url
13. If the url is not a valid one it notifies the user to reenter
14. If the url is valid the portal UI enables the dataset’s metadata form
15. The user completes the metadata fields
16. The user submits the form
17. The portal UI sends the data to the datasets manager
18. The datasets manager stores the data to the HUB’s database
19. The database confirms data addition
20. The datasets manager notifies the portal UI about the completion of the task
21. The portal UI presents a confirmation message to the user
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 27
Figure 3: Link an existing dataset to the OTN Hub sequence diagram
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 28
3.4.2 Create a mashup
1. User navigates to the OTN portal and is presented with the login page
2. User enter his username and password and submits them
3. The portal UI communicates with the user manager passing the credentials
4. User manager communicates with the portal’s database to check the credentials
5. The database manager performs a query
6. And returns the user data in case of successful check
7. User manager notifies the portal UI
8. The portal present a confirmation message to the user for his successful login
9. The user selects the visualizations page
10. The visualization page requires initialization, so it communicates with the visualization tools
11. The visualization tools start their initialization
12. The visualization tools retrieve the available datasets from the HUB’s database
13. The database returns all available datasets
14. The portal UI gets all the visualization page data
15. And presents it to the user
16. The user selects a dataset and any other filtering option
17. The dataset’s information is presented to thepage’s map
18. User selects to save the map
19. The user is presented the download window
20. The user selects a location to his disk that he wishes to save the map and submits
21. The map is saved to the selected disk location in a predefined format
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 29
Figure 4: Create a mashup sequence diagram
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 30
3.4.3 Upload geospatial data to Hub’s internal storage
1. The logged in end user selects hub
2. The portal presents to the user the appropriate page of the HUB’s site
3. User selects to upload his data, so he navigates to the upload page
4. The portal presents to the user the upload page
5. The user fills in the fields related to his data and submits them
6. The portal UI checks the validity of provided data
7. If all data are valid the portal UI sends the data to the data manager
8. The data manager stores the data to the HUB’s database
9. The database notifies the data manager about the successful outcome.
10. The portal UI is notified also from the data manager
11. The portal UI sends a notification regarding the new data to the administrator
12. The administrator logs in and navigates to the notifications page
13. In order to present the new data to the administrator the portal UI communicates with the data manager
14. The data manager requests the new data from the HUB’s database
15. The database returns the requested data
16. The data are sent to the portal UI
17. The page is ready for presentation to the administrator
18. The administrator chooses to verify the new data
19. The portal UI communicates with the data manager in order to make the new data visible to all users
20. The data are flagged as visible in the database
21. The database informs the data manager for the outcome of the action
22. The portal UI also gets notified
23. The portal UI presents a confirmation to the administrator
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 31
Figure 5: Upload geospatial data to the Hub’s storage sequence diagram
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 32
3.4.4 Use a mobile device to upload VGI to the Hub
1. The user has installed the OTN application to his mobile and selects to log in.
2. The mobile application uses the portal services for user authentication
3. The portal upon successful authentication responds with the user details
4. The mobile application present a login confirmation to the user
5. The user selects to navigate to the journeys page
6. The mobile app presents the page to the user
7. The user selects to upload his data and fills in the appropriate fields
8. The mobile app sends the data to the data manager
9. The data manager stores the data to the HUB’s database
10. The data are stored
11. The data manager notifies the mobile app about the successful outcome of the action
12. The mobile app informs the user about the successful storage of his data
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 33
Figure 6: Use a mobile device to upload a VGI sequence diagram
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 34
3.4.5 Receive notifications about a traffic accident
1. The logged in user wants to register to notifications and selects the notification’s page.
2. The notifications page communicates with the notifications manager in order to initialize
3. The notification manager retrieve the user’s settings regarding notifications from the HUB’s database
4. The database returns the notification settings
5. The notifications manager return the settings to the portal ui
6. The portal presents the page to the user
7. The user selects the route(s) for which he wants to receive notifications
8. The notifications page sends the user route(s) to the notifications manager
9. The notifications manager stores the route(s) to the database
10. Notifications manager is notified about the successful addition
11. The notifications page is notified about the successful addition
12. The notifications page presents a confirmation message to the user
13. Another logged in userwants to report an accident and selects the accidents page
14. The page requires initialization so it communicates with the accidents manager
15. The accidents manager retrieves all the recent accidents
16. The accidents data are sent to the accidents manager
17. And from there to the accidents page
18. The portal presents the accidents page to the user
19. The user checks the accidents list and confirms that the accident he wants to report is not in it, so he adds the data for the new accident and submits
20. The accidents page sends the new data to the accidents manager
21. The accidents manager saves the data to the database
22. The storage is confirmed by the database
23. The accidents page presents a confirmation message to the user
24. The accidents manager also informs the notifications manager about the new accident
25. Notifications manager retrieves the twitter accounts of the users registered for notifications on the new accidents route
26. The database responds with the twitter accounts
27. The notifications manager posts the alert about the new accident to the retrieved twitter accounts
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 35
Figure 7: Receive notifications about a traffic accident sequence diagram
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 36
4 OTN Hub Components This section analyses the components comprising the OTN Architecture. The section is structured based on the major components as they are described in the project’s DoW. Figure 8below depicts the topology of these components.
Figure 8 OTN Hub Components
All the subcomponents presented in the high-level architecture (Figure 2) are distributed under the major components. As previously described, a number of existing modules will be re-used in the context of OTN whereas new components will have to be developed in order to cover all the functional requirements. This is why a technical specification questionnaire (ANNEX) was circulated among the technical partners of the project in order to gather information about existing components. For these components, a brief description of their functionality and supported technologies will be provided, while the dependencies with other tools and interoperability aspects will be also specified.
4.1 Front End User Interface The Front End User Interface will provide a unified visual environment to the users of the OTN Hubs. It will include all the necessary pages and sections of the Hub that will host the components that are described in the following paragraphs. These pages include, amongst others, the user profile page, online forms and the general navigation pages. The graphical interface will be implemented inside Liferay and will consist of Liferay portlets, Liferay themes and the Liferay admin UI. A simple, eye-pleasing and engaging theme will be selected in order to help users easily find their way inside the Hub.
Moreover, various administrative sections will have an interface available through the platform. These are described in the next paragraphs and refer to backend components such as the Configuration Management, the Plan Integrator and the Analysis UI.
Finally, nativemobile apps will be developed for OTN and serve as an alternative entrance point for users. The user interface of these applications will be adapted to match the needs of mobile devices without losing the overall look and feel of the desktop interfaces.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 37
4.1.1 Configuration Management
The Configuration Management refers to the administrative interface of Liferay and the interface of the Layman component as described in paragraph 4.5.1. It’s actually not a specific standalone component but it is presented as such to easily refer to all the administrative screens and functionalities of Liferay and Layman. Setting the correct parameter values and adjusting the settings of the various auxiliary modules of the platform can affect the results in a great extent, therefore only privileged users will have the permissions to edit these configurations.
4.1.2 Plan Integrator
Name Plan Integrator and Integration Engine (technically one integrated component)
Type Application (Plan Integrator)
Service (Integration Engine API)
Status Operational onhttp://www.whatstheplan.eu/integrator/
Dependencies
Java 7 (bundled with the application)
Liferay Portal (for authentication, must run on the same host or at least the portal host has to be the reverse proxy through which the application is accessed)
LayMan (for publishing data sets as WMS)
MICKA metadata catalogue (for publishing metadata)
PostGIS database (primary data pool)
Disk storage for storing uploaded data
Parameters Runs as an OSGi application and runs its own bundled instance of Jetty. Combination with other web applications thus has to be achieved through a web server serving as reverse proxy. Jetty port is configurable via system property.
Running environment
Currently runs onwww.whatstheplan.eu(Linux 64 bit). Versions are available for Windows and Linux, both 32 and 64 bit.
Additional information
Installation guide for www.whatstheplan.eu: https://intranet.plan4business.eu/r/projects/integrate/wiki/Integration_Engine_installation
4.2 Back-End Data Aggregator and Open APIs One of the goals of OTN is to identify a wide set of GI/open datasets and find a way to make them available through the OTN Hub via accessible and easy-to-use formats. These tasks will be performed by the components described below.
4.2.1 Data Harverster
This will be aset of tools for gathering, opening and storing non-harmonized but publicly accessible data in the OTN Hub (Fig 2.). The data harvester tools will convert gathered data into RDF format. The DataTank will be
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 38
used as a primary tool for data harvesting.The DataTank's input interface has harvest capabilities: given a frequency, it will fetch data from a certainsource, extract, map it to RDF, load it in a triple store, and configure the publishing interface. When thereare extractors for geospatial data, mapping files will be able to map the data to ontologies and link differentresources.
Name The DataTank
Type Data publishing tool
Status Available athttp://thedatatank.com
Dependencies PHP 5.4+, Apache and Mysql
Parameters -
Running environment
http://demo.thedatatank.com - Runs on any PHP stack.
Additional information
Custom installation intstructions athttp://docs.thedatatank.com/4.3/installation
4.2.2 Data Harmonisation
Data harmonisation is a process of creating uniform data set(s) from heterogeneous data sources ~ by changing the data schema. Both sources of datasets (stored in OTN hub and dynamically linked) can be used as inputs. Harmonized outputs are going to be stored in the primary SQL data storage (data harmonized according to standards) or in the secondary data storage (data harmonized according to resultant application). HUMBOLDT Alignment Editor (HALE)developed in the Humboldt project will be the core of data harmonization tools:
Name HUMBOLDT Alignment Editor (HALE)
Type Application (HUMBOLDT Alignment Editor)
Service (Conceptual Schema Transformer)
Status Downloadable onhttp://www.dhpanel.eu/humboldt-framework/hale.html or http://www.dhpanel.eu/humboldt-framework/cst.html
Dependencies Java 7 (for the application)
A web server ~ e.g. Apache (for the service)
Parameters Runs as a Java application or as a Web Processing Service.
Running environment
Versions are available for Windows and Linux (GTK), both 32 and 64 bit and Mac OS 10.7+ (64 bit).
Additional information
Installation guide:
http://www.esdi-community.eu/projects/hale/files
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 39
User documentation for the CST Web Processing Service:
http://www.esdi-community.eu/projects/hale/wiki/User_documentation_for_the_CST_Web_Processing_Service
4.2.3 OTN Data Catalogue
Name PostgreSQL / PostGIS
Type PostgreSQL: Database Management System
PostGIS: spatial database extender for PostgreSQL
Status Downloadable on http://www.postgresql.org/download/ and http://postgis.net/install, operational in the background of many web sites, e.g.: http://www.whatstheplan.eu
Dependencies A web server ~ e.g. Apache (for the web access).
Each version of PostGIS depends on particular version(s) of PostgreSQL, see dependency table: http://trac.osgeo.org/postgis/wiki/UsersWikiPostgreSQLPostGIS
Parameters Runs as an application
Running environment
Currently runs onwww.whatstheplan.eu(Linux 64 bit). Versions are available for Windows and Linux, both 32 and 64 bit.
Additional information
Installation guide: http://trac.osgeo.org/postgis/wiki/UsersWikiInstall
4.2.4 Open APIs
It is crucial to further exploitation of services provided by OTN platform and the data stored in the OTN Hub, to have a machine readable interface for above described tools:
• The DataTank API is a REST webservice. It is documented in a machine readable fashion using DCAT (at http://{yourhost}/api/dcat) and a discovery document to understand the HTTP calls which can be made (at http://{yourhost}/discovery. It is documented in a human readable format at http://docs.thedatatank.com.
• The HALE tool can be accessed as a Web Processing Service (WPS), defined by OpenGeospatial Consortium.
• The PostgreSQL/PostGIS can be accessed using OBDC or JDBC interface and SQL
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 40
4.3 Geospatial Metadata Catalogue This component includes the deployment of management tools for spatial and temporal data storing, accessing and sharing. The aim is to design and set up a software framework for the effective storage and analysis of large-scale spatial data based on distributed and location aware systems. The emphasis will be given on spatial and spatial-temporal data. The Geospatial Metadata Catalogue will minimally include INSPIRE compliant metadata in order to be interoperable. Micka Catalogue is an existing geospatial catalogue that will be adapted to the needs of the project. The main characteristics of Micka are given in the following table.
4.3.1 MICKA
Name MICKA
Short Description of Functionality
Spatial metadata catalogue. Supports OGS CSW 2.0.2, OGC CSW ISO AP 1.0, INSPIRE metadata profile, INSPIRE discovery service requirements. Extended functionality available (support of GeoRSS, Atom, JSON, DCAT EU AP - RDF, OpenSearch, HTML, PDF output). Web interface for editing, import tools, CSW transaction and harvesting support. Connection to SKOS compatible thesauri (GEMET etc.), multilingual UI, multilingual records.
Past project(s) that the background tool was used
OneGeology, Habitats, Plan4all, Plan4business, EnviroGrids, National INSPIRE portal
Component architecture
N/A
Technologies used in the development
PHP, Javascript
Implemented from scratch
Yes
Third party platform used (version)
N/A
User interaction requirement
• Perform queries to retrieve set of records, show record list, detailed metadata, download xml, …
• On-line metadata editing
• Importing metadata from external sources, local files etc.
• Validation of metadata
• Setting the environment (profiles, harvesting etc.)
Configuration needed for initiation and
• Database setup and connection • Shared libraries address • OSM (or another map service) address
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 41
Name MICKA
runtime • Used thesauri address • Visual (if not default)
o HTML output templates o CSS, icons
Stress Test under heavy load results
Series of tests were performed on Czech national portal to fulfill the INSPIRE network services requirements (30 requests per second for 20 concurrent users). GetCapabilities / GetRecords/ GetRecordById were randomly mixed. Result: cca 50 request (passed).
Provided as a service
Optional
Provided as binary
No
Provided as source code
Yes
Restrictions in use
Commercial license
Available under an open source license
Not decided
Programming languages used
PHP >5.2, Javascript
Design tools dependence
No
Hardware requirements
None
Software requirements
None
Third party Dependencies
PHP >= 5.2, Extensions: XSLT, mbstring
Third party libraries
N/A
Automatic builds support
No
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 42
Name MICKA
( Ant/Maven )
Database Requirements
SQL
Requirement for version control system
No
Offers an API
a. OGC CSW 2.0.2 support according to standard i. Querying (GetRecords, GetRecordById operations) ii. Editing (Transaction, Harvest operations)
b. Extended functionality: i. Additional queryables (listed in Capabilities document)
ii. Additional format (JSON, RDF, Atom, GeoRSS, HTML)
Communication schema supported
API
Asynchronous messaging support
Harvesting – see the standard
Dependencies / Interactions with other OTN components
• HSLayers – links to metadata, composition metadata cataloguing • Layer Manager – layer metadata cataloguing
Input / Output requirements
• ISO 19115/19119/19139 XML input/output • default csw:Record output • Other OGC capabilities documets (SOS, WMS, WFS) form metadata import
Security principles to be taken in consideration
N/A
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 43
4.4 Visualisation Components The visualisation components will be based on the existing tools that are described in the following paragraphs. They will receive various types of data as input and provide advanced map visualisations and other kinds of visualisation methods like time series, graphs, cartographic animations etc.
4.4.1 Thematic Map Viewer
Name Thematic map viewer
Type Application (web application)
Status Operational onhttp://www.whatstheplan.eu/viewer
Dependencies
p4b
• Primary data pool • Micka • Statusmanager • Layman • Map Creator
3rd party (server)
• Mapserver • R
3rd party (client)
• HSLayers • Ext • OpenLayers • JQuery • Proj4js
3rd party (data)
• External WMS services (i.e., www.geoportal.gov.cz/web/guest/wms/) • External tile services (Open Street Maps (basemap), Google (aerial imagery) )
Parameters
Server side is mostly dependent on Micka (stores metadata about compositions), Status manager (loads compositions into the client) , Layman (Geoserver), Mapserver and R (visualization of data: creating WMS services and diagrams), client part on various javascript libraries (HSLayers, OpenLayers, Ext, JQuery, Proj4js)
Running environment
Complex, composed of many parts web application aimed at cartographic data visualization in different techniques from multiple sources
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 44
4.4.2 Location Evaluator
Name Location evaluator
Type Application (web application)
Service - rest api
Status Operational onhttp://www.whatstheplan.eu/evaluator
Dependencies
p4b
• Primary data pool • Thematic Map Viewer • Analysis engine
3rd party (server)
• Java 7 • Servlet container (tomcat) • Jasper library • For full list of libraries
3rd party (client)
• bootstrap • OpenLayers • JQuery
Parameters Server side in Java, client in Javascript, Bundled Jetty, tested on Tomcat,
Running environment
Maven based java web app.
4.4.3 Map Creator
Name MapCreator
Type Web browser application
Status Operational - http://www.whatstheplan.eu/creator
Dependencies • HSLayers • Ext
Running environment
Web browser
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 45
4.4.4 Embed Map
Name Embed map
Type Certain selected compositions from Thematic Map Viewer that user can embed into its web-page. Basically just a link that embeds map compositions/kml/gpx layers in which user is interested
Status Operational. Example: http://dev.ccss.cz/~ovnis/embemap/?&composition%5B%5D=http%3A%2F%2Fwww.whatstheplan.eu%2Fwwwlibs%2Fstatusmanager2%2Findex.php%3Frequest=load%26id=7ea2beb4-5ecb-47d3-ac0b-ecce712439b3&composition%5B%5D=http%3A%2F%2Fwww.whatstheplan.eu%2Fwwwlibs%2Fstatusmanh This is just a link that allows user to embed certain compositions/kml/gpx layers into user’s web-page.
Dependencies p4b
• Thematic Map Viewer
3rd party (client)
• HSLayers • OpenLayers • Bootstrap
Parameters Link that uses functionality of HSLayers and OpenLayers to add compositions/kml/gpx layers to usual Open Layers map window that can be embedded. As a parameters for the link user can enter array of compositions that he is interested in (one way to get the links may be to copy them from MICKA), further kml and gpx layers can be added, zoom level, coordinates of the center
Running environment
Need to have data to add (links to compositions, kml, gpx layers) and HSLayers and OpenLayers js libraries
4.4.5 Mobile Applications
The mobile applications that will be created in the context of OTN will offer a mobile interface to the visualisation components described above. Visualisation of geospatial data on a mobile device requires special ways of retrieving, processing and rendering. This is why a new library will be developed in order to support Android mobile devices.
Name GeoData mobile SDK
Short Description of Functionality
Android native library and reference application for retrieving, processing and rendering geospatial metadata and data on mobile devices.
Past project(s) that the background tool was used
No
Component architecture N/A
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 46
Name GeoData mobile SDK
Technologies used in the development
• Java
• Objective C
Implemented from scratch
Yes
Third party platform used (version)
None
User interaction requirement
Component will provide search and filter forms for retrieving geospatial metadata, and will provide map view of different geospatial data layers, allowing user to scroll and zoom map, as well as to select objects on map for further information retrieval and decision.
Configuration needed for initiation and runtime
The component supposes to be for mobile devices thus it is necessary to have server side I/O interfaces and runtime depends on mobile device.
Stress Test under heavy load results
No
Provided as a service No
Provided as binary Yes
Provided as source code Optional
Restrictions in use N/A
Available under an open source license
Depends on project results – functionality
Programming languages used
Java, Objective C
Design tools dependence • Android: Android Studio 0.5.2
Hardware requirements • Android: CPU: 1.3 GHz ARMv7, RAM: 1 GB
Software requirements • Android: 4.1 Jelly Bean
Third party Dependencies
N/A
Third party libraries N/A
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 47
Name GeoData mobile SDK
Automatic builds support ( Ant/Maven )
N/A
Database Requirements N/A
Requirement for version control system
N/A
Offers an API No
Communication schema supported
Consume Server REST API
Asynchronous messaging support
Yes, async processing of data is planned for the component
Dependencies / Interactions with other OTN components
None
Input / Output requirements
Security principles to be taken in consideration
Mobile device specific data storage security will be used. Component will support SSL encryption for network communication
4.5 Geospatial Middleware The Geospatial middleware contains the components handling users’ requests for data services. These components will support spatial data and metadata management as well as collection, harmonization, visualization and processing of data to extract useful information for decision making. The majority of these components will be adapted versions of existing solutions which are described in the following paragraphs.
4.5.1 GeoServer
Name Layer Manager (LayMan)
Short Description of Functionality
Server and Client that enables to publish geospatial data. Server offers REST API, Client offers web GUI. The value added to the classical publishing process is:
• Easy and “one-click” publication of GeoData, web GUI • Access Control, configuration of layer security and Access Granting • Integration with Liferay portal, HS-Layers Map Viewer, Open Geo
Styler
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 48
Name Layer Manager (LayMan)
Past project(s) that the background tool was used
Plan4business project (http://www.plan4business.eu/), PPRD
Project (http://www.pprd-rdc.net/)
Component architecture
Technologies used in the development
• Client technologies: Java Script, Ext4.
• Server technologies: Python, GDAL, OGR, PostGIS, GeoServer, Liferay.
Implemented from scratch
Yes
Third party platform used (version)
None
User interaction requirement
Server does not. Client web GUI is here to make the geodata publication process easier. User is expected to publish geodata.
Configuration needed for initiation and runtime
Initiation: Overall configuration within the Geoportal, e.g. configure the LayMan itself in the config file, configure appropriate groups in Liferay, establish connection between Liferay and GeoServer (to authenticate users when viewing published layers) etc.
Runtime: Nothing special.
Stress Test under heavy load results
No
Provided as a service No
Provided as binary No
Provided as source code
Yes
Restrictions in use GPL
Available under an open source license
GPL
Programming languages used
JavaScript, Python
Design tools dependence
No
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 49
Name Layer Manager (LayMan)
Hardware requirements
The real requirements fully depend on the load of the server, number of layers, number of users, amount of data etc. GeoServer, PostGIS, Liferay, Python are needed. Client depends on the browser and its resources.
Software requirements Preferably some Unix system, Tomcat, Apache
Third party Dependencies
No
Third party libraries Ext 4, GDAL, OGR, GeoServer, PostGIS, Liferay
Automatic builds support ( Ant/Maven )
No
Database Requirements PostGIS
Requirement for version control system
Preferably Git
Offers an API If the need arise
Communication schema supported
Server REST API will be described during the project
Asynchronous messaging support
REST API is supported
Dependencies / Interactions with other OTN components
None
Input / Output requirements
Security principles to be taken in consideration
Standard network security measures should be applied
Name LayMan
Type REST service (server side)
Web application (client side)
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 50
Name LayMan
Status Operational - http://www.whatstheplan.eu/spatial-data-manager
Dependencies p4b
• LayMan Client depends on LayMan Server • Primary Data storage • Micka • Map Viewer • Styler • Portal
3rd party (server)
• PostGIS • GeoServer • Liferay
3rd party (client)
• Ext
Parameters LayMan server can work standalone and is used by HALE. LayMan client is dependent on LayMan server and provides web GUI. LayMan (Layer Manager) imports geodata into the database and publishes it as OWS in the GeoServer. It also secures the layers and is connected to metadata Micka catalogue. Published layers can be styled in Styler or shown on the Map.
Running environment
LayMan is a middleware between Liferay, GeoServer, PostGIS, Micka and the file system. Server side is in Python, client side is in JavaScript with Ext library.
4.5.2 Analysis Engine
Name Analysis engine
Type Database
REST service
Status Operational
Dependencies 3rd party:
• Jaspsersoft library
Running environment
Maven based application
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 51
4.5.3 Integration Engine
The Integration Engine is technically provided as one integrated component as described in paragraph 4.1.2 of this document.
4.5.4 Traffic Models
Name Volume of traffic on the road network, traffic modeling
Short Description of Functionality
Modeling SW OmniTRANS can model different matrix sources and targets, socio-economic data and a variety of state of the transport network. It offers a number of tools for creating, editing, modeling, analysis, finding and removing errors in transport networks of different options. Can distinguish the purpose of travel, roads and other parameters such as land use and transport system as a major dimension of the project. These characteristics may be related, for example, market segmentation, dividing the types of vehicles, demographic data, can describe the behavior of people, etc. The output data can be exported to other applications (GIS) and various database storage.
Past project(s) that the background tool was used
N/A
Component architecture
N/A
Technologies used in the development N/A
Implemented from scratch
Yes
Third party platform used (version)
Transport Planning Software OmniTRANS ver 6
User interaction requirement
No. It is assumed that the user will operate, respectively use data from a component of other (map) applications.
Configuration needed for initiation and runtime
N/A
Stress Test under heavy load results
No stress tests have been performed "under heavy load" for components that have already been developed. This is an asynchronous computation, load test does not make sense.
Provided as a service Yes, shp file with data on traffic volumes
Provided as binary No
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 52
Name Volume of traffic on the road network, traffic modeling
Provided as source code
No
Restrictions in use N/A
Available under an open source license
N/A
Programming languages used Ruby 1.6
Design tools dependence
No
Hardware requirements
• Processor: Pentium IV with SSE2 support @ 1.3 GHz
• RAM: 1024 MB minimum.
• Disk: 250 MB for the software. Several GB may be required for your modelling projects.
• Screen: Minimal resolution 1280x1024
Software requirements • Operating system: Windows XP (SP3), Vista (SP2) and 7. 32 bit version
Third party Dependencies
No
Third party libraries No
Automatic builds support ( Ant/Maven ) No
Database Requirements SQL-like
Requirement for version control system
No
Offers an API No
Communication schema supported
• input data: roads, traffic generators (places), traffic counts • output data: roads with calculated traffic volumes in attribute
Asynchronous messaging support
No
Dependencies / Interactions with other
N/A
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 53
Name Volume of traffic on the road network, traffic modeling
OTN components
Input / Output requirements
• input data: roads, traffic generators (places), traffic counts • output data: roads with calculated traffic volumes in attribute
Security principles to be taken in consideration
N/A
4.5.5 SensLog
Name SensLog
Short Description of Functionality
SensLog is a software framework for sensor networks. It is an integrated solution includes data model and server-side application which is capable to store, analyze and publish data in various ways. The task of SensLog is to receive measured data, store them properly, pre-process them for easier queries and then publish data for clients through the system of web-services. User interface has proprietary form of web-services in JSON language or standardized form using OGC SOS ver. 1.0.0.
Past project(s) that the background tool was used
AgriSensor, LernSens
Component architecture
Application was designed as server-side application with database part. It receives Observations, Positions and Alert events through HTTP GET methods. Application stores received values in database model. Data can be published through proprietary interface in JSON format or standardized interface in XML format. Simple data visualization is realized in form of charts and maps. Detailed diagram of application is shown below.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 54
Name SensLog
Technologies used in the development
N/A
Implemented from scratch
Yes
Third party platform used (version)
Transport Planning Software OmniTRANS ver 6
User interaction requirement
SensLog enables simple user view on stored data. User interface contains map view to visualize location of sensor unit and charts to visualize measured data during past week.
Configuration needed for initiation and runtime
SensLog needs any servlet container to be deployed on. Apache Tomcat is used usually
Stress Test under heavy load results
No
Provided as a service
No
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 55
Name SensLog
Provided as binary
No
Provided as source code
Yes
Restrictions in use
N/A
Available under an open source license
N/A
Programming languages used Java 6.0
Design tools dependence
Maven 2.4, Eclipse
Hardware requirements
N/A
Software requirements
SensLog is designed as cross-platform application. Needs webserver e.g. Apache Tomcat
Third party Dependencies No
Third party libraries
• org.jvnet.ogc.sos-v_1_0_0-schema ver. 1.0.3
• postgresql ver 8.2-504.jdbc3
Automatic builds support ( Ant/Maven )
Maven ver. 2.4
Database Requirements
PostgreSQL
Requirement for version control system
Subversion
Offers an API
DataService is a service that provides detailed information about units. It contains the following methods:
1. GetUnits
This query on units provides detailed information about each unit classified in groups for
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 56
Name SensLog
selected user. It is possible to get information about connected sensors, first and last time stamp of entered observation; it contains general information about units if they are stored in database, and details about unit holder.
Request example: DataService?Operation=GetUnits
Response:
Formatted row of JSON for one unit is shown below. Every unit row contains information about administrator (holder), last position, connected sensors, after that contains numeric identifier and short description.
2. GetPositions
This request provides users specified number of last positions of all units in current group.
Parameters:
user = user name or rather group name
limit = number of positions
Request example: DataService?Operation=GetPositions&user=admin&limit=1
Response:
Response is in form of text list of requested number of units positions that belongs to specified group. Formatted row for one position is shown below. Every position contains numeric identifier, group identifier, geometry in text format, time stamp of acceptance, unit identifier, X and Y coordinates of position.
3. GetLastPosition
This request provides user last positions of all units in specified group.
Parameters:
user = user name or rather group name
Request example: DataService?Operation=GetLastPositions&user=admin
Response:
Response is similar to response of GetPositions request but the difference is that it contains only one position for every unit.
4. GetLastPositionWithStatus
This request provides user information about alert events and other attributes in addition to previous GetLastPosition request.
Parameters:
user = user name or rather group name
Request example: DataService?Operation=GetLastPositionsWithStatus&user=admin
Response:
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 57
Name SensLog
Response contains additional attributes unlike previous GetLastPosition response. An example of formatted one last position row is shown below. Position contains list of alert events and attached optional attributes, here "is_online", "is_moving" and "ignition_on".
5. GetTracks
This request returns entered number of trajectory geometries of all moving units in entered group.
Parameters:
user = user name or rather group name
limit = number of trajectories
Request example: DataService?Operation=GetTracks&user=admin&limit=1
6. GetRecentTracks
This request returns trajectory geometries of all moving units in entered group.
Parameters:
user = user name or rather group name
Request example: DataService?Operation=GetRecentTracks&user=admin
GroupService
This servlet provides detailed information about user groups. User groups can be arranged in hierarchy. The servlet contains following methods:
1. GetGroups
Request returns information about entered group.
Parameters:
user = user name or rather group name
Request example: GroupService?Operation=GetGroups&user=admin
Response:
Formatted group row is shown below. Group is characterized by group name, own numeric identifier, superior group numeric identifier and parameter if this group has any subordinate groups.
2. GetSuperGroups
Request returns information about superior group to entered group.
Parameters:
user = user name or rather group name
Request example: GroupService?Operation=GetSuperGroups&user=admin
Response:
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 58
Name SensLog
Response is list of all superior groups to entered group. Formatted group row is identical as response example GetGroups above.
3. GetSubGroups
Request returns information about subordinate groups to entered group.
Parameters:
group_id = identifier of superior group
Request example: GroupService?Operation=GetSubGroups&group_id=0
Response:
Response is list of all subordinate groups to entered group. Formatted group row is identical as response example GetGroups above.
SensorService
Servlet SensorService provides information about sensors and enables to get measured or processed data. The servlet contains following methods:
1. GetSensors
Request returns list of sensors connected to entered unit.
Parameters:
unit_id = identifier of selected unit
Request example: SensorService?Operation=GetSensors&unit_id=3002
Response:
Formatted row for one sensor is shown below. Sensor is characterized by time stamp of first and last registered observation, name of observed phenomenon and unit of measurements, numeric sensor identifier, sensor name and short description of sensor.
2. GetObservations
This request provides access to measured or processed observations for entered unit-sensor pair and entered time range. If user doesn't enter time range, servlet returns all available observations for entered unit-sensor pair. Another optional parameter is trunc that executes average of values for entered epoch (hour, day, week...).
Parameters:
unit_id = identifier of selected unit (mandatory)
sensor_id = identifier of selected sensor (mandatory)
from = time stamp of beginning time range (optional)
to = time stamp of end time range (optional)
trunc = average epoch (optional)
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 59
Name SensLog
Request example:
With unit_id and sensor_id:
/SensorService?Operation=GetObservations&unit_id=3002&sensor_id=340060000
With unit_id, sensor_id, from and to:
/SensorService?Operation=GetObservations&unit_id=3002&sensor_id=340060000&from=2011-08-13+10%3A00%3A00%2B0200&to=2011-11-17+10%3A00%3A00%2B0200
With unit_id, sensor_id, from, to and trunc:
/SensorService?Operation=GetObservations&unit_id=3002&sensor_id=340060000&from=2011-08-13+10%3A00%3A00%2B0200&to=2011-11-17+10%3A00%3A00%2B0200&trunc=day
Response:
Response consists of list of observations; formatted row for one observation is shown below. Observation is characterized by position numeric identifier, where was observation measured, time stamp of measure moment and measured value.
AlertService
AlertService provides information about alerts events that arrived in sensor network. Methods allow user to get description of potential alerts connected to specific unit and list of arrived alert events including solving state.
1. GetAlerts
This request provides list of potential alerts for specified unit.
Parameters:
unit_id = identifier of selected unit
Request example: AlertService?Operation=GetAlerts&unit_id=111
Response:
Response consists of list of potential alerts; formatted row for one alert is shown below. Alert is characterized by own numeric identifier and short description.
2. GetAlertEventsByTime
This request provides list of arrived alert events for specified unit and specified time range.
Parameters:
unit_id = identifier of selected unit
from = time stamp of beginning time range
to = time stamp of end time range
Request example:
AlertService?Operation=GetAlertEventsByTime&unit_id=3510291104618800&from=2011-03-24+01%3A00%3A00%2B0200&to=2011-11-17+10%3A00%3A00%2B0200
Response:
Response consists of list of arrived alert events; formatted row for one alert event is shown
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 60
Name SensLog
below. Alert event is characterized by own numeric identifier, identifier of unit position, time stamp, unit identifier and two phases of solving (is solving or already solved).
Communication schema supported
API
Asynchronous messaging support
No
Dependencies / Interactions with other OTN components
N/A
Input / Output requirements
N/A
Security principles to be taken in consideration
RESTful services are secured by user login and password
4.5.6 Maplog
Name MapLog
Short Description of Functionality
MapLog is a tracking tool of mobile units, which can determine the position of the device on the earth in real time. Mobile unit can be equipped with any number of additional sensors e.g., temperature, light conditions, the state of the wearer units (for human e.g. cardiac pressure, for automobiles e.g. tire pressure etc.).
Past project(s) that the background tool was used
N/A
Component architecture
The application was designed as server-side application with database part. It receives Observations, Positions and Alert events through HTTP GET methods. Application stores received values in database model. Data can be published through proprietary interface in JSON format. Application contains graphic web interface to visualize positions of units in map window.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 61
Name MapLog
Technologies used in the development
N/A
Implemented from scratch
Yes
Third party platform used (version)
MapLog was implemented from scratch. The core of MapLog was modified and is used as part of SensLog application
User interaction requirement
MapLog support user interaction, it has a web interface to visualize positions of unit on map and the values of connected sensors.
Configuration needed for initiation and runtime
SensLog need any servlet container to be deployed on. Apache Tomcat is used usually
Stress Test under heavy load results
No
Provided as a service No
Provided as binary No
Provided as source code
Yes
Restrictions in use N/A
Available under an open source license
LGPL
Programming languages used
Java EE v6.0
Design tools dependence
Maven 2.4, Eclipse
Hardware requirements
N/A
Software requirements MapLog is designed as cross-platform application. Need webserver e.g. Apache Tomcat
Third party Dependencies No
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 62
Name MapLog
Third party libraries
• net.sf.jasperreports Version:4.5.0
• net.sf.json-lib Version: 2.3
• org.slf4j.slf4j-log4j12 Version: 1.4.2
• jfree.jfreechart Version: 1.0.12
• postgresql Version: 9.0-801.jdbc4
• net.sf.saxon Version: 8.7
Automatic builds support ( Ant/Maven ) Maven ver. 2.4
Database Requirements PostgreSQL
Requirement for version control system Subversion
Offers an API
/SensorService
Operation=GetSensors
Example: /DBService/SensorService?Operation=GetSensors&unit_id=3510291104522440
Returns:
Information about current sensors status
Information about unit holders
Information about unit
/GroupService
Operation=GetGroups
Example: /DBService/GroupService?Operation=GetGroups&user=admin
Returns:
List of units available for given user
/DataService
Operation=GetLastPositions
Example: /DBService/DataService?Operation=GetLastPositions&user=admin
Returns:
List of units, their last time_stamp and position
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 63
Name MapLog
Communication schema supported
API
Asynchronous messaging support
No
Dependencies / Interactions with other OTN components
N/A
Input / Output requirements
N/A
Security principles to be taken in consideration
RESTful services are secured by user login and password
4.5.7 Mapserver
Name Mapserver
Short Description of Functionality
Server that enables visualization and publishing of geospatial data in form of OGC services.
Past project(s) that the background tool was used
Plan4business project (http://www.plan4business.eu/)
Component architecture
http://www.mapserver.org/introduction.html#anatomy-of-a-mapserver-application
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 64
Name Mapserver
Technologies used in the development
The application is written in C language using many various libraries
Implemented from scratch
Third party platform used (version)
User interaction requirement
None
Configuration needed for initiation and runtime
Setup as fastcgi process on Apache server
Stress Test under heavy load results
No
Provided as a service No
Provided as binary Yes
Provided as source code
Yes
Restrictions in use No
Available under an open source license
Yes (http://mapserver.org/copyright.html)
Programming languages used
C
Design tools dependence
No
Hardware requirements
Minimal
Software requirements Preferably a Unix system, Apache
Third party Dependencies
No
Third party libraries GDAL
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 65
Name Mapserver
Automatic builds support ( Ant/Maven )
No
Database Requirements No
Requirement for version control system
Offers an API Yes, MapScript API
Communication schema supported
Asynchronous messaging support
Dependencies / Interactions with other OTN components
Services will be published to MICKA catalogue. Viewing published services can be done in Thematic Map Viewer or in desktop GIS as QGIS for instance.
Input / Output requirements
Security principles to be taken in consideration
4.6 Service Marketplace The Service Marketplace is the module where the actors of the OTN Hub can interact and new innovative ideas and solutions can emerge. Sponsors will be able to post their insights and service challenges seeking for users to take up on them.
This module will be a custom made Liferay portlet offering all the necessary functionalities to support the aforementioned process. Users will be able to access different sections of the Marketplace based on their role. Sponsors will have the option to create new challenges whereas innovators will be able to respond to these challenges or list their own solutions and try to find a sponsor who will help them form a business solution out of these solutions.
4.7 OTN Community Forum OTN will provide a discussion place where issues, needs, solution ideas and/or new business opportunities around transportation will be hosted. Users will be able to find guidelines on how open data and geospatial datasets can be exploited. Moreover, innovators searching for more detailed business mentoring will be able to enter the Business Support Forum which will be a special section inside the main forum, only available to users with the proper permissions.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 66
The Forum will be a part of the Hub and provided, like the other components, as a Liferay portlet. Gamification elements will be implemented in order to foster the community growth and increase the engagement of the users. Accessing different sections of the forum will be controlled by Liferay’s accessing control system which will ensure that viewing restricted areas of the Forum, such as the Business Support Forum, will be only possible for users having the required access rights.
4.8 Access Control and Identity Management System As described in paragraph 3.3, the Liferay Portal uses industry standard, government-grade encryption technologies, including advanced algorithms such as DES, MD5, and RSA. It offers a customizable single sign-on (SSO) that integrates with Yale CAS, JAAS, LDAP, Microsoft Exchange, and more. Furthermore,Liferay Portal ships with robust user management and security features including password policies, user reminder settings, and complete log-in security procedures. Liferay also abides by OWASP8 guidelines to reduce the risk of security vulnerabilities. Other security features include:
• Pluggable Authentication
• Email Verification
• Session Management
The OTN Access Control and Identity Management System (ACIMS) will heavily rely on these security features of Liferay. Being developed as Liferay portlets, the deployed components will be easily configured by the administrators in order to match the access policy that will be defined in D3.5 ACM Recommendations. Private data which will reside inside the Hub’s storage or unverified VGI data will adhere to the ACM recommendations and the PIA described in D4.1 as well.
The following paragraph presents in details the identified security risks and the mitigation strategy of OTN.
4.8.1 Security Risks and Mitigation
Following the approach for designing the platform described in section 3.1, the following security risks have been identified:
Risk: Insecure access control
Likelihood: Low / Impact: High
Description: A common requirement of most multi-user information systems is to provide a mechanism for access control. Access control comprises identification, authentication and authorization. By providing insecure access control mechanisms in OTN, registered users, city administrators, developers and businesses might be able to access information of other users in the system. Furthermore, an attacker might get access to the OTN platform, which would enable him to use data that are not publicly available, misconfigure system settings or gain access to paid services like advanced data insights or business monitoring.
Risk: Identity theft
Likelihood: Medium / Impact: High
Description: Identity theft is about an attacker who pretends to be someone else. This is a serious risk, especially in an environment like OTN which offers a Marketplace and a Forum where different types of users
8https://www.owasp.org
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 67
can communicate and exchange opinions. An attacker gaining access to the OTN platform as an existing user would have access to the user’s profile and visualisations. If the stolen identity is that of a developer or business which has created private visualisations or bought access to advanced data insights and business mentoring, the attacker could alter the configurations of the visualisations, experiment with data that he is not permitted to access and even take advantage of business mentoring to shape his ideas or even steal existing ones.
Risk: Data Leakage
Likelihood: Medium / Impact: Medium
Description: Data leakage refers to unauthorized third parties gaining access to personal or business-related data. Depending on the feedback and the granularity of the data, an attacker might have access to a large number of personal/business data records. The OTN Platform will maintain open data as well as private analysed sets of data which can be used for advanced insights. Moreover, business ideas and proposed guidelines to implement them may be available in the Community/Business support Forum. Leakage of such data would expose city-related information that might be of high security risk or jeopardise the competitive edge of a user trying to shape a business idea.
Mitigation: Security pattern-based access control
Description: Security patterns are a well-established domain within the IT-security field. Security patterns describe well-proven security solutions for common IT-security problems. They are written by security experts in their respective domains. To implement access control in OTN, a combination of security patterns is required. Figure 2 below depicts a system of security patterns to implement the OTN Platform’s access control mechanism. By implementing the “Single Access Point” pattern, only a one point of access needs to be secured. The “Checkpoint” pattern provides the framework for implementing the required authentication and authorization patterns and its enforcement. Relying on a security pattern approach, the insecure access control risk can be mitigated as only authorized users have access to the OTN platform. Moreover, a secure access control mechanism also indirectly mitigates the risk for identity theft as only the authorized users have access to services and protected data. Furthermore, it prevents data leakage, as all data stored in the OTN platform is only available to authorized users.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 68
Figure 9 System of Security Patterns realizing Access Control
The OTN security features will be applied system-wise, covering all layers of architecture. The OTN ACIMS will ensure that only users with appropriate permissions will be able to access the data relying in the platform’s data storage. Moreover, the external APIs (Service Interfaces) exposed by the platform can be secured by means of encryption over HTTP via SSL, which is the standard protocol for security over the Internet.
Since we are going to use Liferay for OTN platform, any user credentials (for accessing the central platform) will be stored encrypted inside Liferay database, in a Database accessible only by Liferay server. Only the Services Layer and the UI is exposed publicly. Finally, we are not going to store sensitive data inside the platform (e.g. credit card details). If a payment workflow will be required, it will be implemented inside the payment service provider’s website, therefore there is no risk for losing such kind of data.
4.9 Storage 4.9.1 Primary data Storage
Name Primary data pool
Type Database
Status Operational
Dependencies PostgreSQL / PostGIS
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 69
4.9.2 Secondary Data Storage
An RDF store allows storage of RDF data and schema information, and provides methods to access that information. Thus, the two primary components of an RDF store are a repository and a middleware that builds on top of that repository. The middleware can be further divided into components as the access methods can be categorized into methods for adding, deleting, querying and exporting data.
Different repositories are imaginable, e.g. main memory, files or databases, but the access methods should remain the same. Thus, it is reasonable to encapsulate the access to the repository in an own layer, which provides well defined interfaces to the upper layers and can be exchanged if another repository is used. The inference support also resides in this layer as close to the repository as possible
Virtuoso
Name Secondary data pool
Type RDF data management
Status Research and development / experimental
Dependencies SQL Relational Tables and RDF Property/Predicate Graphs); Geo-Spatial support; SPARQL compiler; Jena and Sesame provider performance; JDBC Driver; Conductor CA root certificate management; WebDAV; and the Faceted Browser.
Parameters
Virtuoso is an innovative industry standards compliant platform for native data, information, and knowledge management. It implements and supports a broad spectrum of query languages, data access interfaces, protocols, and data representation formats that includes: SQL, SPARQL, ODBC, JDBC, HTTP, WebDAV, XML, RDF, RDFa, and many more.
Virtuoso delivers sophisticated Data Virtualization functionality enabling the construction of federated views -- which may or may not be materialized -- over heterogeneous RDBMS, Web Services, and Hypermedia data sources.
Running environment
The open-source edition of Virtuoso, which includes a scalable high-performance RDF Quad Store, will be the basis for the LOD2 Stack's knowledge store.
4.9.3 Semantic Indexing Infrastructure
9.3.1 D2RQ
Name Semantic Indexing Infrastructure
Type Service
Status Research and development / experimental
Dependencies Primary data pool, Scondary data pool
Parameters The D2RQ Platform is a system for accessing relational databases as virtual, read-only RDF graphs. It offers RDF-based access to the content of relational databases without
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 70
having to replicate it into an RDF store. Using D2RQ you can:
- query a non-RDF database using SPARQL
- access the content of the database as Linked Data over the Web
- create custom dumps of the database in RDF formats for loading into an RDF store
- access information in a non-RDF database using the Apache Jena API
Running environment
Primary data pool – PostgreSQL, PostGIS,
Secondary data pool – Virtuoso
Sparqlify
Name Semantic Indexing Infrastructure
Type Service
Status Research and development / experimental
Dependencies Primary data pool, Scondary data pool
Parameters
Sparqlify is a SPARQL-SQL rewriter that enables one to define RDF views on relational databases and query them with SPARQL. It is currently in alpha state and powers the Linked-Data Interface of the LinkedGeoData Server
• A novel syntax for view definitions inspired by SQL's CREATE VIEW statement. We believe this to lower the learning curve for defining RDB-RDF mappings.
• A query is rewritten into a single SQL statement, giving all control over query planning to the underlying database system.
• Support of geo-spatial functions: In general, Sparqlify supports mapping custom SPARQL functions to relational ones. Some mappings for PostGIS are already provided (e.g. intersection with polygons).
Running environment
Primary data pool – PostgreSQL, PostGIS,
Secondary data pool – Virtuoso
4.9.4 External Storage Connector
External data connector is realised through before described tools Mapserver, Geoserver and MapCreator. There are no additional tools
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 71
4.9.5 CMS and Datatank Storage
Liferay’s database layer is totally managed by Liferay and portlets communicate with it using the provided Liferay APIs.Postresql is going to be used for the purposes of OTN as the database layer of the Liferay installation.
The DataTank needs a MySQL database as a storage engine. The updates to the model of the database will run through artisan migrations of The DataTank/Laravel PHP framework.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 72
5 Conclusion The current document contains the architecture specifications and design of the integrated OTN platform that will serve as a basis for the development tasks of the project. This architecture description document will be very useful to define and communicate the initial blueprint of the OTN platform. The architecture will continue to evolve throughout the project and we need to make sure that it is consistent with design and implementation work in the other technical work packages and the early pilot activities of the project.
This deliverable presented the activities for the delivery of a baseline implementation of the OTN platform, which acts as the reference point for the actual development of this platform and offers a shared and common background for the Consortium participants on the envisaged technologies that are necessary to build such a platform.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 73
ANNEX: Technical Questionnaire
INTRODUCTION The following questions are requested to be filled by OTN’s technology partners who are going to develop the components in WP3, WP4 & WP5 and that will be integrated in the OTN Hub. Please provide your answers and you are kindly requested not to respond with a simple yes or no. Try to elaborate on your answers by giving an example of use, to the extent that this is possible. Notice: Question 3.3 is the most critical question of this Questionnaire which you are kindly requested to be as analytical as you can.
QUESTIONS
1. Questions about your component 1.1. Please name your component and provide a short description of functionality you plan to deliver in
the project and any foreseen interrelation with another OTN software component.
Component Name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Component Description: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2. Do you have an existing background tool that will be adopted in the project to support your component? If yes, please provide the following: i. Any past project that was used:
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . ii. The component architecture and technologies used in the development (Short description and if
available architecture figure): . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3. Did/Will you implement your component from scratch or is it an extension of a third party platform? If you have extended a third party platform please provide at least the following: i. Name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. ii. Version: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
1.4. Does your component support / require user interaction? If yes, please provide a short description of the expected user input.
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 74
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 1.5. Which configuration your component need for initiation and runtime?
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 1.6. Have you ever performed a stress test on your component under heavy load (for the components that
have already been developed)? If yes, please provide the test results.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 1.7. Which person in your team is the designated component "owner" (for communication purposes)?
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 1.8. Will the component be provided as a service, binary or source code?
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . 1.9. What will be the license of the delivered component?
i. Are there any restrictions in its use? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 75
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . ii. Will it be made available under an open source license?
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . .
2. Development Needs 2.1. In which programming languages does your component depend (i.e. java, C++, etc.)? Please provide
at least the following: i. Name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. ii. Version: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2. Does your component depend on existing design tools (i.e. eclipse, process modeling tool, compilation process, etc.)? Please provide at least the following:
i. Name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ii. Version: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3. Which are the hardware requirements for your component to run (i.e. any restrictions in memory
use, required CPU, etc.)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4. Which are the software requirements for your component to run (i.e. operating system, web servers like tomcat or apache, etc.) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5. Does your component have any third party dependencies or use third party libraries? If yes, please provide at least the following:
i. Name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ii. Version: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 76
2.6. Can you support automatic builds? Can you support Ant or Maven? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7. Does your component have any database requirements? If yes please provide the following: i. required design (i.e. SQL-like, noSQL): . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii. database schema (i.e. ER diagram, JSON format): . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8. Is there a requirement on a specific Version Control System?
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . .
3. Interoperability Requirements 3.1. Will your component offer an API to work as a service?
ii. If yes, please provide the full API Documentation for both input and output streams: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iii. If no, please describe the communication schema supported by your component: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2. Can your component support asynchronous messaging (i.e. non real time response/communication between the components)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3. Which are the foreseen dependencies / interactions with other OTN software components, if any? Please provide at least the following:
i. List of components: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ii. Input and / or output requirements: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
D2.5 OTN Project Architecture Blueprint and Hub Technical Specification
© OTN Consortium Version 1.0- 29/08/2014 77
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . iii. Type and aim of the information that need to be exchanged: . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4. Security principles 4.1. Which are the security principles that should be taken into consideration to protect your
component’s data? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .