fiware from open data to open apis
TRANSCRIPT
http://www.fiware.org
http://lab.fiware.org
Follow @FIWARE on Twitter!
FIWARE: From Open Data to Open APIsSergio Garcia Gomez
Telefonica I+D. FIWARE Data/Context Chapter and Smart Cities Architect
[email protected], @ggsergio (twitter)
Agenda
FIWARE overview
Open Data
Introduction to Context Management
Smart City examples
Open & Agile Smart Cities
1
Cities are where daily life and businesses
actually happen …
2
Smart Cities are not simply about more efficient
municipality services but transforming Cities into
ICT platform enabling innovation local economy
growth and well-being
But … How can the Smart Cities ultimately enable local
economy growth and well-being?
3
ecosystemopen globalsustainable
Build
an
Ecosystem and platform: two tied concepts
4
Open Standard
Platform
ecosystemopen globalsustainable
A successful ecosystem is that one which creates the right
incentives to all stakeholders
5
Entrepreneurs, Developers
• Develop once for a large market
• Easily meet potential customers
• Marketing, promotion
• Ability to test with real data and
end users
• Simple yet powerful APIs that
accelerate product development
The Smart City
• Connect to entrepreneurs
• Put their data at work
• Bring new innovative services to
end users
• Be more efficient
• Social Reputation
Platform technology Providers
• Ability to “coopete”
• Connect to entrepreneurs: jointly
exploit the opportunities
ecosystem
open globalsustainable
Sonny Abesamis
6
An open smat city
Kai SchreiberJason Kuffer
Open Standards
and Specifications
to avoid vendor
lock-in
Modular and non-
intrusive platform to
build a smart city
Enablement of
solutions
interoperability in the
city and portability
among cities
Entertainment
Industry
Tourism
Mobility
Agriculture
7
Software
Platform
Innovation
Experimentation
Nodes
deployment and
management
Startups
funding
Global
footprint
Strategy: Build an ecosystem that will work as catalyst for capturing the
opportunities, engaging data providers and entrepreneurs
The FIWARE Public Private Partnership
eHealth
Smart
Grids
Public
Administration Smart
Cities
Goal:
capture opportunities
derived from the new wave of
digitalization of life and businesses that is coming
FIWARE PPP: main figures
8
… Had a single Company made an
investment like this alone?
24
9countries
partners
122 = 85 + 37 M€FIWARE budget (phases 1-2 + phase 3)
99 = 76 + 23 M€FIWARE funding (phases 1-2 + phase 3)
80 + 20 M€for startups/SMEs
1300Startups/SMEs
21Innovation Hubs
6,5 M€marketing
450 K€sponsorships
next 2 years
16+Cities
95/66 M€budget/funding of
Vertical Use
Cases
18 = 17 + 1FIWARE Lab nodes
FIWARE major differential features
9
• Federation of infrastructures (private/public regions)
• Automated GE deploymentCloud
• Complete Context Management Platform
• Integration of Data and Media Content
Data/Media Context Mgmt
• Easy plug&play of devices using multiple protocols
• Automated Measurements/Action Context updates
IoT Services Enablement
• Visualization of data (operation dashboards)
• Publication of data sets/services
Data/Services Delivery
• Easy incorporation of advanced 3D and AR features
• Visual representation of context information
Advanced Web UI
• Security Monitoring
• Built-in Identity/Access/Privacy ManagementSecurity
• Advanced networking (SDN) and middleware
• Interface to robotsI2ND
Reference Architecture
10
Presentation Layer & APIs
Content/Data Storage, Management, Processing & Analysis
IoT Backend Acquisition/Actuation Layer
Data
• Data is the fuel that enables, inspires and boosts entrepreneurs, remarkably in Smart Cities.
Lab
• FIWARE Lab offers the environment where all the stakeholders meet together around innovation.
FIWARE
• FIWARE provides the technology to build the entrepreneurs and developers ecosystem.
Ingredients to connect an Smart City
11
Open Data/Content approaches
12
Datasets
Existing Datasets(census,
geographical, tourism,...)
Historic Data (from sensors,
events...)
Real Time
Vertical Systems(mobility, events...)
Internet of Things(sensors, Smart
meters...)
Media
Video streams(traffic,
surveillance..)
Audio (microphones),
speaches...
Applications
NGSICKANWEBRTC
KURENTO
Open Data/Content approaches
13
Datasets
Existing Datasets(census,
geographical, tourism,...)
Historic Data (from sensors,
events...)
Real Time
Vertical Systems(mobility, events...)
Internet of Things(sensors, Smart
meters...)
Media
Video streams(traffic,
surveillance..)
Audio (microphones),
speaches...
Applications
CKAN
Open Data Platform
De facto standard platform for open data in Europe and
beyond.
Search & Discover Data:
• Search by keywords
• Browse by facets
• Explore data with previews & visualization
• REST/JSON APIs to access data and metadata
Data Management for publishers
• Easy store & update of metadata and data
Plenty of extensions: harvesting, geographical
information, data visualization….
14
Publishing and Managing Data
15
Search and discovery
16
Metadata
17
Open Data/Content approaches
18
Datasets
Existing Datasets(census,
geographical, tourism,...)
Historic Data (from sensors,
events...)
Real Time
Vertical Systems(mobility, events...)
Internet of Things(sensors, Smart
meters...)
Media
Video streams(traffic,
surveillance..)
Audio (microphones),
speaches...
Applications
NGSI
Being “Smart” requires first being “Aware”
Implementing a Smart City requires gathering and managing context
information describing the current and historic “state” of the city
Context information refers to the values of attributes characterizing
entities relevant to city services, governance and third-party apps
19
Bus
• Location
• No. passengers
• Driver
• Licence plate
Citizen
• Name-Surname
• Birthday
• Preferences
• Location
• ToDo list
Shop
• Location
• Business name
• Franchise
• offerings
Context Information
City Governance System
City Services Third-party Apps
Context Management in FIWARE
The FIWARE Context Broker GE implements the OMA NGSI-9/10 API:
a simple yet powerful standard API for managing Context information
The FIWARE NGSI API is Restful: any web/backend programmer gets
quickly used to it
20
Application/Service
Bus
• Location
• No. passengers
• Driver
• Licence plate
Citizen
• Name-Surname
• Birthday
• Preferences
• Location
• ToDo list
Shop
• Location
• Business name
• Franchise
• offerings
Context Broker
NGSI API
FIWARE NGSI: Context from different sources
Context information may come from many sources:
• Existing systems
• Users, through mobile apps
• Sensor networks
Source of info related to a given entity may vary over time
21
Place = “X”, temperature = 30º
What’s the current temperature in place “X”?
NGSI API
A sensor in a
pedestrian street
The Public Bus Transport
Management systemA person from his smartphone
It’s too hot!
FIWARE NGSI: Basic interaction
Context Producers publish context information by invoking the
updateContext operation on a Context Broker.
Context Consumers can retrieve context information by invoking the
queryContext operation on a Context Broker
22
Bus = “X”, location = (x, y)updateContext
Context Broker
Context Producer
Context Consumer
queryContext
FIWARE NGSI: Subscription to notifications
Context Consumers can be subscribed to reception of context information
complying with certain conditions, using the subscribeContext operation a
ContextBroker exports. Such subscriptions may have a duration.
The Context Broker notifies updates on context information to subscribed
Context Consumers by invoking the notifyContext operation they export
23
Bus = “X”, next_stop = “A”, arrived = “Yes”updateContext (context_info)
Context Broker
Context Producer
Context Consumer
(consumer1)
notifyContext (id, context_info)
Id = subscribeContext (consumer1,
condition, duration
)
FIWARE NGSI: Context Providers
Context Providers can be registered to the Context Broker as “holders” of
certain context information.
A Context Broker will invoke the queryContext or updateContext operations
exported by Context Providers whenever they are queried for, or asked to
update, context information they hold
24
Bus = “X”, location = (x, y)queryContext / updateContext
Context Broker
Context Provider
(provider-x)
Context Consumer
queryContext / updateContext
registerContext (provider-x,
registration_data, duration, id)
)
Integration with existing systems
Context adapters will be developed to interface with existing systems (e.g.,
municipal services management systems in a smart city) acting as Context
Providers, Context Producers, or both
Some attributes from a given entity may be linked to a Context Provider while
other attributes may be linked to Context Producers
25
queryContext (e1,
attr1, attr2)
Context Provider
queryContext (e1,
attr1)
Context Consumer
updateContext (e1,
attr2)
Application
Context Broker
System B
(e.g. Transport
system)
System A
(e.g. GIS, POIs)
FIWARE NGSI: “The SNMP for IoT”
Capturing data from, or Acting upon, IoT devices becomes as easy as
to read/change the value of attributes linked to context entities using
a Context Broker
26
Context Broker
NGSI APINGSI API
GET <Oauth token>
/V1/contextEntities/lamp1/attributes/presenceSensor
PUT <Oauth token>
/V1/contextEntities/lamp1/attributes/status
“light on”
Setting up the value of attribute
“status” to “light on” triggers
execution of a function in the IoT
device that switches the lamp on
Issuing a get operation on the
“presenceSensor” attribute
enables the application to get
info about presence of people
near the lamp
Integration with sensor networks
The backend IoT Device Management GE enables creation and configuration
of NGSI IoT Agents that connect to sensor networks
Each NGSI IoT Agent can behave as Context Consumers or Context
Providers, or both
27
FIWARE Context Broker
IoT
Agent-1
IoT
Agent-2
IoT
Agent-n
IoT Agent
Manager
create/monitor
FIWARE Backend IoT
Device Management
OMA NGSI API (northbound interface)
(southbound interfaces)
MQTTETSI M2M IETF CoAP
NGSI resource visualization
NGSI resources visualization
Open data publication
Once context information is gathered, a lot of useful
complementary FIWARE enablers can be used
30
Context Broker
Advanced Web-based UI (AR,
3D)
Data/Apps visualization
Big Data AnalysisComplex Event
ProcessingMultimedia processing
Target Smart City platform
Smart city platform as
a Data/Knowledge Hub
Non-intrusive, open to
third parties
CKAN
Big Data
Context Broker
Ac
co
un
ting
& P
aym
en
t & B
illing
IDM
& A
uth
Short-term
historic
data
BigData
Processing
Data
Quering/Action,
Publish/Subscr
Open Data
publishingReal-time
processing
BI
ETL
RULES
DEFINITION
TOOL
OPERATIONA
L
DASHBOARD
KPI GOVERNANCE OPEN DATA PORTALS
Service
orchestrator
Context
Adapters
CEP
31
IoT Backend
Device Management
measures /
commands
IoT Broker & Config
Management
(from sensors to things)
IoT/Sens
orOpen Dataactuators
Media
streams
Real Time
Media
Stream
Processing
City
Services
GIS
Inventory
Specific Enablers
Generic Enablers
Cities in FIWARE Lab
32
City (Country) IoT Open Data NGSI Prototype or details
Amsterdam (NL) x x CitySDK-LD
Barcelona (Spain) X API implementation to municipal portal
Espoo (Finland) x Energy consumption dashboard
Helsinki (Finland) x Participation dashboard (CitySDK-Open311)
Las Palmas (Spain) x x Port management dashboard
Lisbon (Portugal) x Mobility and social networks
Lleida (Spain) x x Public transport and accessibility
Logroño (Spain) x x Smart watering, City App
Malaga (Spain) x x x Citizen as a sensor
Rome (Italy) x Data dashboard
Santander (Spain) x x Big Data / Open Data publication of IoT
Sevilla (Spain) x x x Fountains water management, Crowds detection
Torino (Italy) x Security & participation
Trento (Italy) x SmartCampus” Open Service platform
Valencia (Spain) x x Smart Taxi
Vigo (Spain) x x City dashboard
IoT infrastructure from Smart Santander
Description: Integration of the IoT SmartSantander facility on
top of the FIWARE platform
• Devices registration
• Observations generation
• Definition of new dictionaries to support sensor
measurements as well as citizens observations (mainly
social related events).
• Guidelines to access SmartSantander data (both historical
and live data) by using Generic Enablers
Used GEs: Backend Device Management, Cosmos, IdM,
Context Broker, CKAN
Partners: University of CantabriaBackend Device Management [IDAS]
SmartSantanderCore Network
Service StorerResource Configurator
Device Gateway(SensorML, Lightweight SensorML)
Context Broker [ORION] Big Data [COSMOS]
IdM [KeyRock]
PEP-Proxy [Wilma]
FIWARE Architecture
Token Request
ContextRequest
HistoricalRequest
Malaga Citizen as a Sensor
Description: The Malaga CitySense project aims to innovate in
data generation for Smart City through the citizen cooperation
using use of their Smartphone and his sensors.
App that generates data (temperature, light, humidity, pressure,
etc.), shows relevant open data from the city, detects iBeacons
deployed to track tourism visits.
Web application shows data and statistics.
Used GEs: Context Broker, Wirecloud, CKAN
Partners: Soidemer/TopDigital
Helsinki Open311 (issue reporting)
Description: The main objective of the pilot was to prepare a
working prototype software to a) integrate Helsinki Open311
issue reporting API to FIWARE platform b) store Helsinki issue
reports (based on Open311 standard) to FIWARE Orion
Context Broker c) display Helsinki issue reporting API data on
FIWARE Wirecloud environment. For this purpose a dedicated
Orion Context Broker instance was setup using FIWARE Cloud
environment. In addition to this Helsinki Service registry API
was used to receive additional information to be displayed in
mashup environment.
Used GEs: Context Broker, Wirecloud
Partners: Forum Virium
Streets video based crowds detection
Description: This system captures video stream from city video surveillance cameras and uses complex
computational vision algorithms for detecting when a crowd of people is being formed, that is a group of
people moving very slowly or not moving at all. The system detects different levels of crowdness and
published them into the FIWARE Context Broker (Orion), so that applications can subscribe to them and
execute specific actions based on thresholds or other mechanism. A Wirecloud widget allows to define areas
for cloud detection in the canvas.
Used GEs: Kurento, Orion, Wirecloud
Partners: URJC,Naevatec, UPM
KurentoMediaServer
FI-LAB
OrionContextBroker
Applica onServer
h" p://&
Video
Stream
VideoStream
ContextEvents
MediaEvents
WWWtraffic
Citystreets
WWWBrowser
Alarms
Otherapplica ons
37
Telefonica,
Orange,
Engineering &
ATOS to build
and support
standards for
smart cities
through FIWARE
Open Source
Community
31 cities all over Europe & Brazil joins
Open and Agile Smart Cities initiative
38
39
Open & Agile Smart cities initiative
Implementation Driven
(FIWARE Lab & Acceleration, Organicity,
Code4Europe)
Common APIs
(NGSI)
City Models
(CitySDK, etc)
Open Data
Platform
(CKAN)
Helsinki, Espoo, Vantaa, Tampere, Oulu
and Turku in Finland; Copenhagen,
Aarhus and Aalborg in Denmark;
Brussels, Ghent and Antwerp in
Belgium; Porto, Lisbon, Fundão,
Palmela, Penela and Águeda in
Portugal; Milan, Palermo and Lecce in
Italy; Valencia, Santander, Málaga and
Sevilla in Spain; and Olinda (Recife),
Anapólis (Goiás), Porto Alegre (Rio
Grande do Sul), Vitória (Espírito Santo),
Colinas de Tocantins (Tocantins) and
Taquaritinga (São Paulo) in Brazil
http://fiware.org
http://lab.fiware.org
Follow @Fiware on Twitter !
Check latest videos at www.youtube.com/user/FIWARE
Join us!
40
Why FIWARE
Driver What is needed? What does FIWARE bring?
Technology Open, driven by implementation, specs (open
source reference implemenation)
Sustainable investment over time
Open specifications backed by open source
reference implementations (see [1], [2])
100+ M€ of investment (2011-2016)
Experimental
environment
Ability to experiment with real data coming from
cities (not just open historic datasets but real-time
dat).
Free Cloud capacity enabling entrepreneurs to test
and host a permanent showcase of their
applications.
15 cities (7 in Spain) already working on setting
up a connection to FIWARE Lab [3]
3000+ cores, 16Tb RAM and 750+ Tb HD will be
the free computing capacity provided by the
FIWARE Lab Cloud across 16 nodes distributed in
Europe
Incentives for
creating the
ecosystem
Engagement of technology providers,
entrepreneurs, data providers, customers
Funding for first entrepreneurs joining the
ecosystem.
Invesment in promotion and dissemination activities
52 partners, 13 countries (just FIWARE)
100 M€ devoted to fund entrepreneurs in 2014-
2016. Additional opportunities in Horizon 2020.
6,2+ M€ devoted to dissemination
Global footprint Helping entrepreneurs and technology providers to
create opportunities not just in Europe but other
regions (Latam, Asia and, why not, USA)
FIWARE Lab nodes in Mexico and Brazil.
Conversations between EC and public authorities
in Mexico and Brazil to explore collaboration
opportunities
[1] – http://wiki.fi-ware.org/Summary_of_FIWARE_Open_Specifications
[2] – http://catalogue.fi-ware.org
[3] – http://lab.fi-ware.org41
FIWARE = advanced OpenStack-based Cloud capabilities
+ library of APIs that ease development of applications
42
Sustainability
ensured
Driven by
implementation
The FIWARE Accelerator Program
43
How can standard Smart City data models easing
common solutions be defined? The problem
Smart City apps can be ported from one Smart City to
another once their platforms provide the same set of
APIs, that’s why FIWARE brings a rather high value
Without standard data models, Smart City apps would
need to come with adapters that transform data made
available by the city so that it complies with the data
model handled by the app but that has proven to be
easy with OMA NGSI (overall if NGSI is at both ends)
Creation of standard Smart City data models would
allow to avoid performing this kind of adaptation and
make portability of Smart City apps across Smart City
platforms a pretty straightforward task
How creation of these standard Smart City data
models can be fostered?
44
How can standard Smart City data models easing
common solutions be defined? The solution
A “design by committee” approach would not be
the best approach:
• Such kind of approach has proven to be wrong in many
other standardization efforts in the past
• Who grants that the defined model is suitable for what apps
need and developers want to have?
we need a “driven by implementation” approach:
• Identify real applications that solves a real problem and
cities would like to see running in their cities
• Check what data model they have been designed to work
with and take it as input
• Carry out a “data curation” process targeted to extract
commonalities and polish the data model
You will end with a set of standard data models and
soon a portfolio of killer Smart City apps working!
45
How can standard Smart City data models easing
common solutions be defined? FIWARE
The FIWARE Accelerator program enables to identify
killer Smart City apps developed by entrepreneurs
• These applications can as basis for definition of a standard
set of Smart City data models
• FIWARE resources would be devoted to help coordinating
the data curation process
• Involvement in this process becomes also an incentive for
the entrepreneurs to join the program (“I want to influence
the standard so that my app can easily align with it”, “I want
to provide one of the first example applications”)
• There are 80 M€ for entrepreneurs that can be put at work!
Cities connected to FIWARE Lab will play a key role:
• Their data models will be contrasted/analyzed against those
coming from the apps and other cities
• They would get involved in the data curation process
46
FIWARE Generic Enablers (GEs)
A FIWARE Generic Enabler (GE):
set of general-purpose platform functions available through APIs
Building with other GEs a FIWARE Reference Architecture
FIWARE GE Specifications are open (public and royalty-free)
FIWARE GE implementation (FIWARE GEi):
Platform product that implements a given GE Open Spec
There might be multiple compliant GEis of each GE Open Spec
One open source reference implementation of each
FIWARE GE (FIWARE GEri):
• Well-known open source license
• Publicly available Technical Roadmap updated in every release
Available FIWARE GEis, GEris and incubated enablers are
published on the FIWARE Catalogue
47
FIWARE Instances
Future Internet Applications run on top of “FIWARE Instances” that are
built by “FIWARE Instance Providers” upon:
• selection of FIWARE GEis (products) from the FIWARE Catalogue.
• assembly of selected FIWARE GEis with added-value components.
FIWARE Catalogue
FIWARE GE implementations
FIWARE Instance
48
Future Internet App
assemble…
added-value components
Domain-specific platforms = FIWARE + specific
enablers
FIWARE
49
GEs
SMART City
Apps
SMART Factory
Apps
SMART
Agrifood Apps
Domain
specific
enablers
FIWARE Ops: suite of tools for FIWARE providers
DeploymentDeployment of basic Cloud Hosting GEs and Monitoring Adapters in a FIWARE node
Federation Management
Federate a new FIWARE node within a given FIWARE instance (e.g., the FIWARE Lab)
Connectivity ManagementManage connectivity of services across FIWARE nodes of a FIWARE instance
Service Offer ManagementRegistration and deployment of additional Generic Enablers, Specific Enablers and
complementary Future Internet Facilities
Building the FIWARE ecosystem: the vision
FIWARE Catalogue
creation of dedicated
GE instances
Use of Global shared
GE instance
FIWARE provider A
51
Applications
FIWARE provider B
Applications
ff
4,2 M€ promotion campaign
• Campus Party events
• Startup Weekend events
• Chambers of Commerce
• 870 K€ in prizes
100 M€ of funding devoted
to entrepreneurs in phase 3
of the FIWARE program
• 12 nodes in Europe
providing up to 3000+
cores, 16TB+ Ram,
750TB+ HD
• Creation of nodes in
Mexico (1000+ cores)
and Brazil
• Level 1 and Level 2
support for the nodes
• Showcases for
developers,
infrastructures, smart
businesses
FIWARE Lab: A working instance of FIWARE working as
“the meeting point” where innovation takes place