innovative cloud computing...
TRANSCRIPT
ir. Dirk Frigne – @Geosparcir. Frank Maes – @Geosparc
Februari 2014
Innovative Cloud computing solutions
#smespire
Content
1. Context (open standards, open software and open data)
2. Innovative architecture for SDIs*** Coffee time ***
3. Examples of solutions4. Ecosystem of the different stakeholders5. Conclusions and business model
Learning outcome
After the training offer, the participant will be able to:
● Understand the need for open systems, data and standards and the relation between these concepts
● Define and illustrate innovative cloud solutions for SDIs● Describe and summarize examples of cloud solutions for SDIs● Describe and discuss the ecosystem of the stakeholders e.g. client,
developer, integrator,...● Understand how SMEs can take advantage of an open strategy using
cloud solutions
1. Context
Cloud Computing(°)
• Cloud computing is a phrase used to describe a variety of computing concepts that involve a large number of computers connected through a real-time communication network such as the Internet.
• In science, cloud computing is a synonym for distributed computing over a network, and means the ability to run a program or application on many connected computers at the same time.
(°)http://en.wikipedia.org/wiki/Cloud_computing
Spatial Data Infrastructure (SDI) (°)
• A spatial data infrastructure (SDI) is a data infrastructure implementing a framework of geographic data, metadata, users and tools that are interactively connected in order to use spatial data in an efficient and flexible way. Another definition is the technology, policies, standards, human resources, and related activities necessary to acquire, process, distribute, use, maintain, and preserve spatial data.
(°)http://en.wikipedia.org/wiki/Spatial_data_infrastructure
Spatial Data Infrastructure (SDI) (°)
A further definition is given in Kuhn (2005):[2]
An SDI is a coordinated series of agreements on technology standards, institutional arrangements, and policies that enable the discovery and use of geospatial information by users and for purposes other than those it was created for.
(°) http://en.wikipedia.org/wiki/Spatial_data_infrastructure
SDI – General (°)
• No central management of Data
• The data originator and/or owner is responsible
• Tools and services connect via computer networks to the various sources.
• A GIS is often the platform for deploying an individual node within an SDI.
• To achieve these objectives, good coordination between all the actors is necessary and the definition of standards is very important.
• INSPIRE is a European Commission initiative to build a European SDI beyond national boundaries
(°) http://en.wikipedia.org/wiki/Spatial_data_infrastructure
SDI – Software Components(°)
1. a software client - to display, query, and analyse spatial data (this could be a browser or a Desktop GIS)
2. a catalogue service - for the discovery, browsing, and querying of metadata or spatial services, spatial datasets and other resources
3. a spatial data service - allowing the delivery of the data via the Internet
4. processing services - such as datum and projection transformations
5. a (spatial) data repository - to store data, e.g. a Spatial database
6. GIS software (client or desktop) - to create and update spatial data
(°) http://en.wikipedia.org/wiki/Spatial_data_infrastructure
The importance of “Open”
Open Data(°)
Open data is the idea that certain data should be freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control. The goals of the open data movement are similar to those of other "Open" movements such as open source, open hardware, open content, and open access. The philosophy behind open data has been long established (for example in the Mertonian tradition of science), but the term "open data" itself is recent, gaining popularity with the rise of the Internet and World Wide Web and, especially, with the launch of open-data government initiatives such as Data.gov and Data.gov.uk.
(°)http://en.wikipedia.org/wiki/Open_data
Why publish Open Data?(°)
● Efficiency● faster response to other administrations● publish data only once
● source for new apps● f.i. traffic jams, publictransport timetables
● condition for interoperability● focus on own data (SDI principle)● reuse of data
● Validation and valorisation● feedback by users makes data more accurate● data becomes more valuable
●
● (°)http://civic.io/2013/06/22/why-publish-open-data/
Some points to consider (°)
● Privacy● Some datasets needs limitations to groups of users in a certain role
● Who is responsible for the costs (°°)● If the data is created, think about providing it for free● How to manage future costs?● experiment in Flanders (among other regions): free and open for reuse
● Open data is not sufficient● Software and tools are necessary
● open source can play an important role● interoperability is being promoted trough open source software
● Need for agreements in exchange formats● open standards play an important role(°)http://en.wikipedia.org/wiki/Open_data
(°°) See also Introduction to costs/benefits of INSPIRE & possible opportunities - by Catharina Bamps (SADL KU Leuven)
Open Software (°)
Open-source software (OSS) is computer software with its source code made available and licensed with a license in which the copyright holder provides the rights to study, change and distribute the software to anyone and for any purpose. Open-source software is very often developed in a public, collaborative manner. Open-source software is the most prominent example of open-source development and often compared to (technically defined) user-generated content or (legally defined) open-content movements.
A report by the Standish Group (from 2008) states that adoption of open-source software models has resulted in savings of about $60 billion per year to consumers.
(°)http://en.wikipedia.org/wiki/Open-source_software
More liberty
Free and open source software (FOSS)
open software
1. Freedom in the usage of your computer/infrastructure
- free software (not necessary 'gratis')
- in Spanish en French: 'Logiciël Libre'
2. CopyLeft
– as opposed to copyright
1 exception:
Distribution of the work or derived work must guarantee the same level of freedom.
Open Software – advantages
1.Ready for cloud computing
Scalability of solutions and interoperability
No Capital cost for rolling out software licenses
2.License obligations
none, or almost none (see copyleft)
Less managementoverhead for managing licenses
Less lost time with physical protection (dongles)
3.Flexibility
Made to work together with other systems
'Roll your own'
4.ControlDiversification in choice (databases, frameworks, solutions, )Virusfree :-)
Interoperability
Yet another topic we need to explain
Interoperability – types
1. Standardisation of systems
2. Bilateral exchange
3. Standardisation of the exchange language
Open Standards
We are almost there
Open Standards
The process to come to open standards due to W3C:
transparency (due process is public, and all technical discussions, meeting minutes, are archived and referencable in decision making)
relevance (new standardization is started upon due analysis of the market needs, including requirements phase, e.g. accessibility, multi-linguism)
openness (anybody can participate, and everybody does: industry, individual, public, government bodies, academia, on a worldwide scale)
impartiality and consensus (guaranteed fairness by the process and the neutral hosting of the W3C organization, with equal weight for each participant)
availability (free access to the standard text, both during development, at final stage, and for translations, and assurance that core Web and Internet technologies can be implemented Royalty-Free)
maintenance (ongoing process for testing, errata, revision, permanent access, validation, etc.)
Open Standards – advantages(°)
● Avoid Vendor Lock-In● Avoid Reverse Engineering● Freedom of Choice● Sustainability● Interoperability (between components)
(°)http://nl.wikibooks.org/wiki/Open_standaarden#Voordelen_open_standaarden
considerations(°)
● More than 15 definitions in wikipedia● More than 7 governments own definition● Different dialects● Open source <ne> Open Standards ● existing vendor lock-in
(°)http://nl.wikibooks.org/wiki/Open_standaarden#Voordelen_open_standaarden
Open Data
Interoperability
Open Source
Open Standards
2. Innovative architecture for SDI's
Why is Innovation Tough to Define(°)
... and even Tougher to Cultivate• Essential for companies to grow
• it is transformative
“using something new, or something known, but in a different way, different time or a different place.”
• Key ingredients:
– entrepreneurship
– execution
“There’s probably an oversupply of innovation,” said John Rogers, executive vice president of Goldman Sachs, “It’s the execution that makes a difference.”
(°) http://knowledge.wharton.upenn.edu/article/why-innovation-is-tough-to-define-and-even-tougher-to-cultivate/
The geomajas case
• 2005, Flanders, Belgium
– Flemish Government expressed a need for a generic GIS
– Thin client / reusable technology
– Hence the first name given to the project:
• geGIS: generic GIS
• geGIS had the following major specifications:
1) generator for eServices
• Should be deployable in a browser (2006!)
• Should be possible to use as viewer and to manage the information
2) Geographical datasources should be adaptable and independent from any used GIS package generating them
• 3) link with the MAGDA platform if possible
The offer
● DFC – K.U. Leuven was selected as winning consortium:● According to some rumours afterwards:
● the only demo which was convincing :-)● already showing some working technology● the licence model proposed was open source
• Complementary partners:
• DFC had a good track record in software develoment
• K.U. Leuven had know how of the internal processes of the client & had some experience with the client side technology (innovative Javascript based technology)
The risks
Risk description Remediation
New technology experienced team of seasoned engineers
• diverse and large group of stakeholders
• (Many chickens – few pigs)
• installing a small and efficiënt project task force with experienced stakeholders (the pigs)
• reuse of the project • decouple the open source steering comité from the project steering comité. Installation of round table discussions. (start of the community www.gegis.org)
The risks
Risk description Remediation
• Continuation of the projectresults (sustainability)
• promise to support the technology as an open source platform.
• Outside the project the process of writing a businessplan started. (19/3/2007) (Do-ocracy)
Resulted in Geomajas1.entrepreneurship (spinoff Geosparc)
About Dirk Frigne
Charter Member:
Member of the OSGeo Workgroup
• Member of the board:
• > 25 years of experience in software development
• > 20 years experience in “Geo-Software” Development
About Geosparc
• Company founded in 2008, HQ in Gent, Belgium• Main sponsor and owner (IPR) of the open source
development software for WebGIS apps, Geomajas
• Year-long experience and expertise in GIS software for public sector and businesses
• Creating and distributing GIS• Partner-centric business model
How to guarantee the project survives
• For the partner:– How to support the future of this technology and promote reuse
• Write a businessplan
• Publish the project as open source
• Consult the open source community why you did it, and why it is unique and necessary
– How to make sure the investment is valuable and sustainable
• Offer sustainable support for the technology
• Organize training
• Support community events, open to everybody
• Convince others to use the technology and to collaborate
• Persist your idea's
• Look for funding
2. execution
Who is who?
The architecture
Cloud solutions – Types
• Infrastructure as a Service – IAAS → use VM, HW, NW, OS, etc. to run your applications→ e.g. Amazon EC2, Rackspace, MS Azure
• Platform as a Service – PAAS→ build solutions using the service provider's tool stack→ e.g. Google App Engine
• Software as a Service – SAAS→ use service provider's application (use cases)→ e.g. Google Docs and Mail, SalesForce.com
Cloud solutions – Characteristics
• Web based – uses the internet• Services based – SLA
• Availability• Performance• Security
• Shared resources – economy of scale• Scalability – flexible/elastic• Measured by use – storage, hits, users,
bandwidth, etc.
Web based GIS with Geomajas
• Spatial Application Server
• Solid architecture– Services oriented– Stable API contract– Extensible via plug-ins and
interfaces– Scalable
• Strong security mechanism
Under the hood
RDBMSWeb Feature
Services
Security Context
POJO
WMSlayer
OSMlayer
Googlelayer
Geotoolslayer
Web MapServices
Hibernatelayer
Customlayer
Vector Layer Services Raster Layer ServicesOther
Services
CommandCommandCommandCustom
CommandsCommandCommandCommand
CustomServices
Command Dispatcher
Client
SHP GML ...
Geomajas Backend
What runs through our veins?
• Java• GWT• Spring• Geotools• JTS• Hibernate Spatial• Infinispan
1 Language1 IDE
Debugging!Unit testing
POJOComplex relations
Service Performance in Geomajas
• Optimized C/S communication – Protocol (low payload)– Shrinking of JavaScript (by GWT)– Lazy Feature Loading
• Rendering strategies– Configurable pipelines– On-the-fly Rasterization (server side)
• Caching
FetchVector Data
TransformVector Data
PrepareStyle
PrepareFilter
RenderSVG/VML
CachePlug-in
RenderImage
SimplifyVector Data
FetchVector Data
PrepareStyle
PrepareFilter
RenderSVG/VML
CachePlug-in
ClusterVector Data
Street Data
Points of Interest42/100
Configurabe pipelines
43/100
Caching in Geomajas
Securing Services in Geomajas
• Security – multiple organisations/users accessing same service!
• Security Interface• Supports external authentication & authorization• Fine-grained security
– Application access– Layer– Feature– Attribute– Command– Toolbar– Filter
47/100
Security – Authentication
Scalability – important considerations
• Stateless – no record of previous interactions between provider/consumer
• Avoid dependencies between instances of the software
• All instances access same data source(s)→ opportunities leveraging OGC/INSPIRE services→ but dependency on external service provider (SLA!)
49/100
Scalability
50/100
Scalability
51/100
Scalability
52/100
Scalability
53/100
Scalability
3. Examples of solutions
Cloud solutions provided by Geosparc
• MAGDA GEO – SDI at work
• Subsoil Explorer – Scalability in Government Cloud
• Infrastructure Recovery Manager – SAAS in Australia
Customer = E-government Cell Flanders (CORVE)
Goals and expectations• SAAS Web GIS solution for local governments
• Hide technical complexity – e.g. Security certificates
• Ability to load own data sources
• Ability to integrate with existing web applications
• Ability to customize / personalize
• Integration multiple data sources (incl. INSPIRE services)• Public versus protected
• Authentic sources
• Re-use generic functionality (search, print)
• Program Configure geodesks
MAGDA GEO … SDI at work
MAGDA GEO – Geodesk Enterprise Flanders
MAGDA GEO – Management Console
MAGDA GEO – Integration # data sources
MAGDA GEO – Deployment scheme
MAGDA GEO – Network configuration
MAGDA GEO – Relation with Geomajas
• Geomajas based application• Core parts build as generic plug-ins• Contribution back to the project (= win-win)
• Documentation, testing• Internationalisation
• Benefit from extensions/improvements of other users
MAGDA GEO – Security
1. ACM = autentication via eID
2. WebIDM = authorization (rights management)
3. MAGDA GEO Loket = Geodesk (Web GIS application)
4. MAGDA GEO Beheermodule = Management Console
Security in MAGDA GEO – Public versus Protected
Geodesks and Data Layers can be either public or not
Public Geodesk = accessible w/o authentication
Public Geodesk = secured and only accessible to authenticated and authorized persons
Non-public data layers are blocked in a public geodesk (security filter)
Security in MAGDA GEO – Spatial Filter on Territory
Non-public data can be secured using a spatial filter on Territory
“Territory of the Administrator” = shows only the data for the territory of the geodesk admin (ex. geodesk set up by the admin of the city of Gent will only show data for the Gent territory)
“Territory of the User” = shows only the data for the territory of the authenticated user (ex. user of the city of Gent will only see data for the Gent territory / user of the province of Limburg will see data for the whole province)
MAGDA GEO – Restful service
Restful service
SHPKML TXT
MAGDA GEO – OGC services
WFS WMS
Geoserver
4. Ecosystem of the different stakeholders
The best of both worlds: combining an open source
business model with a proprietary one
ir. Dirk FrigneFOSS4G CEE – Wednesday June 19
70
An ecosystem is a community of living organisms (plants, animals and microbes) in conjunction with the nonliving components of their environment (things like air, water and mineral soil), interacting as a system
Business and open source
Driving Forces?
– Respect– Community– You can change the world
72
FOSS4GCommunity
users
integration solutions
developers researchers
developers
CASCADOSS: Model of Berlecon Research (2002)
Software Value Chain
Dual-licensing
Dual-licensing
Support selling
Support selling
Platform Provider
Platform Provider
Consulting
Consulting
Accessorizing
Accessorizing
Software-as-a-service
Software-as-a-service
5. Conclusions and business model
88
Proprietary Business versus open source
Observation: Some individuals are well known
Bill Gates (Microsoft)
Larry Ellison (Oracle)
Jack Dangermond (ESRI)
Steve Jobs (Apple)
All proprietary businesses
89
ESRI Business Model(°)
Core business: creating and distributing proprietary GIS
Sales: sell the products using their own sales staff, international distributors and authorized business partners
Support: provide a variety of support services related to those products.
Geographic strategyUSA: direct sales trough own non-commisioned sales teams
– account managers– business partner coordinators– technical sales engineers
Outside the U.S.:– international distributors– 2200 business partners
(°)http://events.esri.com/uc/QandA/index.cfm?fuseaction=answer&conferenceId=85D7FF7F-1422-2413-199D382F1109DE0A&questionId=2269
90
Critical success factors
What do they have in common?
91
Critical success factors
Core business: creating and distributing proprietary GIS
Sales: sell the products using their own sales staff, international distributors and authorized business partners
Support: provide a variety of support services related to those products.
Geographic strategyUSA: direct sales trough own non-commisioned sales teams
– account managers– business partner coordinators– technical sales engineers
Outside the U.S.:– international distributors– 2200 business partners
Sell ProductsSales & Marketing
92
Critical succes factors
Sell Products?
Open source is all about tools and libraries
Gnu compiler created before Linux
Geotools created before Geomajas
recurring revenues are Key!
Selling hours is great, but... not sustainable (enough)
93
Critical success factors
Core business: creating and distributing proprietary GIS
Sales: sell the products using their own sales staff, international distributors and authorized business partners
Support: provide a variety of support services related to those products.
Geographic strategyUSA: direct sales trough own non-commisioned sales teams
– account managers– business partner coordinators– technical sales engineers
Outside the U.S.:– international distributors– 2200 business partners
ProprietaryProtection
Sell ProductsSales & Marketing
94
Protection – Intelectual Property
bsa.org: (business software alliance)
AdobeAppleAutodeskBentley SystemsCA technologiesCNC Software - Mastercam
IBMIntelIntuitLMSMcAfeeMicrosoftMinitab
OraclePTCQuestRosetta StoneSiemens PLM Software, Inc.SymantecThe MathWorks
95
Protection – Intelectual Property
bsa.org: (business software alliance)
Properly licensed software has a positive impact on national economic activity that is more than three times the impact of pirated software […] And the additional economic value associated with lawful software use is especially pronounced in developing markets: Every dollar invested in properly licensed software in low-income countries yields an astounding $437 in additional national production, on average.
96
Protection – Intelectual Property
geomajas.org: (geo mapping with ajax and svg :-)
OSGeo software is properly licensed ...
Physically protecting your software costs a lot of money, frustration, pain for the users and the developers …
(or maybe the protection industry is part of the added economical value :-)
So: protection is necessary, also for open source, but don't bother the physical protection mechanisms. Respect should be the mechanism for users and developers to enforce the protection. Naive?
97
Protection necessary for valorization
The principle of enterpreneurship is great– Fun– Challenging– New innovations– The future to save the world– (very) hard work
Protection is key– To get a start with your ideas– To claim your territory– To compete with closed source
98
Work together
100
GeomajasCommunity
users
IntegrationPartners
Solution Partners
Developers researchers
otherCommunities
Solution Partners
Protect IP
Protect IP