team aquafu (aperio) · -vue.js + vuestic data analytics - tableau desktop - microsoft azure...

67
Team AquaFu (Aperio) Data Science on Water Data

Upload: others

Post on 22-May-2020

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Team AquaFu

(Aperio)Data Science on Water Data

Page 2: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Analysis of water quality problems - a challenge

● Water Quality sensors,

Flow sensors

● Sensors of different types

on different pipes.

● Unlabeled data.

Page 3: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Tools

Python - Pandas, scikit-learn, jupyter notebook

Python - OSISOFT PI Web API library (written by Marcos Loeff)

Elasticsearch + Kibana. Python code written to Export PI to Elasticsearch

Page 4: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Correlate the Flow Rate & Temperature vs Dist (km)

Page 5: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Correlate the Water Quality (Optical Refraction)

Distance - km

Page 6: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 7: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 8: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 9: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Thanks!

Page 10: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 11: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 12: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 13: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 14: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 15: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 16: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 17: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 18: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 19: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 20: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 21: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 22: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 23: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 24: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 25: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 26: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 27: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 28: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 29: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 30: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 31: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 32: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 33: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 34: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 35: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 36: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 37: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 38: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 39: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 40: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 41: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills
Page 42: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

PyVision

Advanced Anomaly Detection for Water Quality Monitoring

Page 43: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

What

●Replace the monitor lizard

●Advanced Anomaly Detection at all

locations

●Automatically create alarms &

notifications

Page 44: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Why

●Ensuring reliable service

●Enabling faster

maintenance response

●Reduce likelihood of false

alarms by combining

multiple sensors

Page 45: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

How

●HDBSCAN (hierarchical

density-based spatial

clustering w/ noise)

●Looking for spatial

density-adaptive outliers in

multiple dimensions

Page 46: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Production Site Noordbergum - conductivity

Page 47: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Production Site Noordbergum – pH & turbidity

Page 48: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

PI AF

Page 49: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

PI Vision Dashboard

Page 50: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Thank you!

Page 51: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

OSIsoft EMEA 2017 Hackathon event

Page 52: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

OSIsoft EMEA 2017 Hackathon event

„Objects on the presentation might look older than it appear“

Page 53: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Who are we?

5

3

MarekData Plumber

PrzemyslawCoder

Jakub

Coder

GregorThe other

Alexa„her“

Page 54: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Provide our of their future

A connected system that combines realtime data, detects anomalies and provide this to the right person to enable the right action.

5

4

The challenge

Page 55: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

5

5

Provide our of their future

Data is acquired and

transmitted to a central

location

Analytics in real time identify

anomalies. Machine learning

analytics predicts by correlating

signals and past experience

When the data indicates an

issue that can cause a

problem, it is assigned to

the right work force and

affected parties are notified

if required. Interventions are performed

to resolve the issue.

Solutions are recorded.

Outage and incidents are

prevented and private

households dissatisfaction

eliminated

Connected system enabling learning cycles

where gained knowledge can be apply to future

issue to detect earlier or resolve faster

Page 56: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

5

6

What we did

Page 57: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

5

7

PI AssetFramework

-Analysis

-EventFrames

Webtechnologies

-PI Web API

-Realtime data (channels/websockets)

-Batch queries

-EventFrames (fetch and acknowledge)

-Vue.js + Vuestic

Data Analytics

- Tableau Desktop

- Microsoft Azure Machine Learning Studio

- SQL

- PI OLEDB

Extended Userexperience

- Alexa

- Alexa Skills Kit

- AWS Lambda Function (Node.js)

Used technologies

Page 58: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

5

8

Dashboard

Page 59: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

6

1

Data Analytics (Cloud)

Using data extracts via PI OLEDB. Visualizing the data with

Tableau or experimenting with Microsoft Azure Machine Learning Studio by

apply existing algorithms.

Page 60: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

6

2

Data Analytics (Cloud-Integration)

Page 61: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

6

3

Data Analytics (Tableau)

Using data extracts via PI OLEDB. Visualizing the data with Tableau or

experimenting with Microsoft Azure Machine Learning Studio by apply existing

algorithms.

Page 62: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

6

4

Data Analytics (PI Analytics)

Page 63: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Consider all kind of data

+

By categorizing issues / anomalies not just to operative events but public events too (e.g.

football, vacation) is it possible to predict data anomalies by just checking the calendar?

Page 64: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

●A connected system provides a better overview and enables a seamless workflow by having

systems integrated

●Using existing powerfull tools to prevent implementation of things that already exists (e.g.

Cortana Intelligence Gallery)

●Enabling data exploration by using Tableau for vizualisation

●Interaction with the system in new way (Alexa)

●Saving costs predicting events faster and assigning field forces at the right time

How this will help f

Page 65: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Thanks

Page 66: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

E.ON Climate & Renewables -

ConnectPointE.ON is an international, privately-owned energy supplier focused on 3 areas:

● Renewables

● Energy networks

● Customer solutions

It operates in over 30 countries and serves over 33 million customers.

The Company employs above 50 000 people world-wide and annually achieves revenues above 100 billions Euro

E.ON Climate & Renewables (EC&R), headquartered in Essen, Germany, is responsible for E.ON’s industrial-scale renewable energy activities.

EC&R develops, builds and operates large renewable energy assets in Europe and North America.

6

8

ConnectPoint is a Software vendor and System Integrator specialized in

Utilities & Industry sector

The company is present in Poland and Germany with the team of

experienced professional in IT/OT integration and SW development

ConnectPoint started co-operation with E.ON Climate & Renewables in

2013 and since that time the company conducted above 20 projects in

areas like

• Custom Software development

• System Integration in IT/OT area

• Business Intelligence applications development (e.g. OSIsoft PI)

Page 67: Team AquaFu (Aperio) · -Vue.js + Vuestic Data Analytics - Tableau Desktop - Microsoft Azure Machine Learning Studio - SQL - PI OLEDB Extended Userexperience - Alexa - Alexa Skills

Find out

more.If you would like to find out more

about how ConnectPoint can help

transform your utility service with

intelligent network capability, get in

touch with

konrad.kazmierowski@connectpoint.

pl

6

9