environmental data management and analytics
Post on 21-Apr-2017
58 Views
Preview:
TRANSCRIPT
PowerPoint Presentation
Environmental Data Analytics
Dr Prasad Modak, Tausif Farooqui
1
Structure of Presentation
2
EMCs Experiences
Environmental Data Analytics
3
EMC
Experiences on Environmental Data analytics
Dr Prasad Modak had setup the first data base management system in India for environmental data for CPCB in 1986 Which included Air Data managementWater data management, Consent managementsCess calculationsWater Quality data analysis for Godavari Monthly data for 7 years
Indian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths
4
EMC
Experiences on Environmental Data analytics
Indian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths5
EMC
Experiences on Environmental Data analytics
Industries are responsible to collect data for mandatory submission to regulator, by law.Industries also use these data sets for disclosuresAnd in sustainability reportsIndian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths6
EMC
Experiences on Environmental Data analytics
Egyptian Pollution Abatement Programme-phase 2Dr Prasad Modak had setup an online environmental monitoring system which was to serve regulator and financer
Indian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths7
7
EMC
Experiences on Environmental Data analytics Analysis of real-time water quality monitoring data of River Ganga 2015Data generated by sensors installed at 10 stations.Geo-Database for Punjab water supply department 2016Analysis and Dashboard for National Ground Water Management Improvement program- India 2016Indian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths
8
8
EMC
Experiences on Environmental Data analytics
Analysis of air quality in a air shed with 2 power plants - 2016Data from Ambient monitoring, stack measurements, automatic real-time monitoring, and metrological real time monitoring were analyzed Air quality model was built for spatial prediction of pollutant concentrationIndian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths
9
9
EMC
Experiences on Environmental Data analytics
EMC had setup an EMIS for Rourkela steel plant in 2004
Indian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths10
EMC
Experiences on Environmental Data analytics
Corporate Sustainability Report for Glenmark Pharmaceuticals -2015
Indian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths11
EMC
Experiences on Environmental Data analytics
Web based GIS for management for the Ahmedabad-Mehsana Toll Road.Data of Maintenance and toll Cause-effect analysis for accidents Showcased by ESRI an one of the first application of ARC on the web
Indian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths12
EMC
Experiences on Environmental Data analytics
Designing the data structureDesigning the MIS, EMISOnline/offline applicationsEnvironmental Data analysisEnvironmental modelingGIS applicationsCommunity applications crowd sourcing
Indian RegulatorsInternational Financing institutionManufacturing IndustriesInfrastructure and Transport IndustryOur Strengths13
Structure of Presentation
14
Characteristics of Environmental Data
Environmental Data Analytics
15
Characteristics of Environmental Data
MultivariateBIG DataIrregular
FuzzyPrimary Data and Meta Data16
Characteristics of Environmental Data
BIG DATA
Summing up all the characteristics of environmental dataEssentially a BIG DATA management and processing system is neededTo generate meaningful and actionable reports from the data
MultivariateBIG DataIrregular
Fuzzy17
Link to image
http://www.centrodeinnovacionbbva.com/sites/default/files/bigdata_ejemplos_cibbva.jpg17
Characteristics of Environmental Data
Various data sets have different frequency of measurementsAmbient Air Quality parameters are monitored twice a week while stack and metrological data is monitored in real time (15 min period). Other data sets like heath, agriculture are recorded and published monthly or seasonally.
It creates a complicated situation to merge and consolidate all data sets.MultivariateBIG DataIrregular
Fuzzy18
Characteristics of Environmental Data
Data from different types of instrumentationMultivariateBIG DataIrregular
Fuzzy19
http://www.sviva.gov.il/PhotoAlbum/Air%20Quality/3air-mana.jpg
http://3.imimg.com/data3/HK/AI/MY-1830445/250-500x500.jpg
19
Characteristics of Environmental Data
Large volumes of dataUncertainties are very highFrequent and large Data gapsAccuracy of measurement is questionable at some instances
It makes the data not ready for consumption of decision makers
MultivariateBIG DataIrregular
Fuzzy20
To derive value from these data sets, advanced application of DATA analytics is required
21
212121
Role Based reports
Characteristics of Environmental Data22
22
Structure of Presentation
23
Dashboard
Environmental Data Analytics
24
Dashboard
Visual representation of the DataAll the information at one placeKey IndicatorsA single screen for insights and analyticsObjective oriented or goal basedAdvanced statistical/mathematical models What is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau25
DashboardWhat is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau
26
https://www.thebookdesigner.com/wp-content/uploads/2015/04/bigstock-Car-dashboard-background-75463333.jpg
26
DashboardDo you record data or measurements ? YesWhere do you use the recorded data ?For decision makingFor complianceInformation dissemination mass communication
How do you use the recorded data ?Print ?Excel, pivot tables ? Or Visualize ?
For BIG data which method would you prefer? What is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau27
27
Dashboard
BIG DATA AnalyticsTo understand the dataTo manage the dataTo process the captured data in a hierarchical mannerTo visually communicate dataTo generate Role based reports instantlyTo make data driven business or operations decisionsTo create alerts
28What is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau
28
Dashboard
http://www.forbes.com/What is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau29
29
DashboardWhat is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau
30
30
DashboardWhat is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau
31
31
DashboardWhat is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau32
More about comparison
https://bitipsblog.wordpress.com/2016/10/30/test/https://comparisons.financesonline.com/sas-business-intelligence-vs-tableau-software32
DashboardWhat is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau33
More about comparison
https://bitipsblog.wordpress.com/2016/10/30/test/https://comparisons.financesonline.com/sas-business-intelligence-vs-tableau-software33
Dashboard Tableau is a business intelligence (BI) tool
Tableau connects easily to nearly any data source
Tableau allows transforming data into visually appealing, interactive visualizations
Compatibility across Multiple Platforms - desktop tool, web browser, iPad or mobile phone.
What is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau34
DashboardTableau's Data Analytic and Visualization Features Interactive visualizationsSimultaneous Connections with multiple data sourcesAllows to create custom calculationsCharts, Graphs, Heat maps etc.Time series functionsMap visualizations (point, line, polygon)
Access controlMultiple simultaneous dashboards
What is a Dashboard?Why Dashboard?Sample DashboardsToolsTableau
35
Structure of Presentation
36
Dashboard for Air Quality Management
Environmental Data Analytics
37
What does Real Time Environmental Data Analytics Involve?
Dashboard for Air Quality Management
Allows Quick and Dynamic Visualizations
Mines the BIG data to Optimize Sampling Frequencies and get More Value
Uses Real Time Algorithms with Advanced Statistical Tools
Detects Trends, Patterns and allows Forecasting
Helps in establishing association between Emissions and Ambient Quality or understand Source Influences
FeaturesInputParametersillustrationsNew ConceptsThe Future38
Emissions
Ambient Air Quality
Meteorology
Ambient, Stack and Automatic monitoring and metrological data in a single dashboard for effective interpretation, reporting and taking action
What are the important factors for air quality management?
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future39
Analytical features for Ambient Air
Dashboard for Air Quality Management
Missing Values treatmentOutliers Basic Statistics (Mean, Deviations, Percentiles, Type of Distribution)Box Whisker PlotsHistograms and Frequency Diagrams Compliance with standards with timestamps Detecting simultaneous ViolationsLocation Importance Index Correlations Inter parameter and Inter stationTrend Analysis (magnitude, direction and statistical significance)Air Quality Index
FeaturesInputParametersillustrationsNew ConceptsThe Future40
Time Date of observation/ measurementObserved/Measured values of Concentration of Particulate Matter (PM10, PM2.5)Concentration of Gasses ( CO2, NOx )Location of the station Meta data like up time, calibration date, battery level, sensor make, sensor accuracy etc.,. Parameters for Ambient Air
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future41
Analytical features for Stack Emission
Dashboard for Air Quality Management Missing Values Outliers Basic Statistics (Mean, Deviations, Percentiles, Type of Distribution)Box Whisker PlotsHistograms and Frequency Diagrams Compliance with standards with timestamps Detecting simultaneous ViolationsStack Importance Index Predictive Emission Modelling
FeaturesInputParametersillustrationsNew ConceptsThe Future42
Parameters for Stack Emission
Dashboard for Air Quality Management Time Data of observation/ measurementObserved/Measured values of Concentration of SPM, other pollutantsTemperatureGenerationLocation of the stack Meta data like up time, calibration date, battery level, sensor make, sensor accuracy etc.,.
FeaturesInputParametersillustrationsNew ConceptsThe Future43
Analytical features for Metrological data
Dashboard for Air Quality Management Wind rose, Pollution rose Persistent wind roseNon-Parametric Wind Regression (NWR) for Source IdentificationStack-station Influence analysisFeaturesInputParametersillustrationsNew ConceptsThe Future44
Parameters for Meteorological data
Dashboard for Air Quality Management Time Data of observation/ measurementObserved/Measured values of TemperatureWind SpeedWind DirectionRelative HumiditySolar RadiationFeaturesInputParametersillustrationsNew ConceptsThe Future45
Missing Value
Dashboard for Air Quality Management an observation not recorded
Treatment of Missing ValueReplaced by Mean of Window or the selected time spanReplaced by Mean of NeighborsFeaturesInputParametersillustrationsNew ConceptsThe Future46
Demo Dashboard
Visualize parameters NOxSOxPM10PM2.5
Can be extended to include other parameters
Demo is prepared for AIR, a similar dashboard can be developed for any other Environmental Data
Dashboard for Air Quality ManagementFeaturesInputParametersillustrationsNew ConceptsThe Future47
Options in dropdown boxesSelect Visualization Type Maps, Statistics, Charts
MapChartStats
Dashboard for Air Quality ManagementFeaturesInputParametersillustrationsNew ConceptsThe Future48
Dashboard for Air Quality Management
User can only download the PDF and Image of the screen.
User cannot however download the raw data behind it.
FeaturesInputParametersillustrationsNew ConceptsThe Future49
Station 1
Station 2
Parameter 1Parameter 2AveragingPeriod
User options
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future50
Selected stations are located on the map,
Map is sourced from ISRO- Bhuvan
Click to open satellite view of the location in Google maps
Map Visualization
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future51
Green band permitted values in designated water quality class
Black lines higher and lowerStatistically acceptable permitted values, beyond this value is an outlier
Blue line Chart of selected parameter
Red dots Both of the selected parameters are violating applicable standards simultaneously
StationsParameter 1
Parameter 2
-Charts Visualization
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future52
Basic Statistics Number of RecordsMax, Min, Mean, MedianSD, CoV, Number of OutliersPercentage violations, and Longest Contiguous violation for both the selected parameters
StationsParameter 1
Parameter 2
Percentage simultaneous violations by both parameters against relevant standard. Longest Contiguous and Simultaneous violation -Crosstab Visualization
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future53
An outlier is an observation point that is distant from other observations.User option to select method of treating outliersUser option: Show/Hide outlierOutlier is removed and treated as missing value
Outlier
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future54
Basic StatisticsBox Whisker Plots
Dashboard for Air Quality Management Annual Maximum Annual Minimum Annual Mean CPCB standardStandard DeviationFeaturesInputParametersillustrationsNew ConceptsThe Future55
Compliance with standards with timestamps
Dashboard for Air Quality Management
Dates of violationsFeaturesInputParametersillustrationsNew ConceptsThe Future56
Radial ChartDistribution of Wind speeds Vs Direction
Wind Rose
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future57
Inter-station correlations
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future58
Trend Analysis (magnitude, direction and statistical significance)
Dashboard for Air Quality Management
FeaturesInputParametersillustrationsNew ConceptsThe Future59
Radial ChartDistribution of Pollutant Concentration Vs Direction
Pollution Rose
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future60
Compliance Histogram
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future61
Detecting simultaneous Violations
Dashboard for Air Quality Management
FeaturesInputParametersillustrationsNew ConceptsThe Future62
Percent exceedance (how many times is the standard for a parameter violated in relation to the length of the data chain)Extent of contiguous exceedance (how long have been the violation over standard on a continuous basis in relation to the length of the data chain)Magnitude of exceedance (how high are the violations when there is an exceedance over standard, expressed in sum of squares of deviations)
Location Importance Index (LII)
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future63
Location Importance Index (LII)
Dashboard for Air Quality Management Locations could be a receptor, ambient station or a stackFeaturesInputParametersillustrationsNew ConceptsThe Future64
Distribution of Consistency in wind direction Vs Direction in the number of hours in a Radial ChartData Quality check for automatic metereology stationsPersistent wind rose
Dashboard for Air Quality Management FeaturesInputParametersillustrationsNew ConceptsThe Future65
Stack-station Influence analysis
Dashboard for Air Quality Management
Influence of emissions from a particular stack on a particular ambient stationInstances of influence are plotted as time-series
FeaturesInputParametersillustrationsNew ConceptsThe Future66
Stack-station Influence analysis
Dashboard for Air Quality Management
Air Quality measurements are taken at Stack and Ambient
Estimates influence of a stack emission on a specific ambient monitoring station
Longest contiguous duration of a receptor under influence of stack
FeaturesInputParametersillustrationsNew ConceptsThe Future67
Non-Parametric Wind Regression (NWR) for Source Identification
Dashboard for Air Quality Management
NWR has been applied for better understanding of emission sources of influence since early 2000
Technique of NWR is used to identify presence and locations of major emission influence on the observed pollutant concentrations
NWR requires high frequency (e.g. one hour) concentration and meteorological data at the location where emission influences are to be assessedFeaturesInputParametersillustrationsNew ConceptsThe Future68
Non-Parametric Wind Regression (NWR) for Source Identification
Dashboard for Air Quality Management
FeaturesInputParametersillustrationsNew ConceptsThe Future69
69
Source Diagnostics or Environmental Forensics
Dashboard for Air Quality Management
Data science + Environmental scienceApplication of Sophisticated modelling tools, AERMOD for air quality Receptor ModellingSource identification using Met and Pollution Concentration DataAnalyzing of special events Technique of Intervention analyses Predictive emissions modelling system
FeaturesInputParametersillustrationsNew ConceptsThe Future70
AERMOD
Air Dispersion Modelling71US EPA based ModelGaussian plume air dispersion modelPredict downwind pollutant concentrations based on source emissions, meteorological field, and site parameters (land use, terrain features etc.)Model can predict accurately for 50km
FeaturesInputParametersillustrationsNew ConceptsThe Future
Source apportionment
Air Dispersion Modelling72
FeaturesInputParametersillustrationsNew ConceptsThe FutureClick on the box to see Percentage contribution of two power plants of ambient PM10 at a sensitive receptor
Modelled percentage contribution of two power plants of ambient PM10 at a sensitive receptor
Period% contribution of A% contribution of BWinter36.14.7Summer14.313.9Post monsoon 45.37.9Annual32.28.5
73
Integration of Models and Dashboard
Integrated Approach74Air Emission and Air Quality Dashboard integrated with the Dispersion models
Model could be fired from the dashboard itself and required data on emissions, air quality and meteorology will be picked up automatically from data available.
Model will keep re-calibrating itself with actual real time data and this will increase the accuracy of the model outputs.
Models will also help to validate the data collected, thereby improving the quality of monitored data.
FeaturesInputParametersillustrationsNew ConceptsThe Future
PEMS
Predictive EmissionsMonitoring System75FeaturesInputParametersillustrationsNew ConceptsThe Future
PEMS
Predictive EmissionsMonitoring System76Developed on Artificial Neural Network (ANN) conceptsThe model automatically predict emissions data in real time while exploiting the inherent correlation between process variables (flow, temperature and pressure) and emission properties (NOx, SO2, CO, CO2)PEMS can replace CEMS and hence save on costs
FeaturesInputParametersillustrationsNew ConceptsThe FutureClick on the box to see Comparison between CEMS & PEMS
CEMSPEMSCostLarge capital & operational expenditureLess expensive (50% to 75%) than CEMS.MaintenancePeriodic maintenance and calibration of the sensor and analyser is requiredNo additional instrument maintenance (other thannormal process sensors) is requiredAccuracyCEMS are the standard to which PEMS are measuredPEMS are initially calibrated using CEMS. The CEMS are removed when the PEMS becomes operational. PEMS are less accurate than CEMS when the latter is well maintained and calibratedUsesCEMS alarms when emissions are approaching or exceeding an environmental standardPEMS can alarm, but also predict future outcomes and optimize processes in real-time. PEMS therefore leads to higher production efficiency and hence lower cost of production.
77Comparison of CEMS and PEMS
Thank you
Please visit http://dev-data-portal.pantheonsite.io/
Dr Prasad Modak Prasad.modak@emcentre.comTausif Farooqui Tausif.farooqui@emcentre.com
78
Structure of Presentation
79
top related