ai and big data for smart city in silicon valley, usa ... · a silicon valley excellence research...
TRANSCRIPT
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
AI and Big Data For Smart City in Silicon Valley USA- Issues Solutions and Challenges
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Building Smart City Complex Systems for San Jose- Current Project Activities
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
The Research Center Mission and Capability
ldquoTo enrich the lives of its students to transmit knowledge to its students along withthe necessary skills for applying it in the service of our society and to expand thebase of knowledge through research and scholarshiprdquo
SJSU Universityrsquos Mission
bull Provide a multi-disciplinary research platform bull Use SJSU campus and local cities as living laboratories bull Gain innovative research experience bull Learn use and develop cutting-edge technologiesbull Solve complex issues in complex cyber systems
ldquoTo provide a multi-disciplinary research platform for SJSU faculty to create innovations and buildpractical and future solutions with cutting-edge technology to address the issues and challenges inbuilding complex systems and provide a live learning and research experience for SJSU students withrich hands-on experience and skills so that they are well-prepared to meet the future workforceneeds in Silicon Valley
Focuses- Research and develop sustainable technologies intelligent solutions and quality and safe systems
that connect objects people and services-based on trustworthy data using- Validated intelligent techniques
SJSU STCCSrsquos Mission
SJSU and City of San Jose are teamed up as a task force for Smart Cities
SJSU
Multidisciplinary Research Capabilities
Smart City Complex Systems Big Data Services and Analytics
Smart Sensing and PlatformsIoT Cloud and Mobile Clouds
Smart Learning amp Campus
Four Research Areas
Area 3 Smart World
- Smart Resource amp Recycling Systems
- Smart Green and Energy Systems
- Smart Ecological Systems
- Smart Earth Systems Engineering
Area 4 Smart Living
- Smart Home +
- Smart FoodDrinkClothing
- Smart Healthcare
- Smart Living amp Behaviors
Area 2 Smart City
- Smart Streets
- Smart Community
- Smart Transportation
- Smart Government
- Smart City as Lab
- Smart City Safety
Area 1 Smart Campus amp Learning
- Smart Campus Sensor Cloud amp IoT
- Smart Campus Management amp Program
- Smart Interactive Learning
- Smart Campus as Lab
Major Smart City Issues and Challenges in San Jose
Illegal Dumping
How to Build Clean and Green City
Graffiti in the City
How to Provide Safe and Secure City
Where is the money
How to build connected communitiesHow to construct sustainable cities City Big Data Where we can find
- Controlling service for wireless camera system- Hot-Spot station registration - Video object collection amp detection- Data communication with the server- Data communication with mobile APP
San Jose City Hot Spot ndash Illegal Dumping
Mobile Station
Smart Hot-Spot Illegal Dumping Monitor System
- WiFi-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
Mobile Services
City ServiceCloud
One San Jose City Street
Camera-Based Trash Truck
Mobile Station
Mobile APP
Mobile-Edge Based Illegal Dumping Detecting amp Service System
City ServiceCloud
- GPS-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
- Controlling service for wireless camera system- Mobile station registration - Video object collection amp detection- Data communication with the Server- Data communication with mobile APP
Mobile Services
Smart Clean Street Assessment System Using Big Data Analytics
Level 1
Level 1
Level 2
Level 2
Level 3Level 3
- Controlling service for wireless camera system- Mobile station registration - On-Land Trash Assessment on Mobile Station- Data communication with the Server- Data communication with mobile APP
Camera-Based Trash Truck
City ServiceCloud
Mobile APP
Mobile Station
- GPS-based MonitorTracking- Video object detection and learning- Grid-based photo object detection- Communication with mobile APP- Communication with Mobile Station- Real-Time Static AssessmentMobile Services
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Building Smart City Complex Systems for San Jose- Current Project Activities
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
The Research Center Mission and Capability
ldquoTo enrich the lives of its students to transmit knowledge to its students along withthe necessary skills for applying it in the service of our society and to expand thebase of knowledge through research and scholarshiprdquo
SJSU Universityrsquos Mission
bull Provide a multi-disciplinary research platform bull Use SJSU campus and local cities as living laboratories bull Gain innovative research experience bull Learn use and develop cutting-edge technologiesbull Solve complex issues in complex cyber systems
ldquoTo provide a multi-disciplinary research platform for SJSU faculty to create innovations and buildpractical and future solutions with cutting-edge technology to address the issues and challenges inbuilding complex systems and provide a live learning and research experience for SJSU students withrich hands-on experience and skills so that they are well-prepared to meet the future workforceneeds in Silicon Valley
Focuses- Research and develop sustainable technologies intelligent solutions and quality and safe systems
that connect objects people and services-based on trustworthy data using- Validated intelligent techniques
SJSU STCCSrsquos Mission
SJSU and City of San Jose are teamed up as a task force for Smart Cities
SJSU
Multidisciplinary Research Capabilities
Smart City Complex Systems Big Data Services and Analytics
Smart Sensing and PlatformsIoT Cloud and Mobile Clouds
Smart Learning amp Campus
Four Research Areas
Area 3 Smart World
- Smart Resource amp Recycling Systems
- Smart Green and Energy Systems
- Smart Ecological Systems
- Smart Earth Systems Engineering
Area 4 Smart Living
- Smart Home +
- Smart FoodDrinkClothing
- Smart Healthcare
- Smart Living amp Behaviors
Area 2 Smart City
- Smart Streets
- Smart Community
- Smart Transportation
- Smart Government
- Smart City as Lab
- Smart City Safety
Area 1 Smart Campus amp Learning
- Smart Campus Sensor Cloud amp IoT
- Smart Campus Management amp Program
- Smart Interactive Learning
- Smart Campus as Lab
Major Smart City Issues and Challenges in San Jose
Illegal Dumping
How to Build Clean and Green City
Graffiti in the City
How to Provide Safe and Secure City
Where is the money
How to build connected communitiesHow to construct sustainable cities City Big Data Where we can find
- Controlling service for wireless camera system- Hot-Spot station registration - Video object collection amp detection- Data communication with the server- Data communication with mobile APP
San Jose City Hot Spot ndash Illegal Dumping
Mobile Station
Smart Hot-Spot Illegal Dumping Monitor System
- WiFi-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
Mobile Services
City ServiceCloud
One San Jose City Street
Camera-Based Trash Truck
Mobile Station
Mobile APP
Mobile-Edge Based Illegal Dumping Detecting amp Service System
City ServiceCloud
- GPS-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
- Controlling service for wireless camera system- Mobile station registration - Video object collection amp detection- Data communication with the Server- Data communication with mobile APP
Mobile Services
Smart Clean Street Assessment System Using Big Data Analytics
Level 1
Level 1
Level 2
Level 2
Level 3Level 3
- Controlling service for wireless camera system- Mobile station registration - On-Land Trash Assessment on Mobile Station- Data communication with the Server- Data communication with mobile APP
Camera-Based Trash Truck
City ServiceCloud
Mobile APP
Mobile Station
- GPS-based MonitorTracking- Video object detection and learning- Grid-based photo object detection- Communication with mobile APP- Communication with Mobile Station- Real-Time Static AssessmentMobile Services
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
The Research Center Mission and Capability
ldquoTo enrich the lives of its students to transmit knowledge to its students along withthe necessary skills for applying it in the service of our society and to expand thebase of knowledge through research and scholarshiprdquo
SJSU Universityrsquos Mission
bull Provide a multi-disciplinary research platform bull Use SJSU campus and local cities as living laboratories bull Gain innovative research experience bull Learn use and develop cutting-edge technologiesbull Solve complex issues in complex cyber systems
ldquoTo provide a multi-disciplinary research platform for SJSU faculty to create innovations and buildpractical and future solutions with cutting-edge technology to address the issues and challenges inbuilding complex systems and provide a live learning and research experience for SJSU students withrich hands-on experience and skills so that they are well-prepared to meet the future workforceneeds in Silicon Valley
Focuses- Research and develop sustainable technologies intelligent solutions and quality and safe systems
that connect objects people and services-based on trustworthy data using- Validated intelligent techniques
SJSU STCCSrsquos Mission
SJSU and City of San Jose are teamed up as a task force for Smart Cities
SJSU
Multidisciplinary Research Capabilities
Smart City Complex Systems Big Data Services and Analytics
Smart Sensing and PlatformsIoT Cloud and Mobile Clouds
Smart Learning amp Campus
Four Research Areas
Area 3 Smart World
- Smart Resource amp Recycling Systems
- Smart Green and Energy Systems
- Smart Ecological Systems
- Smart Earth Systems Engineering
Area 4 Smart Living
- Smart Home +
- Smart FoodDrinkClothing
- Smart Healthcare
- Smart Living amp Behaviors
Area 2 Smart City
- Smart Streets
- Smart Community
- Smart Transportation
- Smart Government
- Smart City as Lab
- Smart City Safety
Area 1 Smart Campus amp Learning
- Smart Campus Sensor Cloud amp IoT
- Smart Campus Management amp Program
- Smart Interactive Learning
- Smart Campus as Lab
Major Smart City Issues and Challenges in San Jose
Illegal Dumping
How to Build Clean and Green City
Graffiti in the City
How to Provide Safe and Secure City
Where is the money
How to build connected communitiesHow to construct sustainable cities City Big Data Where we can find
- Controlling service for wireless camera system- Hot-Spot station registration - Video object collection amp detection- Data communication with the server- Data communication with mobile APP
San Jose City Hot Spot ndash Illegal Dumping
Mobile Station
Smart Hot-Spot Illegal Dumping Monitor System
- WiFi-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
Mobile Services
City ServiceCloud
One San Jose City Street
Camera-Based Trash Truck
Mobile Station
Mobile APP
Mobile-Edge Based Illegal Dumping Detecting amp Service System
City ServiceCloud
- GPS-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
- Controlling service for wireless camera system- Mobile station registration - Video object collection amp detection- Data communication with the Server- Data communication with mobile APP
Mobile Services
Smart Clean Street Assessment System Using Big Data Analytics
Level 1
Level 1
Level 2
Level 2
Level 3Level 3
- Controlling service for wireless camera system- Mobile station registration - On-Land Trash Assessment on Mobile Station- Data communication with the Server- Data communication with mobile APP
Camera-Based Trash Truck
City ServiceCloud
Mobile APP
Mobile Station
- GPS-based MonitorTracking- Video object detection and learning- Grid-based photo object detection- Communication with mobile APP- Communication with Mobile Station- Real-Time Static AssessmentMobile Services
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
SJSU
Multidisciplinary Research Capabilities
Smart City Complex Systems Big Data Services and Analytics
Smart Sensing and PlatformsIoT Cloud and Mobile Clouds
Smart Learning amp Campus
Four Research Areas
Area 3 Smart World
- Smart Resource amp Recycling Systems
- Smart Green and Energy Systems
- Smart Ecological Systems
- Smart Earth Systems Engineering
Area 4 Smart Living
- Smart Home +
- Smart FoodDrinkClothing
- Smart Healthcare
- Smart Living amp Behaviors
Area 2 Smart City
- Smart Streets
- Smart Community
- Smart Transportation
- Smart Government
- Smart City as Lab
- Smart City Safety
Area 1 Smart Campus amp Learning
- Smart Campus Sensor Cloud amp IoT
- Smart Campus Management amp Program
- Smart Interactive Learning
- Smart Campus as Lab
Major Smart City Issues and Challenges in San Jose
Illegal Dumping
How to Build Clean and Green City
Graffiti in the City
How to Provide Safe and Secure City
Where is the money
How to build connected communitiesHow to construct sustainable cities City Big Data Where we can find
- Controlling service for wireless camera system- Hot-Spot station registration - Video object collection amp detection- Data communication with the server- Data communication with mobile APP
San Jose City Hot Spot ndash Illegal Dumping
Mobile Station
Smart Hot-Spot Illegal Dumping Monitor System
- WiFi-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
Mobile Services
City ServiceCloud
One San Jose City Street
Camera-Based Trash Truck
Mobile Station
Mobile APP
Mobile-Edge Based Illegal Dumping Detecting amp Service System
City ServiceCloud
- GPS-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
- Controlling service for wireless camera system- Mobile station registration - Video object collection amp detection- Data communication with the Server- Data communication with mobile APP
Mobile Services
Smart Clean Street Assessment System Using Big Data Analytics
Level 1
Level 1
Level 2
Level 2
Level 3Level 3
- Controlling service for wireless camera system- Mobile station registration - On-Land Trash Assessment on Mobile Station- Data communication with the Server- Data communication with mobile APP
Camera-Based Trash Truck
City ServiceCloud
Mobile APP
Mobile Station
- GPS-based MonitorTracking- Video object detection and learning- Grid-based photo object detection- Communication with mobile APP- Communication with Mobile Station- Real-Time Static AssessmentMobile Services
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Four Research Areas
Area 3 Smart World
- Smart Resource amp Recycling Systems
- Smart Green and Energy Systems
- Smart Ecological Systems
- Smart Earth Systems Engineering
Area 4 Smart Living
- Smart Home +
- Smart FoodDrinkClothing
- Smart Healthcare
- Smart Living amp Behaviors
Area 2 Smart City
- Smart Streets
- Smart Community
- Smart Transportation
- Smart Government
- Smart City as Lab
- Smart City Safety
Area 1 Smart Campus amp Learning
- Smart Campus Sensor Cloud amp IoT
- Smart Campus Management amp Program
- Smart Interactive Learning
- Smart Campus as Lab
Major Smart City Issues and Challenges in San Jose
Illegal Dumping
How to Build Clean and Green City
Graffiti in the City
How to Provide Safe and Secure City
Where is the money
How to build connected communitiesHow to construct sustainable cities City Big Data Where we can find
- Controlling service for wireless camera system- Hot-Spot station registration - Video object collection amp detection- Data communication with the server- Data communication with mobile APP
San Jose City Hot Spot ndash Illegal Dumping
Mobile Station
Smart Hot-Spot Illegal Dumping Monitor System
- WiFi-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
Mobile Services
City ServiceCloud
One San Jose City Street
Camera-Based Trash Truck
Mobile Station
Mobile APP
Mobile-Edge Based Illegal Dumping Detecting amp Service System
City ServiceCloud
- GPS-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
- Controlling service for wireless camera system- Mobile station registration - Video object collection amp detection- Data communication with the Server- Data communication with mobile APP
Mobile Services
Smart Clean Street Assessment System Using Big Data Analytics
Level 1
Level 1
Level 2
Level 2
Level 3Level 3
- Controlling service for wireless camera system- Mobile station registration - On-Land Trash Assessment on Mobile Station- Data communication with the Server- Data communication with mobile APP
Camera-Based Trash Truck
City ServiceCloud
Mobile APP
Mobile Station
- GPS-based MonitorTracking- Video object detection and learning- Grid-based photo object detection- Communication with mobile APP- Communication with Mobile Station- Real-Time Static AssessmentMobile Services
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Major Smart City Issues and Challenges in San Jose
Illegal Dumping
How to Build Clean and Green City
Graffiti in the City
How to Provide Safe and Secure City
Where is the money
How to build connected communitiesHow to construct sustainable cities City Big Data Where we can find
- Controlling service for wireless camera system- Hot-Spot station registration - Video object collection amp detection- Data communication with the server- Data communication with mobile APP
San Jose City Hot Spot ndash Illegal Dumping
Mobile Station
Smart Hot-Spot Illegal Dumping Monitor System
- WiFi-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
Mobile Services
City ServiceCloud
One San Jose City Street
Camera-Based Trash Truck
Mobile Station
Mobile APP
Mobile-Edge Based Illegal Dumping Detecting amp Service System
City ServiceCloud
- GPS-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
- Controlling service for wireless camera system- Mobile station registration - Video object collection amp detection- Data communication with the Server- Data communication with mobile APP
Mobile Services
Smart Clean Street Assessment System Using Big Data Analytics
Level 1
Level 1
Level 2
Level 2
Level 3Level 3
- Controlling service for wireless camera system- Mobile station registration - On-Land Trash Assessment on Mobile Station- Data communication with the Server- Data communication with mobile APP
Camera-Based Trash Truck
City ServiceCloud
Mobile APP
Mobile Station
- GPS-based MonitorTracking- Video object detection and learning- Grid-based photo object detection- Communication with mobile APP- Communication with Mobile Station- Real-Time Static AssessmentMobile Services
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
- Controlling service for wireless camera system- Hot-Spot station registration - Video object collection amp detection- Data communication with the server- Data communication with mobile APP
San Jose City Hot Spot ndash Illegal Dumping
Mobile Station
Smart Hot-Spot Illegal Dumping Monitor System
- WiFi-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
Mobile Services
City ServiceCloud
One San Jose City Street
Camera-Based Trash Truck
Mobile Station
Mobile APP
Mobile-Edge Based Illegal Dumping Detecting amp Service System
City ServiceCloud
- GPS-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
- Controlling service for wireless camera system- Mobile station registration - Video object collection amp detection- Data communication with the Server- Data communication with mobile APP
Mobile Services
Smart Clean Street Assessment System Using Big Data Analytics
Level 1
Level 1
Level 2
Level 2
Level 3Level 3
- Controlling service for wireless camera system- Mobile station registration - On-Land Trash Assessment on Mobile Station- Data communication with the Server- Data communication with mobile APP
Camera-Based Trash Truck
City ServiceCloud
Mobile APP
Mobile Station
- GPS-based MonitorTracking- Video object detection and learning- Grid-based photo object detection- Communication with mobile APP- Communication with Mobile Station- Real-Time Static AssessmentMobile Services
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
One San Jose City Street
Camera-Based Trash Truck
Mobile Station
Mobile APP
Mobile-Edge Based Illegal Dumping Detecting amp Service System
City ServiceCloud
- GPS-based MonitorTracking- Video object detection and learning- Communication with mobile APP- Communication with Mobile Station- Alerting amp Reporting
- Controlling service for wireless camera system- Mobile station registration - Video object collection amp detection- Data communication with the Server- Data communication with mobile APP
Mobile Services
Smart Clean Street Assessment System Using Big Data Analytics
Level 1
Level 1
Level 2
Level 2
Level 3Level 3
- Controlling service for wireless camera system- Mobile station registration - On-Land Trash Assessment on Mobile Station- Data communication with the Server- Data communication with mobile APP
Camera-Based Trash Truck
City ServiceCloud
Mobile APP
Mobile Station
- GPS-based MonitorTracking- Video object detection and learning- Grid-based photo object detection- Communication with mobile APP- Communication with Mobile Station- Real-Time Static AssessmentMobile Services
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Smart Clean Street Assessment System Using Big Data Analytics
Level 1
Level 1
Level 2
Level 2
Level 3Level 3
- Controlling service for wireless camera system- Mobile station registration - On-Land Trash Assessment on Mobile Station- Data communication with the Server- Data communication with mobile APP
Camera-Based Trash Truck
City ServiceCloud
Mobile APP
Mobile Station
- GPS-based MonitorTracking- Video object detection and learning- Grid-based photo object detection- Communication with mobile APP- Communication with Mobile Station- Real-Time Static AssessmentMobile Services
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Smart Illegal Dumping Service System - Infrastructure
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System
City Wireless Network
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Major Project Objectives
Objective 1 To find out the data-driven emergencyalerting coverages for six different nature disasterscenarios such as earthquakes floods fire accidentsand so on
Objective 2 To find the system performancelimitations and research ability problems inunderlying emergency system infrastructures
Objective 3 To propose the ideas enhancementapproaches and even new alerting systeminfrastructures and solutions for the near future
ProblemsAlert System CoverageAlert System PerformanceLimitations of Current SystemsImprovements for Future Systems
Smart City ndash Smart Emergency Alerting System
Major Challenges
Challenge 1 Lack of big data and lack of useful big data integrationChallenge 2 Lack of built-in testing and simulation servicesChallenge 3 Lack of big data Legacy System with slow performanceChallenge 4 Lack of effective and easy way to get big data
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
How to Provide Smart amp Safe Living Environment
Homes and cars are swampedon Last Wednesday in San Jose
Forest Fire in CaliforniaCalifornia Drought Earthquake in California
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Forest on Fire
camera
Real-Time Forest Fire Monitor Analysis and Alerting System
Satellite Based Forest FireDetectionSensor Based Forest Fire Detection
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
A Smart Graffiti Clean-up System Based on An Autonomous Drone
bull Project Goal
- Building a smart graffiti clean-up system using autonomous drone using smart solution and machine learning
bull Challenges
- Automatic graffiti detection and alerts
- Automatic graffiti clean-up
- Auto Pilot for Drone in City Street
Focused Issue
(a) Graffiti Detection and Reporting
(b) Graffiti Cleaning up
Major Reasons
- High-Cost and Labor Intensive in Clean-Up
- Impact the City Image and Environment
- Affect City Traffic and Transportation Safety
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
A Smart Graffiti Clean-up System Based on An Autonomous Drone
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
City AI and Big Data Analysis for Smart Cities
- Part I - City Illegal Dumping Object Detection
Paper and Report from Akshay Dabholkar Bhushan Muthiyan Shilpa Srinivasan Swetha Ravi Hyeran Jeon Jerry Gao
Paper and Master Project Report fromWei-Chung Chen Xiaoming Chuang Wendy Hu Luwen Miao and Jerry Gao
- Part II - Street Litter Object Detection and Framework
Paper and Master Project Report fromChandni Balchandani Rakshith Koravadi Hatwar Parteek Makkar Yanki Shah Pooja Yelure Magdalini Eirinaki
Paper and Master Project Report fromBharat Bhushan Kavin Pradeep Sriram Kumar Mithra Desinguraj Sonal Gupta and Jerry Gao
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Edge-Based Mobile Smart Service System for Illegal Dumping
Edge-Based Trash TruckAnd Mobile Station
San Jose City
Smart City App
Illegal Dump App
Street Clean Monitor Car
Mobile Street Sweeper truck
Edge-Based Hot-SpotMobile Station
Smart Illegal Dumping Service System City Wireless
Network
Hot Spot
Illegal dumpingMobile Station
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping App
Smart Mobile Illegal Dumping Service System (SMIDS)
Illegal Dumping Service Server
Illegal Dumping Service Manager
Edge-Based Mobile Station
Illegal Dumping Service DB Program
Illegal Dumping Detection Engine
Illegal Dumping Service Connector
Illegal Dumping Mobile Client
IoT Mobile Platform amp Sensors
Computing Vision ampObject Detection
Illegal Dumping Controller
Illegal Dumping Reporter
Illegal Dumping Analytics
Illegal Dumping Service Protocol
Illegal Dumping Server UI
Illegal Dumping Service Protocol
Edge-Based Data Repository
Illegal Dumping Dashboard
Mobile Edge Computing Platform
QoS amp Security
QoS amp SecurityCrimeSafty
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Edge-Based Automatic City Illegal Dumping Object Detection
Group 1
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
AI Model and Technology
bull Convolutional Neural Network (CNN) is a practically useful algorithm especially in computer vision tasks by reduced complexity with resolving overfitting problems from deep learning
bull CNN inside is made of simple repeated matrix multiplications without branch operations
- Create illegally dumped material labeled dataset- Create lmdb database from the dataset- Tweak the existing model as per need and train the model using- Caffe interface with the generated lmdb files- Deploy the model file on the embedded platform Jetson TX1- Test the model for prediction accuracy
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Approaches
Approach 1 (Naive Approach)Frequently dumped wastes identified in 102 illegal dumping images provided by City of San Jose6 classesBaseline Network AlexNetIterations 100Image dataset size161 Mb
0
10
20
30
40
50
60
70
Chair Mattress Table Furniture Sofa Trash
Prediction Accuracy of Approach 1
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Approaches
Approach 2 Adding more classes based on Approach 1 11 classesBaseline Network AlexNet and GooglenetIterations 5000Image dataset size 802 Mb
0
20
40
60
80
100
120
Cart Chair Clean Area Electronics Furniture Mattress Sofa Table Trash Trash Bags Tree
GoogLeNet
AlexNet
The disadvantage of this approach was the model tends to predict clean area even for the image with illegal dump object because the feature map for clean areas was more compared to normal images
Prediction Accuracy of Approach 2
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Approaches
Approach 3 Using pre-processed imagesCropping image dataset for better training8 classesBaseline Network AlexNet and GooglenetIterations 10000Image dataset size 114 Gb
Solution improvement
bull Image pre-processing to clearly define the region of interest
bull Excluding Clean Area classification
bull Classes Aggregation
0
10
20
30
40
50
60
70
80
90
100
Cart Electronics Furniture Mattress Sofa Trash Trash Bags Tree
GoogLeNet()
AlexNet()
Prediction Accuracy of Approach 3
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Approaches
Approach 4 Energy Efficiency Approach- TensorRT
- We used NVIDIA TensorRT that helps shrink model size - Uses two approaches to shrink trained model size
1 Quantization2 Optimizations by applying vertical and horizontal layer fusion
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Approaches
2276 2277
414
2276
413
847
1138
12
1138
12
0
50
100
150
200
250
300
AlexNet AlexNet GoogleNet AlexNet GoogleNet
Approach 1 Approach 2 Approach 3
Original (in MB) With TensorRT (in MB)
Approaches NetworkOriginal (in MB)
With TensorRT(in MB)
Approach 1 AlexNet 2276 847
Approach 2 AlexNet 2277 1138
GoogleNet 414 12
Approach 3 AlexNet 2276 1138GoogleNet 413 12
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Detecting Engine (By Group 2)
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Engine ndash Case Study I
Conclusions bull Great to have localization but at the cost of classification accuracy bull Localization before classification localization capability determines the classification ability
Experimental design - Train Faster R-CNN using VGG from scratch in Caffe - Localization (represented by bounding boxes) and classifications (represented by a class label and probability)
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Engine ndash Case Study II
Conclusions bull Decent three-class classification accuracy bull Does not handle different viewpoints well possibly because the training set does not contain enough images from all viewpointsbull Does not handle multiple objects well possibly because the training set does not contain enough images with multiple objects
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 3 - Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Engine ndash Case Study III
Experimental design a Retrain the last layer of Inception v3 as a softmax in TensorFlowb Number of classes = 3
Conclusionsbull Overfitting is an issue because the final test accuracy is 100 bull Possible solutions to prevent overfitting are collect more images and apply regularization (L2 or dropout)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Engine ndash Case Study IV
Conclusion for case study 4 bull Good seven-class classification accuracy bull To improve the accuracy we could use more images and increase the intra-class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7- Images with slight duplicate images of different sizes (data augmentation)
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Engine ndash Case Study V
Conclusion for case study 5 - Better seven-class classification accuracy- To improve the accuracy we could use more images and increase the intra-
class variations of each category
Experimental design - Retrain the last layer of Inception v3 as a softmax in TensorFlow- Number of classes = 7
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Illegal Dumping Object Detection Engine ndash Case Study V
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
A Street Cleanliness Assessment System for Smart City using Machine Learning and Cloud
Objective
- Support real-time automatic digital data-driven analysis tracking and monitoring of city street cleanliness
- Provide automatic response management service solutions for city cleanliness
- Offer a real-time dashboard of city Mayor and public about city street environment and cleanliness based on digital cleanliness protocols and standards Digital Colored City Street Map
Digital Colored City Street Digital Colored City Street Block
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Street Cleaning Monitoring
System Architecture
Street Cleaning UI Street Cleaning Dashboard
Street Cleaning Reports
Streets BlocksMobile Stations
Street Cleaning Detection Engine
Street Cleaning Detection Analytics
Street Cleaning DB service
Engine DB (NoSQL) Application DB (MySQL)
Street Cleaning Service Protocols
Mobile StationConnection Module
ServiceRequestModule
Street Cleaning Service Manager
Admin Feedback Dispatch
DB Connection ControlModule
UI ConnectionModule
Mobile Client(MS)
Controller
MS Computing
MS Monitoring
MS RepoStreet Cleaning Security
MS Security
ACLAuthentication
EncryptionSession Mgmt
Role Based Authorization
Performance Alerts
Historical
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Highlights
Distributed hybrid deep learning based image processing pipeline
Deep Learning FrameworkModel Agnostic Phases
On Demand Scaling based on the volume of input images
Easily extend object clustersclasses to detect new objects
Embed localization and classification information inside image metadata (exif)
Real Time dashboard visualizing the cleanliness of the streets
Reduce the operational cost and Optimize resource allocationChallenges
bull Blurred Imagesbull Perspective of Photosbull Making the pipeline to respond in real
timebull Optimizing the models to run in
resource constrained environment foredge processing
Dependenciesbull High specification Server with GPU
support powerful processor and large memory capacity
bull Availability of Training Data
Contribution
Deep Learning-based framework for litter detection and classification
Capture street images using garbage truck mounted cameras
Send to image processing pipeline multiple phases where litter objects are detected andsegmented
Group images by geo-location to give a coherent view of the cleanliness of the street
Display results on interactive dashboard
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Pipeline Curb and Street Detection and
Localization
Obstacle Detection (Cars People)
Object Detection and Clusteringinto high level classes(Glass Metal Liquid)
Object Classification (Bottles Cans Leaves)
Result Integration
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Dashboard View
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
City Street View
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
NewsFeeds View
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Clean-up Service Requests View
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Manage Cleanup Crew View
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Notifications View
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
System Implementation
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Original Image
- Input by user
Phase 1
- Street Detection
(Deepmask)
Phase 2
- Object Detection
(Deepmask)
Phase 3
- Object Classification
(Tensor Flow)
Phase Outputs
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Image Annotation View
Annotation Tool ndash Edit ViewDashboard
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Image Pipeline View
Dashboard Detailed View
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
0 50 100 150 200 250
1
3
5
Phase Performance (In Seconds)
Phase 1 Phase 2 Phase 3
0
50
100
150Cleanliness Prediction (Clean Street Images)
Expected Actual Average
0
50
100
Dirty_1 Dirty_2 Dirty_3 Dirty_4 Dirty_5 Dirty_6 Dirty_7 Dirty_8
Cleanliness Prediction (Images with Little Litter)
Expected Actual
020406080
100
Cleanliness Prediction (Images with Little Litter)
Expected Actual Average
Phase wise performance of pipeline for images with varying level of cleanliness Red bar ndash Phase 1 Orange ndashPhase 2 and Green ndash Phase 3
Cleanliness score computed for 10 clean streets Correct classified instances 8 Accuracy = 810 = 80
Cleanliness score computed for 8 streets with little litter(dirty streets) Correct classified instances 5 Accuracy = 58 = 625
Cleanliness score computed for 10 streets with a lot of litter (very dirty streets)Correct classified instances 8 Accuracy = 810 = 80
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Future Research Directions Challenges and Needs for Smart Cities
A Silicon Valley Excellence Research Center Smart Technology Computing and Complex Systems (STCCS) ndash SmartTech Center
San Jose State University
Presented by Jerry Gao PhD Professor Director
SJSU and City of San Jose are teamed up as a task force for Smart Cities
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
How to Build Connected Smart Cities
Social networks
IT networks
Sensor NetworksCloudsWireless networks and mobile clouds
City Government
Smart Home
Community amp Neighborhood
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Intellectual Merit
o Tasks 1 Multi-modal Communication and Network Interoperabilityndash Multi-modal reliability
interoperabilityo Tasks 2 Software-defined Edge Cloud
ndash Remote control orchestration SDN policy engine
o Tasks 3 Mobile Enabled Resilient Quality Test and Automatic Validation ndash Seamless emergency testing
service large-scale automatic repeatable test QoS
Broader Impacts
o Task1 Cloud dashboard Help multiple departments to better manage the rescue and relief resources
o Task2 CommunitiesProvide a robust connectivity solution enable rapid and effective emergency response
Switch
Router
Switch
Basestation
Internet Backbone
Gateway Gateway
XLink
break XLink
break
Community Gateway
WiFi
Moving Gateway
WiFi
Legend
Link Break
Internet
Peer to Peer Link
X X
User Gateway
X
Internet Service is down
Community Network
Switch
WiFi
Internet Service is normal
X
NSF EAGER Grant - Creating a Community Infrastructure for Interoperable Emergency Connectivity (for City Community)
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Smart and Connected City Neighborhoods and Communities
City Hall and Departments
City Library
Community Center
City Neighborhoods
ProgramsOrganizationsServices)School
Hospitals
Businesses
People Different Groups
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Smart City Big Data
Transportation amp Traffic Big Data
Environment Big Data
City Cyber InformationInfrastructure
Smart City Big Data
City Emergency preparedness Big Data
Community Big Data
Networking amp Mobility Big Data
City IoT Big Data
City GovernmentBig Data
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
How to Build Sustainable Smart Cities
Sustainable Transportation
Green amp Sustainable Living Resource
Sustainable Infrastructure
Sustainable CyberInfrastructure
Sustainable EconomicBusiness
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Smart City Big Data Challenges
It has a big data junk yard
No well-defined service-orientedbig data platform for smart cities
Big Data Ownership
Smart City Big Data
Lack of well-defined Big Data Warehouseswith Quality AssuranceSolutions
Big Data Quality andCertification
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
City AI and Big Data Research Needs and Challenges
AI Dynamic Modeling
TransportationTraffic Behavior Modeling
Mobile ObjectModeling
Dynamic EnvironmentModeling
City Safety Modeling People CommunityBehavior Modeling
People DynamicBehavior Modeling
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Challenges and Needs in Building Smart City Complex Systems
Smart City Big Data
Issue 1 Too many isolated information islands flexible systematic information classification and integration
Issue 2 How to build sustainable smart city system infrastructures Systematic sustainable software-defined solution for IoT infrastructures network infrastructures cloud and mobile cloud infrastructures
Issue 3 Where are the service-oriented open platformsframework for smart city system development FiWare hellip
Issue 4 Where are the smart city big data service platform to support diverse smart city systems and solutions No found
Issue 5 Trustworthy smart city big data Smart city open big data public open data New big data businesses and production solutions for trustworthy smart city big data Well-defined quality assurance standards and quality validation service platforms
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
SJSU
IEEE Joint International Conferences on Smart World and Smart City in Silicon Valley
- IEEE Smart World Congress 2017 in Silicon ValleyURL httpieee-smartworldorg2017smartworld
IEEE Smart World Congress 2017 IEEE Smart City Innovation 2017
- IEEE Smart City Innovation 2017 August 2017- Conjunction with IEEE SmartWorld 2017 San Francisco Bay USAURL httpieee-smartworldorg2017sci
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
SJSU
Related References
Autonomous UAV Forced Graffiti Detection and Removal System Based on Machine LearningbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Practical Study on Quality Evaluation for Age Recognition Systems
bullAug 2017
bullSEKE2017 - The Twenty-Ninth International Conference on Software Engineering and Knowledge
Engineering
An Edge-Based Smart Mobile Service System for Illegal Dumping Detection and Monitoring
in San JosebullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
Data-driven Forest Fire analysisbullAug 2017
bull2017 IEEE International Conference on Smart City and Innovation
A Survey on Quality Assurance Techniques for Big Data ApplicationsbullApr 2017
bullIEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND
VALIDATION
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS
Related References
Big Data Validation Case StudyApr 2017IEEE BigDataService 2017 - International Workshop on QUALITY ASSURANCE AND VALIDATION
Data-Driven Water Quality Analysis and Prediction A SurveyApr 2017The 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND APPLICATIONS
Bike-Sharing System A Big-Data PerspectiveJan 2017 Smart Computing and Communication
Quality Assurance for Big Data Applicationsndash Issues Challenges and NeedsJul 2016The Twenty-Eighth International Conference on Software Engineering and Knowledge Engineering
On Building a Big Data Analysis System for California DroughtbullApr 2017
bullIEEE BigDataService 2017 San Francisco April 7-10 2017
Big Data Validation and Quality Assurance ndash Issuses Challenges and NeedsbullMar 2016
bullIEEE 10th IEEE International Symposium on Service-Oriented System Engineering
Data-Driven Water Quality Analysis and Prediction A Survey
bullApr 2017
bullThe 3rd IEEE International Conference on BIG DATA COMPUTING SERVICE AND
APPLICATIONS