Data analytics and mash-upData analytics and mash-up
Real time analytics of employment data
Team Shadowfax
05/03/23CMPE 272 - Class Project 1
AgendaAgenda
• What are data analytics and mash-up?• Architecture of web mash-up• Implementation• Implementation architecture• How is it going to be useful to the user ?
05/03/23CMPE 272 - Class Project 2
Data analytics and mashupData analytics and mashup
• Data Analytics is examining of raw data from which we can retrieve the information after work around with the data
• In the data analytics there is no assumption of data unlike data mining where little adjustments are made to draw conclusion
• Includes data cleaning, transformation and documentation• Data from different data sources are analyzed and converted into
information after which all the information from different sources are clubbed together into a single data source and presented to the user in a useful fashion.
05/03/23CMPE 272 - Class Project 3
Data/Web mash-upData/Web mash-up
• Data mashup is data integration of structured, web & unstructured information available freely on different sources
• More and more reusable structured data appears on the Web. Users want to extracts related data from various resources and combine in a human readable format, such as HTML, PDF, or text
• Challenges: Structured, unstructured data is available on multiple public resources in
various formats. There is a file format mismatch, schema mismatch, data representation mismatch. Hence first challenge is to extract this data into common data format.
05/03/23CMPE 272 - Class Project 4
Architecture of web mashupArchitecture of web mashup
• The basic idea is to extract, analyze, aggregate and visualize.
Divided into three layers:• Presentation: This is the user interface of mashups. The technologies used are JSP, HTML, and JavaScript • Information Extraction and
delivery: The functionality to access data through APIs (LinkedIn or any other APIs)is provided using API service • Data: Data is retrieved from various open data sources and is managed in a database
05/03/23CMPE 272 - Class Project 5
ImplementationImplementation
Idea• The basic concept – System facilitating user to search for labor stats and
related jobs
Data Scavenging• Exemplifying application – Need for data.
• Employment/Unemployment Datao Wdie range of data poolso Various granularity level of datao County specific data
• Zip code – County connectedness
• Job related data
05/03/23CMPE 272 - Class Project 6
ImplementationImplementation
Data Transformation & mashing•Connecting 2 databases•Simple Java Parser Implementation•Loading final structured data into MySql.
Information staging•Shaped up data presentation to the user.
o J2EEo MySQLo JS & REST API
05/03/23CMPE 272 - Class Project 7
Implementation architecture Implementation architecture
05/03/23CMPE 272 - Class Project 8
How is it going to be useful to the How is it going to be useful to the
user ?user ?
• The information is presented to the user using a web page and is available to the user on the web any time from any place
• The information available with Linked In requires the user to be a registered user with them to share the job listing information with the user
• User would be able to search for a job based on his preference of region ,climate and others
05/03/23CMPE 272 - Class Project 9
ReferencesReferences
• http://www.bls.gov• http://www.populardata.com• http://developer.linkedin.com• http://birming.jvmhost.net
05/03/23CMPE 272 - Class Project 10
Thank You!
Demo
05/03/23CMPE 272 - Class Project 11
Q&A
05/03/23CMPE 272 - Class Project 12