Data analytics and mash-upData analytics and mash-up

Real time analytics of employment data

Team Shadowfax

05/03/23CMPE 272 - Class Project 1

AgendaAgenda

• What are data analytics and mash-up?• Architecture of web mash-up• Implementation• Implementation architecture• How is it going to be useful to the user ?

05/03/23CMPE 272 - Class Project 2

Data analytics and mashupData analytics and mashup

• Data Analytics is examining of raw data from which we can retrieve the information after work around with the data

• In the data analytics there is no assumption of data unlike data mining where little adjustments are made to draw conclusion

• Includes data cleaning, transformation and documentation• Data from different data sources are analyzed and converted into

information after which all the information from different sources are clubbed together into a single data source and presented to the user in a useful fashion.

05/03/23CMPE 272 - Class Project 3

Data/Web mash-upData/Web mash-up

• Data mashup is data integration of structured, web & unstructured information available freely on different sources

• More and more reusable structured data appears on the Web. Users want to extracts related data from various resources and combine in a human readable format, such as HTML, PDF, or text

• Challenges: Structured, unstructured data is available on multiple public resources in

various formats. There is a file format mismatch, schema mismatch, data representation mismatch. Hence first challenge is to extract this data into common data format.

05/03/23CMPE 272 - Class Project 4

Architecture of web mashupArchitecture of web mashup

• The basic idea is to extract, analyze, aggregate and visualize.

Divided into three layers:• Presentation: This is the user interface of mashups. The technologies used are JSP, HTML, and JavaScript • Information Extraction and

delivery: The functionality to access data through APIs (LinkedIn or any other APIs)is provided using API service • Data: Data is retrieved from various open data sources and is managed in a database

05/03/23CMPE 272 - Class Project 5

ImplementationImplementation

Idea• The basic concept – System facilitating user to search for labor stats and

related jobs

Data Scavenging• Exemplifying application – Need for data.

• Employment/Unemployment Datao Wdie range of data poolso Various granularity level of datao County specific data

• Zip code – County connectedness

• Job related data

05/03/23CMPE 272 - Class Project 6

ImplementationImplementation

Data Transformation & mashing•Connecting 2 databases•Simple Java Parser Implementation•Loading final structured data into MySql.

Information staging•Shaped up data presentation to the user.

o J2EEo MySQLo JS & REST API

05/03/23CMPE 272 - Class Project 7

Implementation architecture Implementation architecture

05/03/23CMPE 272 - Class Project 8

How is it going to be useful to the How is it going to be useful to the

user ?user ?

• The information is presented to the user using a web page and is available to the user on the web any time from any place

• The information available with Linked In requires the user to be a registered user with them to share the job listing information with the user

• User would be able to search for a job based on his preference of region ,climate and others

05/03/23CMPE 272 - Class Project 9

ReferencesReferences

• http://www.bls.gov• http://www.populardata.com• http://developer.linkedin.com• http://birming.jvmhost.net

05/03/23CMPE 272 - Class Project 10

Thank You!

Demo

05/03/23CMPE 272 - Class Project 11

Q&A

05/03/23CMPE 272 - Class Project 12