hobbit in a nutshell - edf2016
TRANSCRIPT
![Page 1: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/1.jpg)
HOBBITin a Nutshell
Axel Ngonga
Horizon 2020GA No 688227
01/12/2016–30/11/2018
Joint Event Post-EDF 2016Eindhoven, Netherlands
July 1st, 2016
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 1 / 17
![Page 2: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/2.jpg)
A Lot of Data
1
1http://www.ibmbigdatahub.com/infographic/four-vs-big-dataNgonga (InfAI) HOBBIT in a nutshell July 1st, 2016 2 / 17
![Page 3: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/3.jpg)
A Lot of Tools
2
2https://cdn.datafloq.com/cms/os_big_data_open_source_tools-v2.pngNgonga (InfAI) HOBBIT in a nutshell July 1st, 2016 3 / 17
![Page 4: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/4.jpg)
Core Questions
Developers: How good is my tool?Vendors: Who is my tool good for?Users: Which tool(s) should I use formy application?
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 4 / 17
![Page 5: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/5.jpg)
Many Questions
Where are the current bottlenecks?Which steps of the data lifecycle arecritical?Which solutions are available?Which key performance indicatorsare relevant?How well do or should toolsperform?How do existing solutions performw.r.t. relevant indicators?
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 5 / 17
![Page 6: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/6.jpg)
GERBIL
Evaluation platform for NER/NEL9 reference annotation systems11 reference datasetsBenchmarking 10× fasterArchiving of resultsCiteable URIsAdditional analysisOpen-source projectLocal deploymentNormalized implementation of KPIsOnline instance athttp://gerbil.aksw.org/
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 6 / 17
![Page 7: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/7.jpg)
GERBIL
Evaluation platform for NER/NEL9 reference annotation systems11 reference datasetsBenchmarking 10× fasterArchiving of resultsCiteable URIsAdditional analysisOpen-source projectLocal deploymentNormalized implementation of KPIsOnline instance athttp://gerbil.aksw.org/
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 6 / 17
![Page 8: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/8.jpg)
GERBIL
Annotator TasksNIF-based Annotators 2519Babelfy 958DBpedia Spotlight 922TagMe 2 811WAT 787Kea 763Wikipedia Miner 714NERD-ML 639Dexter 587AGDISTIS 443Entityclassifier.eu NER 410FOX 352Cetus 1
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 7 / 17
![Page 9: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/9.jpg)
HOBBIT
Rationale
A community-driven benchmarking framework for the community
Focus on Big Linked DataCover all steps of the Linked Data lifecycle
Used by a growing number of companiesMature and maturing technologies
Open benchmarks based on industrial dataand use cases
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 8 / 17
![Page 10: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/10.jpg)
HOBBIT
Rationale
A community-driven benchmarking framework for the community
Focus on Big Linked DataCover all steps of the Linked Data lifecycle
Used by a growing number of companiesMature and maturing technologies
Open benchmarks based on industrial dataand use cases
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 8 / 17
![Page 11: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/11.jpg)
HOBBIT
36-month projectProject begin: Dec. 1st, 2015Project volume: ca. 4 million Euros10 partners
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 9 / 17
![Page 12: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/12.jpg)
Aims
1 Gather real requirementsPerformance indicatorsPerformance thresholds
2 Provide universal benchmarking platformStandardized hardwareComparable results
3 Develop benchmarks based on real data4 Periodic benchmarking challenges5 Periodic reporting6 Found independent Hobbit association
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 10 / 17
![Page 13: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/13.jpg)
Overview
Data Collection
Industrydata
Measure Collection
Benchmark Creation
Benchmark 1
KPIsTasks
KPIsTasksKPIsTasks
KPIsTasks
KPIsTasks
KPIsTasks
Benchmark 2
Benchmark n
HOBBITPlatform
Solution 1
Solution k
Solution 2
Challenges
Reports
Participants/Community
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 11 / 17
![Page 14: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/14.jpg)
We offer a benchmarking platform
Controller
Data Generator
Task Generator
Data Generator
Data Generator
Task Generator
Task Generator
FrontendSystem Adapter
System
data flowcreates component
Store
SPARQL Endpoint
Analysis
BenchmarkEvaluator Module
Eval. Store
Message BusNode Observer
Logging
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 12 / 17
![Page 15: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/15.jpg)
We offer a benchmarking platform
Addresses all steps of the LinkedData LifecycleBenchmarks derived from industryuse casesReal data under the benchmarksScalable size of benchmarksOpen-source implementationLocal instance on server clusterUses established deploymenttechnologies
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 13 / 17
![Page 16: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/16.jpg)
We offer benchmarks
Streaming and static deterministic benchmarksRealistic benchmarksControlled volume and velocity
Generation and AcquisitionConversion of XML into RDFEntity recognition and linkingRelation extraction
Analysis and ProcessingLink DiscoveryMachine LearningSupervised and unsupervised
Storage and CurationTriple storesVersioningIncl. updates
Visualization and ServicesQuestion AnsweringFaceted BrowsingUsage-based benchmarks
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 14 / 17
![Page 17: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/17.jpg)
We offer datasets
Twitter7 datasetca. 476 million tweetsca. 17 million users
ClueWeb12ca. 733 million websites1+ billion annotations
Printing Machineryca. 6.5 trillion events1500 printing machines
LIVEDca. 2.5 billion measurements6 households, two years
Injection molding industryca. 120 million measurements
Traffic data archiveca. 15 trillion speed measurements100+ million road segments
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 15 / 17
![Page 18: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/18.jpg)
We need ...
Your use casesParticipate in the surveyJoin the HOBBIT communityProvide KPIsProvide datasetsJoin the platform development
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 16 / 17
![Page 19: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/19.jpg)
Thank You
http://project-hobbit.eu/get-involved/
http://goo.gl/forms/1iRIoG4Xpb
https://twitter.com/hobbit_project
Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 17 / 17
![Page 20: Hobbit in a Nutshell - EDF2016](https://reader034.vdocuments.us/reader034/viewer/2022042706/5879f1c91a28ab70298b4dad/html5/thumbnails/20.jpg)