freebase - semantic technologies 2010 code camp

FreebaseA socially managed semantic database

Jamie TaylorSemTech 2010 Data Camp

Freebase has Many Types of Things

12 Million Topics

A Multiplicity Strong Identifiers

http://www.ellerdale.com/topics/view/0080-6ba0

http://rdf.freebase.com/ns/en.berlin_wall

http://musicbrainz.org/artist/7f347782-eb14-40c3-98e2-17b6e1bfe56c

http://rdf.freebase.com/ns/authority.musicbrainz.7f347782-eb14-40c3-98e2-17b6e1bfe56c

http://www.bbc.co.uk/music/artists/7f347782-eb14-40c3-98e2-17b6e1bfe56c

education

nationality

contained-by

education

member-of

eventalbums

label

contained-by

contains

member-of

400 MillionRelations

What’s in Freebase?

http://www.myspace.com/shakira

http://www.facebook.com/shakira

http://twitter.com/shakira

http://www.daylife.com/topic/Shakira

http://www.bestbuy.com/site/She+Wolf…

http://www.guardian.co.uk/music/shakira

http://www.last.fm/music/Shakira

http://www.netflix.com/RoleDisplay/Shakira/20046629

99% pure

All data undergoes rigorous QA before load

Major focus is reconciliation

Use sampling to assure 99% accuracy

Data that does not meet 99% accuracy is not loaded

What's been built on Freebase?

Up to 100,000 Queries a Day

Quarterly dumps of graphhttp://download.freebase.com

Users extend the data model

Users contribute data

The Freebase Commons·American football ·Internet·Anime/Manga ·Language·Architecture ·Law·Astronomy ·Library·Automotive ·Location·Aviation ·Martial Arts·Awards ·Measurement Unit·Baseball ·Media Common·Basketball ·Medicine·Bicycles ·Metaweb Types·Biology ·Meteorology·Boats ·Military·Broadcast ·Music·Business ·Olympics·Celebrities ·Opera·Chemistry ·Organization·Comics ·People·Common ·Geography·Computers ·Projects·Conferences ·Protected Places·Cricket ·Publishing·Data World ·Radio·Digicams ·Rail·Education ·Religion·Engineering ·Royalty·Event ·Soccer·Clothing and Textiles ·Spaceflight·Fictional Universes ·Sports·Film ·Symbols·Food & Drink ·Tennis·Freebase ·Theater·Games ·Time·Geology ·Transportation·Government ·Travel·Hobbies and Interests ·TV·Ice Hockey ·Video Games·Influence ·Visual Art

Top-level domains

schema = vocabulary

The Scope of Schema

10,448 Properties

describing

4,936 Types*

organized into

641 Domains

(77 Commons)

*types with 10 or more instances

Type Instances

1

10

100

1,000

10,000

100,000

1,000,000

10,000,000

100,000,000

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000

Rank

Inst

an

ces

Strength through Exemplars

>10 instances,4936 types

1424 Commons

MQL

[{ "name" : null, "type" : "/film/film"}]

Metaweb Query Language

[{ "name" : null, "type" : "/film/film",

"directed_by":{"id":"/en/george_lucas"}, "starring":[{

"actor":{"id":"/en/harrison_ford"}}]

}]

MQL

[{ "name" : null, "type" : "/film/film",

"directed_by":{"id":"/en/george_lucas"}, "starring": [{ "actor": { "name": null, "film": [{ "film": {"id": "/en/the_great_escape"} }] } }]}]

Donald PleasenceTHX 1138

Freebase Suggest

{ "/type/object/name":"Blade Runner", "/type/object/type":"/film/film", "/film/film/starring/actor":["Harrison Ford", "Rutger Hauer"], "/film/film/director":"Ridley Scott", "/film/film/release_date_s":"1981" }

Reconciliation

[{ "id":"/guid/9202a8c04000641f8000000000009e89", "name":["Blade Runner", "Bladerunner"], "score":1.4320519, "match":true, "type":["/common/topic", "/film/film","/media_common/adapted_work", "/award/award_winning_work", ]}, { "id":"/guid/9202a8c04000641f80000000002643d0", "name":["Blade"], "score":0.48852453, "match":false, "type":["/common/topic", "/film/film", "/award/award_winning_work", "/award/award_nominated_work", ]}

http://data.labs.freebase.com/recon/

Topic Blocks

Topic API

http://www.freebase.com/experimental/topic/standard?id=/en/ncis

Shortcut to building Topic displays

Two forms:

basic (names, types, description)

standard (basic + keys, properties)

Geo Search API

http://www.freebase.com/docs/geosearch

Semantic Spatial Semantic

Gridworks

Acre Development Environment

Getting Started++• Freebase Documentation Hub

• http://www.freebase.com/docs

• Developer Mailing List• http://lists.freebase.com/mailman/listinfo/freebase-discuss

• http://freebase.markmail.org

• Real Time help on IRC• Freenode #freebase

• Freebase Happenings• http://blog.freebase.com

• About the Graph Store• Google: "ACM SIGMOD schema last tuple store"