03 december 2012 by abe lederman, ceo
DESCRIPTION
Deep Web Technologies Show and Tell Presentation to. 03 December 2012 By Abe Lederman, CEO. Abe Lederman. Deep Web Technologies was founded by Abe Lederman in 2002. BS & MS Degrees in Computer Science from MIT A co-founder of Verity, acquired by Autonomy (now HP) - PowerPoint PPT PresentationTRANSCRIPT
© 2012 Deep Web Technologies, Inc.
03 December 2012By Abe Lederman, CEO
Deep Web TechnologiesShow and Tell
Presentation to
© 2012 Deep Web Technologies, Inc. 2
Abe Lederman
Deep Web Technologies was founded by Abe Lederman in 2002.
–BS & MS Degrees in Computer Science from MIT
–A co-founder of Verity, acquired by Autonomy (now HP)
–Developed SciSearch@LANL (part of “Library without Walls”)
–25 years experience in Information Retrieval
© 2012 Deep Web Technologies, Inc. 3
About Deep Web Technologies...
• 20 person company based in Santa Fe, New Mexico
• Over $5M in DOE SBIR Grants (2003-2011)
• Pioneer/trailblazer in federated search
• 100+ solutions in production
© 2012 Deep Web Technologies, Inc.
Customers Include...
Government:• Defense Technical Info
Center (DTIC)• Office of Sci. & Tech. Info
(DOE-OSTI)• UN Economic Comm. for
Africa (UNECA)• European Space Agency
Corporate:• Boeing • BASF• Intel• HP• P&G
Academic:• Stanford University• George Mason
University• Texas Medical Center• University College of
Cork
Public Portals:• WorldWideScience.org• Science.gov• Biznar• Mednar• ScienceResearch.com
© 2012 Deep Web Technologies, Inc. 5
Develop 3 POC’s (Top 10 DB, 5 Catalogs, Digital Repositories
Launch xSearch for Science & Engineering (28 sources)
Expand xSearch to include Social Sciences & Humanities. Also, expanded later in the year for GSB sources (170 sources)
In November 2011 the Charleston Advisor Review was published
Upgrade and Expand xSearch to 200 sources in December 2012
History of Partnership
2007
2010
2011
2011
2012
© 2012 Deep Web Technologies, Inc.
2008 PR
© 2012 Deep Web Technologies, Inc.
© 2012 Deep Web Technologies, Inc. 8
© 2012 Deep Web Technologies, Inc. 9
© 2012 Deep Web Technologies, Inc. 10
Federated Search allows users to submit a real-time search in parallel to multiple information sources and retrieve aggregated, ranked and de-duplicated results.
What Is Federated Search?
© 2012 Deep Web Technologies, Inc. 11
In Other Words…One Search, Many Sources
Internal Sources
Blogs & Wikis
SubscriptionSources
Public Web Sources
Reports
News & Social Media
Begin Search
© 2012 Deep Web Technologies, Inc. 12
xSearch Status
• Upgraded early fall to v. 3.2.2• GSB linked to xSearch• 200 collections in application• 30 new connectors in acceptance
testing (roll-out imminent)
© 2012 Deep Web Technologies, Inc.
Janu
ary
Mar
chMay Ju
ly
Sept
embe
r
Novem
ber
0
1000
2000
3000
4000
5000
6000
7000
User Queries by Month/Year
2012 User Queries2011 User Queries2010 User Queries
© 2012 Deep Web Technologies, Inc.
Janu
ary
Febr
uary
Mar
chAp
ril
May
June
July
Augu
st
Sept
embe
r
Octob
er
Novem
ber
Decem
ber
0100,000200,000300,000400,000500,000600,000700,000
Source Queries
2012 Source Queries 2011 Source Queries2010 Source Queries
© 2012 Deep Web Technologies, Inc.
Web
of S
cien
ce
ABI/I
nfor
m G
loba
l
PubM
ed
Enviro
nmen
tal S
cien
ces & P
ollu
tion
Man
agem
ent
Engi
neer
ing
Villa
ge
Sociol
ogical
Abs
tract
s
Busine
ss S
ourc
e Com
plet
e
Scop
us
JSTO
R
ACS
Publ
icat
ions
Perio
dica
ls A
rchi
ve O
nlin
e
Proj
ect M
use
0
2000
4000
6000
Top click-throughs: Jan 1, 2012 – November 30, 2012
© 2012 Deep Web Technologies, Inc. 16
Explorit Release 3.2.3Starting customer upgrades
• Visual clusters• Full-text filters• Content type/Media
type• Integration with
Zotero, Mendeley
© 2012 Deep Web Technologies, Inc. 17
© 2012 Deep Web Technologies, Inc. 18
© 2012 Deep Web Technologies, Inc. 19
© 2012 Deep Web Technologies, Inc.
© 2012 Deep Web Technologies, Inc. 21
Explorit Release 4.0Coming mid-2013
• Dynamic tab searching• Thesaurus-based searching
(Do you also want to search for?)
• Personal library• Big Data mashups
(Enhanced content)• Faceted navigation
© 2012 Deep Web Technologies, Inc. 22
Big Data Mashups
© 2012 Deep Web Technologies, Inc. 23
Related Content
• Major Science portals • Science News • Patent Databases • Scholar Networks• Subscription Sources• Public Databases• Open Access Journals
© 2012 Deep Web Technologies, Inc.
Article Grouping
© 2012 Deep Web Technologies, Inc.
Non-Invasive imaging
© 2012 Deep Web Technologies, Inc.
Linking Open Data Cloud Diagram by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/
© 2012 Deep Web Technologies, Inc.
Nature has 297 million triples.
© 2012 Deep Web Technologies, Inc. 28
Journey to 10,000 sources
© 2012 Deep Web Technologies, Inc. 29
Scalability Challenges
• Source selection • Ranking and organizing of results • Traffic management • System load management • Finding, building, and maintaining
connectors
© 2012 Deep Web Technologies, Inc. 30
Scalability - Divide and Conquer
ScienceResearch.com
WorldWideScience.org
Other Federated Search Engines
ScienceAccelerator
Science.gov
© 2012 Deep Web Technologies, Inc. 31
© 2012 Deep Web Technologies, Inc. 32
© 2012 Deep Web Technologies, Inc. 33
Multilingual WorldWideScience.org
© 2012 Deep Web Technologies, Inc. 34
How Multilingual Federated Search Works
Ranked resultstranslated by Microsoft to user’s language
Results returned to user
EXPLORIT
Microsoft Translator
German
Chinese
Russian
Queryin user’s language
Ranked resultsin user’s language
Queryto be translatedfor each source
Queryin source’slanguage
Foreign language
search engines
Resultsin source’slanguage
Ranking
© 2012 Deep Web Technologies, Inc.
© 2012 Deep Web Technologies, Inc.
Translated
Original
© 2012 Deep Web Technologies, Inc.
© 2012 Deep Web Technologies, Inc.
© 2012 Deep Web Technologies, Inc.
© 2012 Deep Web Technologies, Inc. 40
© 2012 Deep Web Technologies, Inc. 41
ESN – x2
© 2012 Deep Web Technologies, Inc. 42
© 2012 Deep Web Technologies, Inc. 43
© 2012 Deep Web Technologies, Inc. 44
UNECA ASKIA Portal (United Nations – Access Scientific Knowledge in Africa)
© 2012 Deep Web Technologies, Inc. 45
© 2012 Deep Web Technologies, Inc. 46
© 2012 Deep Web Technologies, Inc. 47
© 2012 Deep Web Technologies, Inc.
© 2012 Deep Web Technologies, Inc.
© 2012 Deep Web Technologies, Inc. 50
BASF
© 2012 Deep Web Technologies, Inc. 51
© 2012 Deep Web Technologies, Inc. 52
© 2012 Deep Web Technologies, Inc. 53
© 2012 Deep Web Technologies, Inc. 54
© 2012 Deep Web Technologies, Inc. 55
© 2012 Deep Web Technologies, Inc. 56
Find It!TMC’s Link Resolver
© 2012 Deep Web Technologies, Inc. 57
© 2012 Deep Web Technologies, Inc. 58
© 2012 Deep Web Technologies, Inc. 59
© 2012 Deep Web Technologies, Inc. 60
© 2012 Deep Web Technologies, Inc.
© 2012 Deep Web Technologies, Inc. 62
© 2012 Deep Web Technologies, Inc. 63
Abe’s Stanford ProjectsWISHLIST
• Assist SULAIR in integrating Explorit preview via Web Services into new library portal.
• Integration with SearchWorks
• Develop a stand-alone portal focused on Stanford core-competency (Energy, Environment, …)
© 2012 Deep Web Technologies, Inc. 64
Abe’s Stanford ProjectsWISHLIST (cont.)
• Develop Chinese Explorit in collaboration with Library of Congress and/other library
• xSearch / Explorit for Stanford Medical School
• Mash up Big Data (Link Data, Mendeley, citations) and articles
• Data Portal (expansion of Data searching in WorldWideScience.org)
• Integration with Sakai or other CMS
© 2012 Deep Web Technologies, Inc. 65
Explore our Applications
• xSearch• WorldWideScience.org• Science.gov• Ciencia.Science.gov• DTIC Multisearch
© 2012 Deep Web Technologies, Inc. 66
Thank you!
View this presentation online