Download - Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search
![Page 1: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/1.jpg)
Benchmarking the Effectiveness of Associating
Chains of Links for Exploratory Semantic Search
Laurens De Vocht Selver Softic, Ruben Verborgh, Erik Mannens, Martin Ebner, Rik Van de Walle
![Page 2: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/2.jpg)
:Paris
?
2
![Page 3: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/3.jpg)
:Paris :Anne_Hidalgo
:mayor
3
![Page 4: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/4.jpg)
:Paris :Anne_Hidalgo
:mayor
:Bethlehem,_PA
?
4
![Page 5: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/5.jpg)
:Anne_Hidalgo
?
5
![Page 6: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/6.jpg)
:Anne_Hidalgo
?
:Bethlehem,_PA
![Page 7: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/7.jpg)
:Anne_Hidalgo
:Bethlehem,_PA
Exploratory Semantic Search Engine
7
![Page 8: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/8.jpg)
?
8
![Page 9: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/9.jpg)
:Paris
9 :mayor :Anne_Hidalgo
< :birthPlace :San_Fernando,_Caldiz
9 :country :Spain
< :birthPlace :Edward_Ferrero
9 :battle :Battle_of_Roanoke_Island
< :battle :Charles_Adam_Heckman
9 :birthPlace :Easton,_Pennsylvania
9 :mouthMountain :Lehigh_River
9 :city
:Bethlehem,_Pennsylvania
A
9
![Page 10: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/10.jpg)
:Paris
< :capital :France
< :citizenship :Cyril_Bourlon_de_Rouvre
9 :education :Aerospace_engineering
< :occupation :Dick_Johnson_(glider_pilot)
9 :almaMater :Mississippi_State_University
< :almaMater :Clara_Southmayd_Ludlow
9 :birthPlace :Easton,_Pennsylvania
< :mouthMountain :Lehigh_River
9 :city
:Bethlehem,_Pennsylvania
B
10
![Page 11: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/11.jpg)
B A
?
![Page 12: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/12.jpg)
How effective does an exploratory semantic
search engine reveal initially hidden associations,
as chains of links between interlinked resources?
![Page 13: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/13.jpg)
Introduction
Exploratory Search
Benchmark Model
Motivating Example
Discussion and Conclusion
![Page 14: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/14.jpg)
Introduction
Exploratory Search
Benchmark Model
Motivating Example
Discussion and Conclusion
![Page 15: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/15.jpg)
[EXPLORATORY SEARCH: FROM FINDING TO UNDERSTANDING, Machionini, 2006]
Lookup Learn Investigation
Exploratory Search
`Learning searches involve multiple iterations and return sets of
objects that require cognitive processing and interpretation’
`Searches that support investigation involve multiple iterations that take place
over perhaps very long periods of time and may return results that are critically
assessed before being integrated into personal and professional knowledge bases’
Definition
15
![Page 16: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/16.jpg)
1. Lookup
2. Relate/Expand
Lookup and learn: interpretation
16
![Page 17: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/17.jpg)
lookup
expand relate
An exploratory semantic search engine
17
![Page 18: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/18.jpg)
lookup
:Paris
Paris
18
![Page 19: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/19.jpg)
expand
:Paris
:Paris
19
![Page 20: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/20.jpg)
relate
20
![Page 21: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/21.jpg)
Introduction
Exploratory Search
Benchmark Model
Motivating Example
Discussion and Conclusion
![Page 22: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/22.jpg)
Iterative Exploratory Queries
Exploratory Semantic Search Engine
Datasets Baseline
Effectiveness
22
![Page 23: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/23.jpg)
Effectiveness
The effectiveness E indicates the overall perception of the results by
the users taking into account expert-user feedback.
# user marked relevant objects
E =
# retrieved objects
Note:
E can be interpreted as precision in traditional IR.
Typical IR examine both precision and recall.
23
[TALKEXPLORER, Verbert et al., 2013]
![Page 24: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/24.jpg)
Introduction
Exploratory Search
Benchmark Model
Motivating Example
Discussion and Conclusion
![Page 25: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/25.jpg)
Motivating Example
ResXplorer.org
Everything Is Connected Engine
Virtuoso
User Study Extracted Queries
25
![Page 26: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/26.jpg)
User Study Extracted Queries
1. lookup; 2. expand; 3. relate
26
![Page 27: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/27.jpg)
LDOW
P(0)
P(1)
P(2)
P(3)
Effectiveness : Interpretation
27
![Page 28: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/28.jpg)
Sample Results
Everything Is Connected Engine
28
![Page 29: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/29.jpg)
Sample Results
Virtuoso
29
![Page 30: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/30.jpg)
Introduction
Exploratory Search
Benchmark Model
Motivating Example
Discussion and Conclusion
![Page 31: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/31.jpg)
Limitations
Only indicate comparisons to baseline within the same use case.
Not possible to use the benchmark as a leverage to compare different
approaches across use cases
Could better demonstrate in which aspects an exploratory approach
excels traditional systems.
31
![Page 32: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/32.jpg)
Future Work
Put the results in perspective by indicating the nuances among different
expert user ratings.
Especially when there is expert disagreement or inconsistencies.
Facilitate generalization of the preliminary search context,
so results for engines can be reusable across datasets: avoiding that a
certain engine’s results differ strongly when changing the data and queries.
Make sure that the approach is generic and can be applied to other search
contexts with different data and use cases.
32
![Page 33: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/33.jpg)
Benefits
Compare exploratory search engines to a baseline:
show use cases when the baseline can be outperformed;
for which queries the ‘engine under test’ is relatively more effective.
Sensitive to initial query keywords as inputted by the user:
when there are inconsistencies or vague terms,
even mismatches in the query context, or when expert users disagreed.
33
![Page 34: Benchmarking the Effectiveness of Associating Chains of Links for Exploratory Semantic Search](https://reader033.vdocuments.us/reader033/viewer/2022052915/58a4dc611a28ab34318b5f11/html5/thumbnails/34.jpg)
Contact
@laurens_d_v
http://slideshare.net/laurensdv
http://semweb.mmlab.be/