An Exploratory Search for Presentation Contentsbased on Slide Semantic Structure

Yuanyuan WangKyoto Sangyo University, Japan

circl.wang@gmail.com

Yukiko KawaiKyoto Sangyo University, Japan

kawai@cc.kyoto-su.ac.jp

Kazutoshi SumiyaUniversity of Hyogo, Japansumiya@shse.u-hyogo.ac.jp

Abstract—MOOC is a crucial platform for improving ed-ucation; students are able to browse various presentationcontents through the Web. Any single presentation contentcan only cover a small fraction of knowledge in a specificdomain by a given query, and thus offers a limited depthof information. Students then have to go through seriesof presentation contents, but this would be time-consumingand difficult to explore relevant information from variouspresentation contents. Therefore, we aim to build a novelexploratory search tool based on a meaningfully structuredpresentation, called “iPoster.” The system places elements suchas text and graphics of slides in a structural layout with azooming user interface (ZUI) by semantically analyzing theslide structure. Through this, iPoster can support studentsinteractively browsing slides, for retrieving and navigatinginformation from other presentation contents by consideringthe students’ browsing behavior. In this paper, we discusstwo types of exploratory search, (1) focused searching basedon well-matched browsing behavior that enables users obtaindetails of specific topics; and (2) exploratory browsing basedon partially-matched browsing behavior that enables the usersfind various relevant information on topics of interest.

Keywords-exploratory search; presentation contents; iPoster;

I. I NTRODUCTION

Slide-based presentation tools, such as Microsoft Power-Point or Apple Keynote is now one of the most frequentlyused tools for educational purposes. A huge amount of slide-based educational materials for MOOC, are freely sharedon Web sites such as Coursera1 and SlideShare2. Thus, notonly students who missed a lecture or presentation, but alsoanyone interested in a topic can study the presentation ontheir own. Therefore, techniques are in demand that willefficiently find appropriate information worth learning fromthe vast numbers of presentations available. Although manytechniques for searching and recommending presentationslides have been proposed, some problems remain from theviewpoint of exploratory search. One problem is currentslideshow mode of presentations does not allow users op-erate freely on the presentations for stimulating the users’interests. Recently, Prezi3 utilizes a ZUI as an alternative

1https://www.coursera.org/2http://www.slideshare.net/3http://prezi.com/

Figure1. Conceptual diagram of an exploratory search tool by iPoster

to the traditional slides, allowing users easily operate thepresentations. Another problem is any single educationalmaterial only cover a small fraction of knowledge in aspecific domain by a given search query, and thus offersa limited depth of information. The users then have togo through various presentation contents, but this will betime-consuming and difficult to find relevant informationfrom multiple presentation contents. Therefore, users willbe required to browse them in structural layouts with ZUIs,and easily obtain information meets the users’ specific needsby considering user browsing behavior.

As depicted in Figure 1, we present an exploratory searchtool that generates a meaningfully structured presentationby using the slides, which is called an iPoster. With ourexploratory search tool, (1) users can interactively browsean iPoster, therefore, (2) the iPoster can explore informationfrom other presentations by considering user browsing be-havior; and (3) represent and navigate information meets theusers’ specific needs. To achieve our goal, the iPoster can beimplemented by 1) extracting textual and graphic elementsin slides and semantic relationships between them; and 2)organizing elements in structural layouts with zooming andpanning transitions based on a idea of Prezi. In semanticstructure analysis, we first extract elements by examiningthe presentation context of the particular elements in the

665

slides. The semantic relationships between these elementsare determined using implicit hyperlinks in slides, based onslide structures. Specifically, we derive the slide structureby focusing on the itemized sentences in the slide text. Forproviding an overview of the content, we utilize a hierar-chical structure, combined with a stacked Venn. Finally, ouriPoster is generated in a structural layout based on semanticrelationships, using a ZUI, which can enable users to explorethe presentations easily and efficiently.

The next section reviews related work. Section 3 describesour semantic structure analysis model. Section 4 explainsiPoster generation. Our exploratory search on presentationsand conclusions are given in Sections 5 and 6, respectively.

II. RELATED WORK

A variety of applications address the weaknesses of thecurrent slideware tools in the presentation and authoringdomains. Our approach in an iPoster builds on the strength ofexploratory search. Lanir et al. [1] proposed a MultiPresenterapplication that leverages spatial reasoning capabilities torelate content through dual-screen projection. Although theiPoster does not adopt the dual-audience-display paradigm, itaddresses the need to navigate through elements dynamicallyduring the presentation. NextSlidePlease [2] creates anddelivers presentations in a nonlinear fashion. The iPoster issimilar to this work, as we utilize a structural layout withthe ZUI, rather than one or more slide lists, to allow usersinteractively browsing and automatically navigating.

Exploratory is constantly being changed and shaped bya range of related research. White et al. [3] suggests thatbrowsing is a cognitive and behavioral expression of ex-ploratory behavior and she claims that it has four elements:(1) glimpse a scene; (2) target an element of a scene visuallyand/or physically; (3) examine items of interest; and (4)physically or conceptually acquire or abandon examineditems. Therefore, our method according to this, offer anoverview (glimpses), the ability to operate the contentthrough various presentations (exploratory browsing). Detloret al. [4] developed a model of information seeking that com-bines both browsing and searching. It suggests that muchof Ellis’s model [5] is already implemented by componentscurrently available in Web browsers. We then applied thismodel to search presentations by considering user browsingbehavior.

III. SEMANTIC STRUCTUREANALYSIS

A. Element Extraction

There are two elements, i.e., textual elements and graphicelements, from presentation slides based on itemized sen-tences based on the XML files of slides. We define the slidetitle is the 1st level, the first item of text within the slide bodyis the 2nd level, and the depth of the sub-items increases withindentation level (3rd level, 4th level, etc.).

1) Textual Elements:We define textual elements as topicsthat focus on noun phrases in slides. Based on the presen-tation context, a topic that frequently appears at the higherlevels (i.e., slide title) in neighboring slides. The topics thatappear in the title of a slide and the body of other slidescan be considered to indicate its context in a presentation.Then, we extract topics by locating the same noun phrases indifferent slides, at varied levels. If a noun phrasek appearsat different levels in slidessi andsj , thenk is a candidatefor being one of the topicsT in the presentation.

T = {(k, si, sj)|lmax(k, si) ̸= lmax(k, sj)} (1)

where,T is a bag of noun phrases that can be considered ascandidates for topics.lmax(k, si) returns the highest level ofk in si. For instance, when the highest level is the title, i.e.,the 1st level, thenlmax(k, si) is 1; and when the highestlevel is the 3rd level, thenlmax(k, si) is 3. Whenk appearsat different levels,k is determined as a candidate for topicsby Eq. (1). Then, the weight ofk in T is defined using thelevels ofk, and the distance betweensi andsj , as follows:

I(k) =1

lmax(k, si)+

∑k,si,sj∈T

(1

lmax(k, sj)· 1

dist(si, sj)

)(2)

wheredist(si, sj) corresponds to the strength of the associ-ation betweensi andsj , and it denotes the distance betweensi andsj . If k appears at a high level insi andsj , and thedistance betweensi andsj is short,I(k) of k is high.

2) Graphic Elements:When compared to pure textualelements, images are more attractive, appealing and in-formative from a psychological standpoint. Therefore, wedefine graphic elements as images corresponding to the topiccandidates in slides, given that the surrounding text of theimages are similar to the topic candidates. We consideredthat the images used to describe the content in slides, anda slide title can be a subject of the content. When thesimilarity exceeds a predefined threshold by calculating theSimpson similarity coefficient, the images are recognized asthe corresponding images of the topic candidates.

B. Determination of Semantic Relationships

Semantic relationships between elements are determinedfrom a document tree of a presentation to enable users obtainrelevant information between the key elements. Preliminaryideas are given in an algebraic query model [6] as well.

1) Basic Definitions and Algebra:The presentation con-tent shown in Figure 2 is represented as a rooted orderedtreeD = (N,E) with a set of nodesN and a set of edgesE ⊆ N × N . There exists a distinguished root node fromwhich the rest of the nodes can be reached by traversing theedges inE. Each noden, except the root, has a unique parentnode, it of the document tree is associated with a logicalcomponent, i.e.,< title > or < sections >, based onan XML file in the given presentation. Functionwords(n)

666

Figure2. An example of pairwise fragment join of a document tree

returns the noun phrases of the corresponding component inn. A partial tree ofD with a given noun phrase as its rootis defined as a fragmentf . It can be denoted asf ⊆ D. Aslide is a fragment by the slide title.< n1, n2, n3 > is theset of nodes in slide2 and a fragment of the document tree.

To formally define the semantic relationships between theextracted elements, we first define operations on fragments,and sets of fragments using a pairwise fragment join [6].Let Fx and Fy be two sets of fragments inD, then, thepairwise fragment join ofFx andFy, denoted asFx ▷◁ Fy,is defined to extract a set of fragments. This set is yieldedby computing the fragment join of every combination of anelement inFx and an element inFy, in pairs, as follows:

Fx ▷◁ Fy = {fx ▷◁ fy | fx ∈ Fx, fy ∈ Fy} (3)

An example of operation for pairwise fragment join. It refersthe sample document tree in Figure 2. For the given twotopicsx = nutrition andy = fruit, whereFx = {< n3 >,< n5 >}, Fy = {< n4, n5, n6, n7 >, < n19 >}, Fx ▷◁ Fy

produces a set of fragments{< n3 >▷◁< n4, n5, n6, n7 >,< n5 >▷◁< n4, n5, n6, n7 >, < n3 >▷◁< n19 >,< n5 >▷◁< n19 > by Eq. (3).

2) Semantic Filters:We determine semantic relationshipsbetween the given topics,x andy, using the set of fragmentsproduced by taking pairwise fragment join as semanticfilters. For this, we define four types of semantic filters byconsidering the horizontal and vertical relevance, as well asthe structural semantics from the document tree.

Table ISEMANTIC RELATIONSHIPS WITH SEMANTIC FILTERS

Types Horizontal Vertical Hierarchy Inclusion

x shows y < α either l(x) < l(y) eitherx shows y ≥ α < β l(x) < l(y) either

x describes y < α either l(x) > l(y) eitherx describes y ≥ α < β l(x) > l(y) eitherx likewise y < α either l(x) = l(y) eitherx likewise y ≥ α < β l(x) = l(y) eitherx has-a y < α either either fx ⊇ fyx has-a y ≥ α < β either fx ⊇ fyx part-of y < α either either fx ⊆ fyx part-of y ≥ α < β either fx ⊆ fy

• Horizontal distance: supposing,hdist(ti, tj) denotesdistance between the nodes of the slide titlesti and tjin slides containingx andy, we set the threshold valueα at |N |/2, i.e., half the total number of nodes in thedocument tree. Ifhdist(ti, tj) ≤ α, the distance be-tween slides containingx andy is short (i.e., relevant).

• Vertical distance: when distance between the slidescontainingx andy is long, and they are at the low levelsin slides, they can be less relevant in the document tree.Supposing,vdist(r, q) denotes the distance between theroot noder and the node containingx or y, we set thethreshold valueβ at ave(depth), which is an averageof the depth of levels. Ifvdist(r, q) ≤ β, the level ofthe node containingx or y is high (i.e., relevant).

• Hierarchy: we compare the levels ofx and y in thefragments based on the theory of hierarchical seman-tics. Whenl(x) < l(y), the level ofx is higher thanthe level of y; x is a superordinate concept ofy (yis a subordinate concept ofx). Contrarily, l(x) > l(y)denotes that the level ofx is lower than the level ofy;x is a subordinate concept ofy (y is a superordinateconcept ofx). Whenl(x) = l(y), the level ofx is sameas the level ofy; they have coordinate concept.

• Inclusion: whenfx ⊆ fy, the fragment ofx is includedin the fragment ofy, i.e., fx is a partial tree offy.

3) Semantic Relationship Types:We determine five typesof semantic relationships between the given noun phrases,xand y, by combining the semantic filters of Table I. Formeasuring the relevance betweenx and y, we focus onthe horizontal distance and thevertical distance. Here,when thehorizontal distance between them is long, thevertical distanceshould be short. We determine hierarchicalrelationships,show, describe, and likewise, by focusingon hierarchy. In x shows y, l(x) < l(y) meansx is asuperordinate concept ofy. In x describes y, l(x) > l(y)meansx is a subordinate concept ofy. In x likewise y,l(x) = l(y) meansx and y have coordinate concept witheach other. We determine inclusion relationships, which arehas-a and part-of , by focusing oninclusion. In x has-ay, fx ⊇ fy means that the concept ofx includes the conceptof y. In x part-of y, fx ⊆ fy means that the concept

667

of x is included in the concept ofy. When x and y failto match these determinations of semantic relationships,xand y are independent. Therefore, a numbers of semanticrelationships betweenx and y are formed from a set offragments produced by taking the pairwise fragment join; asemantic relationship is determined by majority.

We conduct multiple presentations based on this semanticstructure analysis, the semantic relationships follow a tran-sitivity law, e.g., iff x shows y in presentationA, y showsz in presentationB, then it is assumed thatx shows z.

IV. IPOSTERGENERATION

We generate an iPoster possessing two features: (1) anoverview of elements from the slides, retaining this featureof traditional posters; and (2) a ZUI, promoting user inter-action and reflecting the semantics of the elements.

A. Determination of Element Layouts

When hierarchical relationships between two elements,reveal a hierarchy applied as to a hierarchical structure.Show or describe maps a parent-child relationship, ifxshows y (y describes x), then we markx in a parentarea andy in a child area, suggesting that the layer ofxis higher than the layer ofy. Moreover, likewise mapsa sibling relationship, ifx likewise y, then we locatexand y in the same layer. Inclusion relationships betweentwo elements, reveal a logical relationship of inclusion andexclusion applied as to a stacked Venn. Ifx has-a y (ypart-of x), we conceive an area ofy that is included in anarea ofx, and that the area ofx is larger than the area ofy.

B. Determination of Element Transitions

To utilize a ZUI, (1) users can browse the iPosters withtheir operations, such as zoom-in, zoom-out, and pan; (2)users can browse the iPosters without their operations byautomatically navigations with transitions between elements.The transitions discussed here explain the kinds of visualeffects that are applied to the semantic relationship types.

When show (describe) between two elements. Then,firstly the view must be zoomed-out from the focusedelement to an area of both, following which; it must bezoomed-in to the target element. Therefore, the transitionsinclude passing through the area of both, which helps usersto easily grasp the super-sub relation existing between them.

Whenlikewise between two elements, the transitions be-tween them include zooming-out from the focused elementto an area enclosing both the elements and their parentelement, and then zooming-in to the target element. Then,the transitions provide their parent element helps users toeasily know they are subservient to the same concept.

Whenhas-a (part-of ) exists between two elements, thetransition between the two elements pans from the focusedelement to the target element. Therefore, this simple anddirect transition between the two elements helps users to

easily understand that they are dependent on each other, andthat there exists an inclusion relationship between them.

The transitions between two independent elements includezooming-out from the focused element to all elements, andthen zooming-in to the target element. These transitions helpthe user to easily know that they are irrelevant.

As depicted in Figure 3, we generated iPosters usingactual Lecture♯1 4 for Database at Portland State Universityby Prof. Laura Bright. We can easily find that this lectureemphasized the content ofRelational Database.

V. EXPLORATORY SEARCH ON PRESENTATIONS

We build an exploratory search tool that aids users tosearch multiple presentations in search results by a givenquery: (1) focused searching and (2) exploratory browsing.Then, we measure dependence of the structure of the iPosterbased on user browsing behavior, as follows:

D(H) =1

|H| − 1

|H|−1∑n=1

1

dist(en, en+1), en ∈ H (4)

Here,H is a browsing history based on user browsing be-havior.en is a browsed element inH. We define the browsedelement, focusing on zoom-in operations of elements bythe users, that the elements can be considered as the usersare interested in. Then, we calculate a degree ofD(H) byusing average of relevance between the browsed elements.Function |H| returns the number of the browsed elements,|H| − 1 then denotes the number of edges between them.dist(en, en+1) is a shortest distance betweenen and en+1

in an order, which is calculated by the number of edgesbetween the browsed elements on the structure of the iPoster,then, dist(en, en+1) ≥ 1. When dist(en, en+1) returns1,the relevance betweenen and en+1 is closest. In this case,we set the threshold valueγ at |H|/|Ep|, |Ep| denotesthe number of nodes in a partial tree included all browsedelements of the structure of the iPoster. IfD(H) ≥ γ,the browsing behavior can be considered well-matched onthe structure of the iPoster; contrarily, ifD(H) < γ, thebrowsing behavior can be considered partially-matched onthe structure of the iPoster.

A. Focused Searching of Presentation Contents

When a user browses along the structure of the iPosterfocused on a topic and its subtopics with zoom-in operations,we assume that it is focused searching based on well-matched browsing behavior, he wants to get details of thefocused topics. Algorithm 1 describes a procedure for fo-cused searching in a sub-structural layoutFS = (Ed, R, P ).Ed is a set of elements related tox by users’ operations asan input. r is a type of semantic relationshipsR definedin Table I. P is a set of presentations in search resultsby a given query. This procedure representse′ related to

4http://web.cecs.pdx.edu/h̃owe/cs410/lectures/RelationalIntro 1.ppt

668

Figure3. An Example of a generated iPoster based on our proposed method

Algorithm 1 ExploreFS = (Ed, R, P )

Require: x is an element in a given presentationp, lastbrowsed by a user with a zoom-in operation.

Ensure: R = {(e, e′, r)|e, e′ ∈ E, e, e′ ∈ p′}R ⇐ ϕfor all presentationp′ in a given domaindo

if r is show relationshipthene ⇐ xR ⇐ (x, e′, r)P ⇐ p′

end ifend for

x accordingto (x, e′, r), in which r is show for findingdetails ofx.

B. Exploratory Browsing of Presentation Contents

When a user browses topics in apart on the iPoster, weassume that it is exploratory browsing based on partially-matched browsing behavior, he wants to get much relevantinformation of the browsed topics. Algorithm 2 describes aprocedure for exploratory browsing in a sub-structural layoutEB = (Ew, R, P ). Ew is a set of elements related tox andy by users’ operations as an input. This procedure representse′ related tox and y according to(x, y, r), in which r islikewise for finding relevant information ofx andy.

Figure 4 illustrates an example of exploratory browsing, auser firstly zooms-in to the area of ‘Forest Ecosystem,’ afterthat zooms-out it and zooms-in to the area of ‘Food Chain,’and zooms-out it and zooms-in to the area of ‘Products.’ Weconsidered that he wants to get a lot of information about‘Food Chain’ and ‘Products’ along the content related to

Algorithm 2 ExploreEB = (Ew, R, P )

Require: x, y are elements in a given presentationp,browsed by a user with a zoom-in operation.

Ensure: R = {(e, e′, r)|e, e′ ∈ E, e, e′ ∈ p′}R ⇐ ϕfor x, y such that(x, y, r) ∈ R in p do

for all presentationsp′ in a given domaindoif r is likewise relationshipthene ⇐ x, yr ⇐ describeR ⇐ (x, e′, r) = (y, e′, r)P ⇐ p′

if r is descibe relationshipthene′ ⇐ zR ⇐ (e, z, r)P ⇐ p′

end ifend if

end forend for

‘Forests and Humans.’ Due to ‘Products’likewise ‘FoodChain,’ and theydescribe ‘Forests and Humans,’ ‘Forestsand Humans’ has its details (i.e., ‘Products’ and ‘FoodChain’) only inPA

5. In this work, we can extract ‘NitrogenCycle’ and ‘Rainforest Animals’, whichdescribe ‘Forestsand Humans’ inPB

6, and represent a whole of ‘Forestsand Humans’ with its details (i.e., ‘Products,’ ‘Food Chain,’‘Nitrogen Cycle,’ and ‘Rainforest Animals’). In addition,

5http://teacherweb.com/AB/GilbertPatersonMiddleSchool/MsDavid/Tree-Types-2b-Posting-version.ppt

6http://www.marinepolicy.net/cparsons/Ecology/12-Forests.PPT

669

Figure4. An example of exploratory browsing based on partially-matched browsing behavior

the iPoster can automatically navigate the whole of ‘Forestsand Humans’ to ‘Nitrogen Cycle’ and ‘Rainforest Animals.’The iPoster firstly zooms-out from the area of ‘Food Chain’(i) to the whole area of ‘Forests and Humans’ (ii) thatshows an overview of ‘Forests and Humans’ with ‘NitrogenCycle’ and ‘Rainforest Animals.’ Next, the iPoster zooms-in to ‘Nitrogen Cycle’ (iii) and ‘Rainforest Animals’ (v),respectively. It helps the user to understand details ‘NitrogenCycle’ and ‘Rainforest Animals’ of ‘Forests and Humans.’

In this way, when many elements are extracted, we need toconsider how to select candidate elements from presentationcontents such as weights of the elements.

VI. CONCLUSIONS ANDFUTURE WORK

In this paper, we built an exploratory search tool forpresentation contents based on iPoster generation, whichrepresents textual and graphic elements in a structural layoutwith ZUIs, to promote user interaction. In order to generatean iPoster, we introduced a semantic structure analysismodel for extracting elements and determining semanticrelationships between them from slides. The iPoster enablesusers to browse and explore easily and efficiently throughvarious presentations.

In the future, we plan to consider a collaborative ex-ploratory searching tool, which will provide a way tosummarize already encountered information. The tool couldtailor these summaries to the respective skill levels ofcollaborators.

ACKNOWLEDGMENTS

This research was supported in part by Strategic Infor-mation and Communications R&D Promotion Programme

(SCOPE), the Ministry of Internal Affairs and Communica-tions of Japan, and a Grant-in-Aid for Scientific Research(B)(2) 26280042 from the Ministry of Education, Culture,Sports, Science, and Technology of Japan.

REFERENCES

[1] J. Lanir, K. S. Booth, and S. Wolfman, “An Obser-vational Study of Dual Display Usage in UniversityClassroom Lectures,” inJournal of Human-ComputerInteraction, vol. 28, no. 4, 2013, pp. 335–377.

[2] R. Spicer, Y. Lin, A. Kelliher, and H. Sundaram,“NextSlidePlease: Authoring and Delivering Agile Mul-timedia Presentations,” inACM Transactions on Mul-timedia Computing, Communication and Applications,vol. 8, no. 4, 2012, pp. 53:1–53:20.

[3] R. W. White and R. Roth, “Exploratory Search: Beyondthe Query Response Paradigm,” inSynthesis Lectures onInformation Concepts, Retrieval, and Services, vol. 1,no. 1, 2009, pp. 1–98.

[4] B. Detlor, C. W. Choo, M. MacKenzie, and D. Tunbull,“Information Seeking and Use in Diverse OrganizationalContexts,” inProc. of the American Society for Infor-mation Science and Technology, vol. 46, no. 1, 2009,pp. 1–4.

[5] D. Ellis and M. Haugan, “Modelling the InformationSeeking Patterns of Engineers and Research Scientistsin an Industrial Environment,” inJournal of Documen-tation, vol. 53, no. 4, 1997, pp. 384–403.

[6] S. Pradhan, “An algebraic query model for effective andefficient retrieval of xml fragments,” inProc. of the32nd international conference on Very large data bases(VLDB 2006), 2006, pp. 295–306.

670