CUNI at MediaEval 2014 Search and Hyperlinking Task:Search Task ExperimentsPetra Galuščáková and Pavel PecinaCharles University in Prague, Faculty of Mathematics and PhysicsInstitute of Formal and Applied Linguistics

• Enable users to find information relevant to the submitted textual query in a collection of audio-visual recordings (TV programmes provided by BBC).• Available metadata, subtitles, and automatic transcripts (LIMSI, LIUM, and NST-Sheffield).

1. System Description • The recordings were divided into shorter segments.• We employed the Terrier IR system and its implementation of the Hiemstra language model with stemming and stopwords removal.• Segmentation: we used segments of fixed length and we used our segmentation system based on Decision Trees (DT).

2. System Tuning• We tuned segment length for fixed length (60 seconds) and DT-based (50 and 120 seconds) segmentation.• We experimented with post-filtering of the retrieved segments.

• We employed the metadata.

• The fixed-length segmentation outperforms the DT-based segmentation with 120-seconds-long segments.• 50-seconds-long DT-segments outperform the fixed-length segments measured by MAP and precision-based measures, but are outperformed by fixed-length segments using the MAP-bin andMAP-tol measures.

AcknowledgmentsThis research is supported by the Czech Science Foundation, grant number P103/12/G084, Charles University Grant Agency GA UK, grant number 920913, and by SVV project number 260 104.

Transcripts Segmentation Segment Length Metadata Overlap MAP P 5 P 10 P 20 MAP-bin MAP-tol

Subtitles Fixed 60s No No 0.4209 0.7933 0.7433 0.5950 0.3192 0.3155Subtitles Fixed 60s Yes No 0.5127 0.7467 0.7267 0.6100 0.3433 0.3023Subtitles Fixed 60s Yes Yes 4.3527 0.7867 0.7733 0.7683 0.4150 0.1459Subtitles DT 120s Yes No 0.3692 0.7467 0.7133 0.6050 0.2606 0.2157Subtitles DT 120s Yes Yes 16.3486 0.8400 0.8367 0.8433 0.3172 0.0515Subtitles DT 50s Yes No 0.8028 0.7867 0.7667 0.6933 0.3199 0.2350

LIMSI DT 120s Yes Yes 4.6366 0.7133 0.7300 0.7617 0.3706 0.1007LIMSI Fixed 60s Yes No 0.4725 0.6667 0.6633 0.5467 0.3160 0.2696LIUM DT 120s Yes Yes 4.0709 0.6533 0.6800 0.6900 0.3345 0.099LIUM Fixed 60s Yes No 0.4371 0.6333 0.6400 0.5367 0.2651 0.2327

NST-Sheffield DT 120s Yes Yes 10.0198 0.7267 0.7533 0.7650 0.3342 0.0675NST-Sheffield Fixed 60s Yes No 0.4645 0.6667 0.6600 0.5667 0.2974 0.2598

• The highest results are achieved on the subtitles.• Metadata improve the results.• Segmentation into fixed-length segments is very effective, but it can be outperformed.• MAP-tol measure may be preferred if retrieved segments can overlap.

• The best results are achieved in experiments using subtitles.• In most of the cases, LIMSI scored better than LIUM and NST-Sheffield transcripts and NST-Sheffield scored better than LIUM.• In most cases, the concatenation of the segment with metadata improved the results.

4. Conclusion

Tab1. Results of the Search sub-task for different transcripts, segmentation types, segment lengths, meta-data, and removal of overlapping segments.

3. Results

DT-based Segmentation:• For each word in the transcript, it decides whether the segment ends after this word or not.

• Makes use of cue word n-grams, cue tag n-grams, silence between words, capital letters, division given in transcripts, and the output of the TextTiling algorithm.

• The score of all measures, except the MAP-tol measure, are notably higher in the experiments which do not use post-filtering.

• In these cases, the relevant retrieved segments frequently overlap each other.

• User may have already seen the relevant segment → these segments could not be expected to be so beneficial.