yalms: a data structure for storage and replay of multimedia content in digital classes

.

YALMS Basics

M.Zhanikeev, H.Koide -- [email protected] -- YALMS: A Data Structure for Storage and Replay of Multimedia Content -- http://tinyurl.com/marat140116 -- 2/21...

2/21

.

YALMS Basics

.YALMS.....

.... stands forYetAnotherLectureManagementSystem

• a new generaction --HTML5- and cloud-based LMS 06

• one function: video annotation robot 07

• another function: Group Drive -- a P2P version of cloud storage 08

• high-performance multimedia system based on multicore, SSD, etc. 09

06 M.Zhanikeev+1 "YALMS: Yet Another Lecture Management System" IEICE ET研究会 (2013)

07 M.Zhanikeev+1 "...Automatic Slicing of Screen Capture Video using Slide PDFs" IEICE ET研究会 (2013)

08 M.Zhanikeev+1 "YALMS: A Group Drive API for Cloud-Based Classrooms" IEICE IN研究会 (2013)

09 M.Zhanikeev+1 "...Solid State Drives in Rich Multimedia Content Processing, Storage and Streaming" ITE-MMS研究会 (2013)


3/21

.

YALMS Roadmap

YALMS: Intro

YALMS.annotation

2013.07・ET研

2013.09・ET研

YALMS.streaming2014.01・ET研

YALMS.hiperf2013.10・ITE-MMS研

YALMS.cloud

YALMS.format

YALMS.done!

DASH Adaptive HTML5

PDF video annotation

cloudssync

groupdrive

SSD

recording

2014.1・LOIS研

2013.11・IN研

replaymultimedia

storage

multicore


4/21

.

YALMS.format : Objectives


5/21

.

YALMS.format Objectives

1. pack all the contents of a lecture into a single .yalms file◦ audiovideo, slides, files and versioning, etc.

◦ official components: AV for audiovideo,S for slides or screen capture video,C isfor files and versioning

2. replay .yalms files in a browser in standalone mode◦ directly from .yalms file -- no backend system


6/21

.

Objectives ... Visuallly

VA

S

Step 1: Record

C

Long-TermStorageLecturer

Files

YALMSRobot

ViewersHTML5

Step 2: Add PDF

Lecturer

SYALMSRobot


7/21

.

YALMS vs... : Related Works


8/21

.

YALMS.format vs Multimedia System

.Traditional Multimedia System..

.

• powerfull back-end• HTTP streaming in realtime

.YALMS.format..

.

• .yalms file is a binary file -- can be stored anywhere◦ like in cloud storage

• no streaming, the file is simply traversed using exiting APIs◦ any RESTful API, cloud storage, local filesystem


9/21

.

YALMS.format vs SMIL

5: 6:<par> 7: <video src="joe-video"/> 8: <text src="http://www.example.com/joe-text"/> 9: <seq> 10: <audio src="rtsp://www.example.com/joe-audio"/> 11: <audio src="tim-audio"/> 12: </seq> 13:</par> 14: 15:

joe-videojoe-text

jow-audio tim-audio

• close but not enough

• need to pack actualcontent into the file


10/21

.

YALMS.format in Browser


11/21

.

HTML5 FunctionalityBrowserAbilities

Flash

HTML5

Traditional

Binary

WebSockets

WebWorkers

passByRefBlob ByteArray

postData( data, [ data])structuredCopyNew Chrome feature

Adaptive Streaming?

JSON, BSON

?


12/21

.

HTML5: the SaveAs Function

• NOTE: JS now natively can save binary chunks to local filesystem


13/21

.

YALMS.format : Design


14/21

.

YALMS.format : Design

VA

S

Step 1: Record

CLecturer

YALMS.format

YALMSRobot

Watchfilesystem

Watchscenechange

Step 2: Add PDF

Lecturer

S

AV

SE

CEPE

Annotation

SEX

• even-driven packing

• events are specific to eachcontent type

• events are used to cut.YALMS files intoblocks


15/21

.

YALMS.format : Events

1. SE : screen capture events◦ screen capture is video, so -- scene changes

2. CE : binary file diffs◦ there are LibXDiff 15 and Xdelta 16, the latter is preferred

3. PE : page events -- replaces SE◦ blocks based on page changes

15 "LibXDiff" http://www.xmailserver.org/xdiff.html (2013)

16 "Xdelta Project Page" http://xdelta.org/ (2013)


16/21

.

.YALMS File Design

The First Lecture.yalms

The entire fileTime

Head Tail

0

Block border types1 Fixed minimal size2 Scene change (I-Frame)3 Page change4 New File(s)5 New File Diff(s)

Block Header Designseqno flags Offset

Block length in bytesBlock 1: type(3) + offset(29)... (the same for all blocks)Block 1 (binary data) ...Block

border

Byte

s Bits

offset

8 16 24

…

48

12


17/21

.

YALMS.format Robot and Screen

PNG

VIDEO

SA V

MISC.C

CE

Screen

Browser

YALMSRobot

Get binary VA

SE PEor

Auto “download file” “download” on click

HTML5File API


18/21

.

Wrapup


19/21

.

Wrapup

• working version at 02

• release plan: end of March◦ the entire package will be offered as open source

02 myself "Homepage of the YALMS.format Subproject" https://github.com/maratishe/yalms.format (2013)


20/21

.

That’s all, thank you ...


21/21

.

[01] MEXT (2013)The enPiT Projecthttp://www.enpit.jp

[02] myself (2013)Homepage of the YALMS.format Subprojecthttps://github.com/maratishe/yalms.format

[03] W3C (2013)Synchronized Multimedia Integration Language (SMIL 3.0)http://www.w3.org/TR/SMIL/

[04] W3C (2013)HTML5: W3C Working Drafthttp://www.w3.org/TR/html5/

[05] W3C (2013)File API: W3C Working Draftwww.w3.org/TR/FileAPI


21/21

.

[06] M.Zhanikeev+1 (2013)YALMS: Yet Another Lecture Management SystemIEICE ET研究会

[07] M.Zhanikeev+1 (2013)...Automatic Slicing of Screen Capture Video using Slide PDFsIEICE ET研究会

[08] M.Zhanikeev+1 (2013)YALMS: A Group Drive API for Cloud-Based ClassroomsIEICE IN研究会

[09] M.Zhanikeev+1 (2013)...Solid State Drives in Rich Multimedia Content Processing, Storage and StreamingITE-MMS研究会

[10] M.Zhanikeev+0 (2013)...Content Aggregation in Browsers using Recent Advances in HTML5COMPSAC


21/21

.

[11] M.Zhanikeev+0 (2013)Experiments with application throughput in a browser with full HTML5 supportIEICE ComEx

[12] H.Pucha+3 (2008)Adaptive File Transfers for Diverse EnvironmentsUSENIX

[13] D.Frey+2 (2012)Probabilistic Deduplication for Cluster-Based Storage Systems3rd ACM SoCC

[14] D.Meyer+1 (2012)A Study of Practical DeduplicationACM Transactions on Storage (TOS)

[15] (2013)LibXDiffhttp://www.xmailserver.org/xdiff.html


21/21

.

[16] (2013)Xdelta Project Pagehttp://xdelta.org/

[17] IETF (2002)The VCDIFF Generic Differencing and Compression Data FormatRFC3284


21/21

yalms: a data structure for storage and replay of multimedia content in digital classes

Technology

yalms files

blockss yalms

yalms roadmap

yalms basicsm

replay of multimedia

format yalms file

data structure

yalms file official