can we benefit from solid state drives in rich multimedia content processing, storage and streaming?
DESCRIPTION
Multimedia storage today is mostly based on traditional (spinning) hard drives. Solid state drive (SSD) technology -- hard drives based on flash memory -- has existed for several years but does not yet offer large storage capacity per hardware unit. However, SSDs have unique technical characteristics which can be exploited for on-demand handling of rich multimedia content. This paper looks into this potential.TRANSCRIPT
.
The Big Picture: YALMS
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 2/17...
2/17
.
YALMS: Yet Another LMS• feature: input, processing, and delivery in equal shares 17
Step 1: Record and Stream
V A
S Creator
Viewers
Content Server
HTML5
HTML5 Muxer
Record Screen Streaming
Step 2: Annotate
Lecturer
S
Auto- annotate
C
17 M.Zhanikeev+1 "YALMS: Yet Another Lecture Management System..." IEICE・ET研 (July 2013)M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 3/17...
3/17
.
A Business Day at YALMS
• many screen and video streams uploaded by users 15
• many concurrent video annotation tasks (PDF + VIDEO) 16
• constantly changing popularity distribution of content, needs
dynamic relocations
15 "YALMS Project Home" https://github.com/maratishe/yalms (June 2013)
16 "YALMS Annotation Robot (software behind this paper)" https://github.com/maratishe/yalms.annotation (September 2013)
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 4/17...
4/17
.
YALMS Topics
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 5/17...
5/17
.
YALMS: The Big SSD Question
.The Question is..
.... can such a rich multimedia architecture benefit fromSSDs?• WHY1: SSDs have properties different from those of HDDs
◦ read/write, random/sequential
• WHY2: SSDs are becoming cheaper (MLC) so why not to use them?
• WHY3: parallel/multicore processing is not kind to HDDs but SSDscan cope
• WHY4: 50Mbytes/s throughput is not enough any more
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 6/17...
6/17
.
Where Are We Now? ExistingResearch
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 7/17...
7/17
.
Existing Research (1)• roughly, two groups:
1. SSD versus HDD research• filesystem abstraction and scheduling -- load balancing between SSD and HDD 01• SSD as cache memory 0203
2. parallel processing and themulticore• job graph partitioning 18• local Big Data processing 19• multicore streaming 11-- 14
01 J.Kim+4 "Disk Schedulers for Solid State Drives" 7th ACM EMSOFT (2009)
02 S.Rizvi+1 "Flash SSD vs HDD: High Performance .... Multimedia Storage Systems" 2nd ICCET (2010)
03 Y.Lee+1 "Mapping granularity and performance tradeoffs for solid state drive" The Journal of Supercomputing (2013)
18 X.Sui+3 "Parallel Graph Partitioning on Multicore Architectures" 23rd LCPC (2010)
19 R.Chen+2 "Tiled-MapReduce: Optimizing Resource Usages of Data-parallel Applications on Multicore..." 19th PACT (2010)
11 Y.Wei+3 "Energy-Efficient Real-Time Scheduling of Multimedia Tasks on Multi-Core Processors" ACM SAC (2010)
14 V.Galiano+3 "SSD architecture such as multi-channel and multi-way" EURASIP Journal ... Signal Processing (2013)
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 8/17...
8/17
.
Existing Research: What is Missing?
.In a Nutshell, Existing Research is.....
.
• extremely specialized and is built for one application only
• ... or, in business, the use is very primitive -- a temporary memory buffer
• ... at the same time.. YALMS needs generic load balancing◦ one design fits all
• changes along the timeline should cause changes -- dynamic optimization• ...
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 9/17...
9/17
.
The YALMS.hiperf project
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 10/17...
10/17
.
Meet the YALMLS.hiperf Project
Multimedia System
Creators
Viewers
Manage
HDDs HDDs HDDs HDDs HDDs SSDs
Cores Cores Cores
• a pool of SSDs,HDDs, etc.
• a separate pool ofCPU cores
• steady arrival anddeparture rates oftasks/processes
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 11/17...
11/17
.
YALMS.hiperf : The Overall Design
YALMS.hiperf
YALMS.hiperfRobot
A Multimedia System
ProcessProcessProcessProcessProcessProcessProcessReq(details)
Rep(agent)
AgentsAgentsAgentsAgentsAgentsHDDHDDHDDHDDHDDs
SSDsSSDsSSDsSSDsSSDsSSDs ….
• a firmware approach
• tasks/processes sendrequests
• replies point to agents
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 12/17...
12/17
.
YALMS.hiperf : The Basic Unit
…
Proc
essin
g
logic
SSD
HDD
ENGINE
Network storage
Input Next step
Read/write? Seq./rand.? (Chunk) size
HiPerf/Normal? Longer plans?
Multicore
Fork Join
• traditional parallelprocessingformulation
• engine performsruntimeoptimization
• tasks are assignedclasses
01 J.Kim+4 "Disk Schedulers for Solid State Drives" 7th ACM EMSOFT (2009)
02 S.Rizvi+1 "Flash SSD vs HDD: High Performance .... Multimedia Storage Systems" 2nd ICCET (2010)
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 13/17...
13/17
.
YALMS.hiperf : Runtime Optimization.Basic Formulation..
.
start from zero, collect history, thenoptimize accordingly • tasks should naturally have the few
heavy hitters distribution◦ if not, tasks should be partitioned
• then simply
minimize∑
i∈tasks
∑j∈time
lagi,j (1)
• ... details in future publications
03 Y.Lee+1 "Mapping granularity and performance tradeoffs for solid state drive" The Journal of Supercomputing (2013)
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 14/17...
14/17
.
Wrapup
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 15/17...
15/17
.
How Far is The Project?• software for multicore video annotation on top of SSD
◦ mostly reads, very few writes
• currently working on the generic version of software, will release soon
screen video
SlidesPDF
LaTeX/BeamerKeynote
PowerPoint…
VLCffmpeg
I-Frames I-Frames
pages PNGs
ANNOTATION
slices
PNGstimestamps
End productscenes cleanup
splitpdf2png
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 16/17...
16/17
.
That’s all, thank you ...
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17
.
[01] J.Kim+4 (2009)Disk Schedulers for Solid State Drives7th ACM EMSOFT
[02] S.Rizvi+1 (2010)Flash SSD vs HDD: High Performance .... Multimedia Storage Systems2nd ICCET
[03] Y.Lee+1 (2013)Mapping granularity and performance tradeoffs for solid state driveThe Journal of Supercomputing
[04] (2012)Data Compression in the Intel Solid-State Drive 520 SeriesIntel Technology Brief
[05] (2013)Cisco Content Delivery Engine 250Cisco Data Sheet
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17
.
[06] B.Sajadi+3 (2011)...SSDs for Large-Scale Interactive Graphics ApplicationsSymp. on 3D Graphics and Games (I3D)
[07] D.Zhao+1 (2013)HyCache: a User-Level Caching Middleware for Distributed File SystemsACM HPDC
[08] M.Ryu+2 (2011)Impact of Flash Memory on Video-on-Demand Storage: Analysis of Tradeoffs2nd ACM MMSys
[09] (2013)Performance ... On DemandConcurrent Computer Corporation (STEC), Case Study
[10] M.Ryu+2 (2012)Why ...flash-based multi-tiered storage systems performing poorly for HTTP videostreaming?22nd NOSSDAV
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17
.
[11] Y.Wei+3 (2010)Energy-Efficient Real-Time Scheduling of Multimedia Tasks on Multi-CoreProcessorsACM SAC
[12] S.Sankaraiah+3 (2011)GOP Level Parallelism on H.264 Video Encoder for Multicore ArchitectureIPCSIT
[13] T.Dias+2 (2010)H.264/AVC framework for multi-core embedded video encodersSoC
[14] V.Galiano+3 (2013)SSD architecture such as multi-channel and multi-wayEURASIP Journal ... Signal Processing
[15] (June 2013)YALMS Project Homehttps://github.com/maratishe/yalms
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17
.
[16] (September 2013)YALMS Annotation Robot (software behind this paper)https://github.com/maratishe/yalms.annotation
[17] M.Zhanikeev+1 (July 2013)YALMS: Yet Another Lecture Management System...IEICE・ET研
[18] X.Sui+3 (2010)Parallel Graph Partitioning on Multicore Architectures23rd LCPC
[19] R.Chen+2 (2010)Tiled-MapReduce: Optimizing Resource Usages of Data-parallel Applications onMulticore...19th PACT
M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 17/17...
17/17