Ginga Middleware on a SoC for Digital TelevisionSet-Top Box

Bruno Policarpo Toledo Freitas, Altamiro Amadeu SusinPPGEE - UFRGS

Federal University of Rio Grande do SulPorto Alegre, Brazil

Email: {bruno.policarpo,altamiro.susin}@ufrgs.br

Alexsandro BonattoPGMICRO - UFRGS

Federal Institute of Rio Grande do SulPorto Alegre, Brazil

Email: alexsandro.bonatto@restinga.ifrs.edu.br

Abstract—This paper presents the porting of the GingaMiddleWare onto a Digital Television System-on-Chip. Ginga isan open-source software layer compliant to the Brazilian DigitalTelevision Standard. The porting of Ginga is being carried outon a home made FPGA platform containing a Leon-3 processorrunning Linux, and the implemented audio and video decodersand graphics processing engine. The SoC has an external memoryto store reference frames, OS and Ginga, user interface andapplications with local and remote interactivity. Furthermore,accelerators are being designed to boost the SoC performance byimplementing in hardware the most processor demanding Gingaprimitives, with the profiling being made on a Ginga Virtualmachine, therefore creating a ”Ginga-Ready” platform.

Keywords—Ginga; Digital Television; Middleware; Set-Top Box

I. INTRODUCTION

Digital Television (DTV) has many benefits over traditionaltelevision systems that go beyond better audio and videoquality. Digital information medium enables the compressionof data at high rates, allowing the transmission of high defini-tion images through the same broadcast channel. Furthermore,the combination of information compression techniques, datapacking and multiplexing enabled to transmit other types ofdata through the same television channel, like an ElectronicProgram Guide and Software Applications. By this way, DTVoffers an increasing possibility of interactivity between thetelevision viewer and the broadcast station, locally generatedgraphics and messages, and even downloading of softwareapplications through the television channel.

The DTV platform needs to run an Operating System (OS)to execute software applications and user interface commandsto deliver possibilities needed for interactivity, like live editionof content. A Set-Top Box (STB) for Digital Television mayhave decoders for video, audio, and closed-captions data andalso support to run DTV applications. The middleware isa common interface between software applications and theOS that allows to run applications on whatever platform,independently on the hardware configuration or OS flavor.

For the Brazilian Digital Television System (SBTVD),the middleware’s specification was done by the Telemidia-Labs on PUC-RJ, on a work which resulted on the ITU-T Recommendation for IP-TV standard [1] and a referenceimplementation [2]. In parallel, an ongoing project aims to

develop a System on Chip (SoC) for the Brazilian DigitalTelevision System to be prototyped on programmable devices(FPGA), containing all the modules of the STB except the RFfront end and the demodulator. This STB implementation con-tains a CPU used to manage the user interface and peripherals.A fully compatible system with SBTVD standard can then beimplemented by porting the middleware to this CPU.

This, however, brings a whole new set of challenges forthe current SBTVD Architecture. First, there is the problemof porting the reference implementation of the middlewareto the CPU used for the SoC. Second, considering an initialporting is done, it will be running on a system with very limitedresources, in contrast with the native environment of a normaldesktop. Third, considering that the port now runs flawlessly,there is a whole new set of hardware to communicate.

This paper concentrates on the analysis and new additionsbeing made to deliver a full Ginga-Ready SoC. The nextsections are organized as follows: Section II presents the SoCplatform description, starting with the middleware and thenthe SoC, following by how these two still separated worldsmust work together. Section III presents the porting processesof the operating system and the middleware to the hardwareplatform. Section IV presents the testing methodology used, itsresults and the initial optimizations planned. Finally, Section Vpresents the conclusions.

II. PLATFORM DESCRIPTION

This section will describe the hardware-software platformcomposing the SBTVD, starting with a description of howGinga models a DTV application and then how it is trans-lated on its internal architecture. After that, the SoC will bepresented and then how it is connected with the middleware.

A. The GingaNCL Middleware

The Middleware Ginga [3] defines NCL (Nested ContextLanguage) as its Application Programming Interface (API) fordeveloping DTV applications for the SBTVD. NCL is a xml-based declarative language and, as such, its focus is on tellingwhat will happen instead of the steps to make it happen asan imperative language like C would do. To make such high-level approach possible, declarative languages must target aspecific application domain and define a model to describe

978-1-4799-2507-0/14/$31.00 c©2014 IEEE

these applications so that they can be written according to thismodel.

NCL’s model for describing DTV applications is based onthe idea of relating different types of media over time andspace. To illustrate this concept, a very simple NCL programis shown on Fig. 1, which shows an image for 3 seconds onthe left half of the screen, with a starting delay of 2 seconds.It starts by declaring where the image will be shown, which isdone by the ”region” node with id ”left half of screen”. Thisregion is used by the descriptor node with id ”desc” to tellhow the image will be shown. In this case, aside from beingshown on the left half of screen, the descriptor also tells thedelay and the duration that any media using it will have. Withthe ”where” and ”how” defined, it is needed now to tell whatwill be shown, which is done by the ”media” node. It tells thename of the local image to load and the associated descriptordefined before. Lastly, the presentation needs to be told whenit should start. This is done by ”port” node by defining thecomponent ”image” as its starting media. By default, when aNCL program starts it plays all components declared by anyports.

<?xml version="1.0" encoding="ISO-8859-1"?><ncl id="simpleNCL" ><head>

<regionBase><region id="left_half_of_screen"

width="50%" height="100%"top="0%" left="0 %" />

</regionBase>

<descriptorBase><descriptor id="desc"

region="left_half_of_screen"delay="2s"

explicitDur="3s" /></descriptorBase>

</head><body>

<port id="start"component="image" />

<media id="image"src="img.jpeg"descriptor="desc" />

</body></ncl>

Fig. 1. A very simple NCL program which loads an image into the left halfportion of the screen.

B. The Middleware Architecture

Fig. 2 shows the middleware’s architecture. To achieve thetransparency from implementation and support different com-binations of hardware and operating systems, the middlewareis divided in two strongly coupled but independent parts: theCommon Core and the Presentation engines. The Presentationengine is responsible for parsing the NCL and scheduling theappropriate events. On our example of Fig. 1, it would createan event at 2 seconds where an component ”image” withdescriptor ”desc” is show, and another at 5 seconds wherea program termination signal is sent. These events trigger theloading of appropriate players by the Common Core, whichhave the function of translating these high level commands intolower library calls. Taking again the initial NCL example, themedia ”img.jpeg” would translate into a call to a jpeg library toload it and the descriptor ”desc” would scale the image usingthe graphical API configured on the Common Core engine.

Fig. 2. Ginga Middleware Architecture. The Presentation Engine does thetask of interpreting and scheduling events. The Common Core translates thosehigh level commands into library and system calls.

As a result, the services offered by a given middlewareconfiguration depend on what players are implemented on suchconfiguration. If a player does not exist for a requested media,like an unknown image extension, the application silentlyignores the command received. This rule is valid not onlyfor the media types but also for other optional functionalityimplemented by the Common Core. For example, to implementIPTV services, a tuner, a Transport Stream (TS) parser, and aData Processing unit for the DSM-CC protocol [4] must bebuilt and configured

C. Set-Top Box Hardware Architecture

The STB hardware platform used in this work includesa Leon-3 processor, a multi-client memory controller (MMC)module and hardware modules for video decoding, video out-put control and closed captions decoding. The STB hardwarearchitecture used in this work to perform the software tests isbased on the platform presented before in [5], without the AACaudio decoder and the MPEG2 TS demultiplexer. Also, a postdecoding video processing module presented in [6] is used inthis platform to combine graphics with the decoded video andto control the video output module. This design is implementedin a Virtex-6 FPGA board described in VHDL and uses37,420 slice registers and 47,406 slice LUTs, occupying 53%of the XC6VLX240T FPGA. The Leon3 processor and theH.264/AVC video decoder run at 75 MHz and the MMC runsat 200 MHz.

Both the decoded video and the on-screen display datagenerated by the software applications are stored in the sameshared memory to be composed by the video output module.Due to the elevated memory access rates and bandwidth, anefficient memory hierarchy design to achieve real-time perfor-mance for the decoding of high definition video sequences.For this high speed memory, a 1GB 64-bit wide DDR3-800 isused to store OS, data and video graphics.

D. Extended Set-Top Box with the middleware

Adding a middleware expands the functionalities of theSoC beyond than just decoding the main audio and videofrom the broadcaster but that demands changes to the systemaccordingly. The extended SoC with Ginga is shown on Fig. 3.

Fig. 3. Digital television set-top box architecture. The Ginga infrastructureis bolded; the interfaces it uses are on grey, and the dotted lines mark the datapaths needed.

The programs sent by the broadcaster can now be processed bythe middleware, so the Demux shown on Fig. 3 needs now toalso filter them and send to the CPU Leon-3. Furthermore, withthe addition of network capability, the MPEG2 TS can nowbe received encapsulated through a network protocol, makinganother possible input for the Demux.

The addition of a network also makes interactivity possible.This is done by embedding a NCLua script on the NCLapplication [7] that connects to the broadcaster. Any userinput can now be sent through this channel, processed by thebroadcaster and a response sent back. On NCL, a NCLua scriptis a special case of a media node, processed by a player onthe Common Core built by default.

III. PORTING THE SOFTWARE COMPONENTS

This section will present how the software componentsrunning on the Leon-3 CPU were ported, starting with theLinux development environment. A description of how theFramebuffer abstraction layer was adapted to the SoC willfollow, ending with the middleware’s porting process.

A. Porting the Operating System

To deliver a full Ginga SoC it is necessary to give an op-erating system behind it. Aeroflex Gaisler [8] makes availabletwo environments for embedded Linux development: SnapGearand Linuxbuild. The first has solid driver implementations forLeon3 peripherals, which was desirable for integration not onlyfor the middleware but for the project’s needs; and Linuxbuild,which had a more recent kernel version and used Buildroot forfile system generation.

Given the fact that Buildroot solves problems regarding themiddleware’s dependencies and is, overall, a better automated

build system, Linuxbuild was chosen over SnapGear. Thishowever brought up the problem of drivers implementation.Gaisler already provides most of the needed ones but twoimportant pieces were missing: the video drivers, needed forGinga to display its work, and the general purpose I/O, whichis used to communicate the operating system with the Demuxthrough the peripheral bus.

B. The Framebuffer

A crucial element that needs to be supported by theoperating system is a graphical backend. The middlewareoffers a variety of options for it; the one chosen for this workwas DirectFB [9], since it is a graphical API designed forembedded systems. It works on the top of Linux’s graphicalhardware abstraction layer called framebuffer[10], so a work-ing driver for it is needed. This driver would then have to mapthe OSD memory region of the DDR3 for the API to use it.

However, the current SoC architecture for the OSD channelmirrored the implementation for the video decoder, based onmacroblocks, which uses a pixel mapping that the framebufferdoesn’t have. Also, video standards like H.264 uses YCbCr asits default color space for coding and decoding, so a conversionfrom this to RGB color space is needed on the driver sothe DirectFB applications won’t be affected. These two factscombined would translate into a very unusual - and hard toimplement - driver design.

Due to this complications, the OSD channel was remod-eled. The macroblock pixel organization on memory waschanged to a plain scan one and the pixel depth was increasedto 32 bit using ARGB pixel format with Alpha channel,as described before in [6]. This alpha channel is used todetermine the composition between the main video and thegraphics generated by the CPU. These changes avoid the needto write custom functions for reading and writing pixels onthe framebuffer due to the color space conversion and pixelcoordinate remapping.

C. Porting the Middleware

Ginga will be running over the Operating System, however,since the CPU is based on SPARC v8 architecture, this meansthat it needs to be ported first. At this stage, the main goal wasto generate a middleware miming the desktop implementationon the SoC - the adaptation to it would come afterwards. Astarting configuration suggested by Telemidia Labs was usedfor the port.

This initial configuration also gave the libraries, applica-tions and protocols that needed to be ported as well. They werefirst searched through Buildroot to see what already had beenported; afterwards, a porting of what was not found was done.Porting, in this case, means just compiling without errors - themain goal is just satisfying the dependencies. At this stage, themost common issues on porting were incompatible code beingcompiled, for example, specific code for x86 processors.

The Middleware itself consists of a series of 18 packages,counting both the Common Core and the Presentation engines.Rules for compilation under Buildroot were created, and buildstested on the SoC to detect problems and correct them. At thisstage, the most common problem encountered was the lack of

proper support by the configuration rules for cross-compilation.These issues have been solved, easing the middleware’s portingto other kinds of systems that use Buildroot altogether.

IV. TEST METHODOLOGY AND RESULTS ANALYSIS

A testing methodology was made using the validation suitefor GingaNCL [11]. These tests were run on a virtual machineappliance [12], to be compared with the ported environment.However, three problems were noted by using this testsuite.First, the whole process was very long because of the highnumber of tests (400 on total). Second, the validation is doneby comparing a description of what should happen with whathas happened, which is done manually by an external observer.Third, some of them required to be terminated manually,increasing the already high burden of execution.

To simplify the analysis on the ported environment, avirtual machine miming the final configuration of the portingwas created and a full run of tests were done to check eachone for three conditions. One, if the test was matching itsdescription. Two, if the test wasn’t of error detection, meaningthat their return code was 0. Three, if the test didn’t requiremanual termination. These criteria reduced the initial 400 teststo 171 and allowed the creation of a semi-automatic testprocedure on the ported environment. This procedure consistsof automatic execution of tests where the only input needed isif the test was successful or not.

A. Results Analysis

Running the 171 tests on the ported machine, 84 wereconsidered to be successful - in other words, having the samebehavior of the virtual machine. Observing the failed tests,it was noted that the vast majority had strict time-stampedevents. As an example, one test shows a blue image for3 seconds and then changes to a red image for another 3seconds, ending the application afterwards. What happened onthe ported environment is that the blue image would take alot longer to show, the transition to the red image didn’t occurand the application ended abruptly.

The initial hypothesis for this was that, given that theevent scheduler of the middleware needs to work as a real-time system but it is not implemented as one, the transitioningevents were happening in the middle of a media processing,resulting on the errors observed. A counter-test was made,extending the timestamps to 10 times higher, and the oncefailed tests started to succeed.

Besides these tests, a DirectFB benchmarking routine wasused to evaluate the frame rate for software applications run-ning over the implemented platform. A routine called df andiwielded only 1.4 frames per second on the ported environment.Setting an initial objective of 30 fps, the optimizing hardwareneeds to give 21 times better fps than the plain port.

B. Optimizing the Middleware

The results on Section IV-A made clear that optimizationswill be needed so the reference implementation of the mid-dleware runs on the SoC. Changing Ginga’s software imple-mentations to the hardware ones of the SoC is one possibility,but it requires delving deeper into both the hardware and the

middleware and then change both, which is a sensitive anddemanding task.

A more realistic and safer approach would be implementingoptimizations that are independent from both the middlewareand the SoC current state. A good candidate that meets thiscriteria is the graphic back-end of the middleware. A specifichardware to accelerate it can be built separately from the SoCwhile being transparent to the middleware. In the proposedapproach, the middleware accelerator is connected directlyto the MMC module and making its interface available touserspace through a connection with the CPU bus.

V. CONCLUSIONS AND FUTURE WORK

This work presented the porting challenges and solutionsbeing developed to make a full compatible SBTVD SoCwith Ginga. A full Set-Top Box would be composed by thehardware decoders, accelerators for the middleware’s bottle-necks, an additional data path to feed the CPU with programssent by the broadcaster, and support for Transport Streams(IPTV) coming from the network. Optimizations are requiredsince the embedded environment has limited computationalresources. The starting point chosen for such optimizationsis the graphical backend because it is isolated from both themiddleware and the SoC. This requires a more detailed studyof its bottlenecks. Future work include executing a profiling ofthe middleware, which is currently in progress, and connectingthe hardware facilities to the middleware.

VI. ACKNOWLEDGMENTS

This work was supported by CNPq through PPGEE

REFERENCES

[1] Nested context language (NCL) and Ginga-NCL, ITU-T Recommendation H.761, April 2009. [Online]. Available:http://www.itu.int/rec/T-REC-H.761

[2] T. Labs. Gingancl reference implementation. [Online]. Available:http://git.telemidia.puc-rio.br

[3] L. Soares, M. Moreno, C. De Salles Soares Neto, and M. Moreno,“Ginga-ncl: Declarative middleware for multimedia iptv services,” Com-munications Magazine, IEEE, vol. 48, no. 6, pp. 74–81, 2010.

[4] Information Technology - Generic coding of moving pictures and asso-ciated audio information - Part 6: Extensions for DSM-CC, ISO/IEC13818-6 Standard, 1998.

[5] A. B. Soares, A. C. a. Bonatto, and A. A. Susin, “Development of a SoCfor digital television set-top box: Architecture and system integrationissues,” Int. J. Reconfig. Comput., vol. 2013, pp. 1–10, Jan. 2013.

[6] M. Negreiros, H. A. Klein, A. Bonatto, A. Soares, and A. Susin, “To-wards a video processing architecture for SBTVD,” in ProgrammableLogic (SPL), 2012 VIII Southern Conference on, 2012, pp. 1–6.

[7] F. Santanna, C. Neto, R. Azevedo, and S. Barbosa, “Development ofinteractive applications for digital television with ginga and nclua.”[Online]. Available: http://www.telemidia.puc-rio.br/sites/telemidia.puc-rio.br/files/MCNCLua.pdf (In portuguese)

[8] A. Gaisler. Embedded linux for leon cpu. [Online]. Available:http://www.gaisler.com/index.php/downloads/linux

[9] Directfb. [Online]. Available: http://www.directfb.org

[10] S. Venkateswaran, Essential linux device drivers, 1st ed. Upper SaddleRiver, NJ, USA: Prentice Hall Press, 2008.

[11] T. Labs. Gingancl testsuite. [Online]. Available:http://testsuite.gingancl.org.br

[12] ——. Gingancl exhibition tools. [Online]. Available:http://www.gingancl.org.br/en/ferramentas