3.1 technologies for the convergence of … › strl › results › annual2013 › 2013...3.1...

3.1 Technologies for the convergence of broadcasting and telecommunications

3.1.1 Hybridcast

We are continuing our research on Hybridcast, which is a service platform to provide flexible and extensible new servic-es that take advantage of the characteristics of broadcasting and telecommunications networks. In FY 2013, we released the first version of this service to the public and continued with R&D to make the service more efficient and standardization. We showed the appeal of Hybridcast by demonstrating it to visitors at the NHK STRL Open House and other exhibitions at home and abroad.

■ Practical use of HybridcastThe IPTV Forum published the first edition of Hybridcast

technical specifications in March 2013. (1)(2)(3) Operational speci-fications for receivers were also created and standardized, and manufacturers started selling TV sets supporting Hybridcast in June. NHK launched Hybridcast services (Figure 1) in Sep-tember 2013 and followed up by launching more in December.

Prior to the launch, NHK STRL had participated in Hybridcast promotion projects and contributed to the studies on services to be offered by NHK, facilities and equipment, and operational

NHK STRL is studying technologies that take advantage of broadcasting and telecommuni-cations networks to offer new services and en-hance programming.

We participated in at IPTV Forum’s standard-ization activities that led to Hybridcast techni-cal specifications being issued in March 2013. TV sets based on these specifications came to mar-ket not long after, and NHK launched Hybridcast services in September. We also worked on ad-vanced Hybridcast and Hybridcast for 8K Super Hi-Vision broadcasting. In addition, we contrib-uted to the standardization of advanced Hybrid-cast at IPTV Forum Japan.

To provide more attractive hybrid services, we are conducting R&D on technologies that use program related information and social network-ing services (SNS). We studied methods to direct Internet users to various programs and related services by linking the broadcasters’ program-related information with open data. We also stud-ied ways to promote TV viewing by analyzing the viewing behaviors of SNS users. We developed a system to allow users to share CG content while communicating with each other through an SNS. We studied a technology to provide program-re-lated information matching viewers’ preferences by monitoring their viewing behavior and infer-ring their interests. We developed a method to analyze the viewers’ behavior based on gazes, prototyped a viewing behavior monitoring sys-

tem, and experimentally studied the relation be-tween gazes and viewers’ interests.

Along with our efforts to improve the conve-nience of hybrid services, we are developing technologies to provide secure and reliable ser-vices. We developed a new authorization scheme to control different license management systems for broadcast and telecommunications servic-es in an integrated framework. We developed a verification system for the next-generation Con-ditional Access System (CAS) and contributed to standardization activities at the Information and Communications Council. As a cryptogra-phy technology to provide more secure and reli-able services, we developed an attribute-based encryption scheme with which personal infor-mation is stored securely in the cloud and only service providers who meet certain conditions can decrypt the information. To protect content, we developed a cryptography method capable of tracing the key owner.

For program distribution over a communi-cations network, we developed a distributed broadcasting system that enables easy time-shift viewing of past programs over a network and a push content delivery method to distrib-ute information via communication channels in synchronization with broadcast content. We also developed a hybrid live streaming system linking peer-to-peer (P2P) and content delivery network (CDN) distribution.

16　｜　NHK STRL ANNUAL REPORT 2013　

3 Convergence of broadcasting and telecommunications

Figure 1. Example of Hybridcast services

aspects. We demonstrated a wide variety of Hybridcast servic-es together with commercial broadcasters and pay TV service providers at the NHK STRL Open House 2013, using Hybrid-cast compliant receivers developed in cooperation with man-ufacturers and telecommunications carriers. We exhibited a “home screen” service, active program guide, scroll news, and “TV60 Minogashi-Natsukashi,” a service to provide past pro-grams on demand. . We also exhibited an interactive picture drawing application (Figure 2), musical instrument sessions application (Figure 3), multilingual closed-captions and the TV note service, which TV links tablet devices and synchronizes service provision with the progress of programs. We improved the public’s understanding of Hybridcast and showed the fea-sibility of various services. Examples of Hybridcast services to be offered by commercial broadcasters were also exhibited (4).

Dissemination of Hybridcast and the growth of its services will require an environment to enable participation by a wide range of application developers. Whereas a versatile HTML authoring tool can be used to develop the Hybridcast applica-tions themselves, which are written in HTML 5, an operational verification is also needed to preview how the developed ser-

vice functions on various devices. Thus, we developed an ap-plication development support tool capable of previewing the linkage of programs and portable devices on a PC and exhib-ited it at the NHK STRL Open House (Figure 4) (5).

■Technologies for advanced HybridcastWe are studying the technical specifications for the next ver-

sion of Hybridcast (called advanced Hybridcast) and Hybrid-cast for the 8K Super Hi-Vision era.

The new functions include: a system model of non-broad-cast managed applications allowing various service providers other than broadcasters to participate; a recording and re-production functionality; precise synchronization using time stamps; and a sophisticated device linkage interface.

We developed an application distribution management sys-tem (Figure 5) to study non-broadcast managed applications. It can register and distribute applications and issue signature keys to authenticate them. We exhibited a system model for other service providers than broadcasters to provide applica-tions at the NHK STRL Open House.

With the launch of 8K Super Hi-Vision (SHV) broadcasting in 2020 in mind, we studied possible 8K Hybridcast services that could take advantage of the large, extremely high-definition screen of this new form of broadcasting (Figure 6). We exhib-ited a live broadcasting service for covering marathons at the NHK STRL Open House. A high-definition map to show the current location is displayed together with multiple windows showing video coverage of the race and “tweets” from social networking services (SNS) provided by broadcasting and tele-

NHK STRL ANNUAL REPORT 2013　｜　17

3　Convergence of broadcasting and telecommunications3.1 Technologies for the convergence of broadcasting and telecommunications

TV screen

Tablet display

Figure 2. Example of Hybridcast services at the NHK STRL Open House (Interactive picture drawing application)

Tablet devices

TV screen

Figure 3. Example of Hybridcast services at the NHK STRL Open House (Musical instrument sessions application)

Application preview window

Preview management window

Figure 4. Application development support tool

Figure 5. Application distribution management system

communications. Together with other divisions involved in the development

of SHV, we studied the SHV data broadcasting and designed a reference model of the protocol stack based on MPEG Media Transport (MMT) we also investigated data coding, transmis-sion methods and closed captions.

■ Hybridcast standardizationAt ITU-R, we contributed to a draft new Recommendation

for the general and technical requirements of hybrid broad-

casting and drafted a report on the different forms of hybrid broadcasting in the world, including Hybridcast and Europe’s HbbTV. In the Future of Broadcasting Television (FOBTV) group, we participated in a study on a system model based on the use cases submitted to date. At the Technical Plenary/Ad-visory Committee Meeting (TPAC) of the World Wide Web Con-sortium (W3C) working on HTML5 standardization, NHK re-ported on the establishment of Hybridcast specifications at the domestic IPTV Forum Japan in March 2013 and on the launch of Hybridcast services in September. These reports conveyed the advantages of Hybridcast to the world. On the domestic front, we worked on standardization of advanced Hybridcast and HTML5 browser specifications at the IPTV Forum Japan.

We will continue with our research on more sophisticated services and standardization for Hybridcast. We will also in-vestigate hybrid services for the SHV age and standardize SHV data broadcasting.

[References](1) IPTVFJ STD-0010, “Integrated Broadcast-Broadband System Specifi-

cation Version 1.0”, IPTV Forum Japan(2) IPTVFJ STD-0011, “HTML5 Browser Specification Version 1.0”, IPTV

Forum Japan (3) M. Takechi: “Technical Specifications and Trend of Hybridcast,”

NHK STRL R&D, No. 142, pp. 12-19 (2013) (in Japanese)(4) A. Baba, H. Oomata, K. Matsumura, M. Takechi, S. Sunasaki: “Pro-

totype Hybridcast Services at NHK STRL, Open House 2013,” ITE Technical Report, Vol. 37, No. 41, pp. 17-20 (2013) (in Japanese)

(5) K. Majima, K. Matsumura, Y. Hironaka: “Development of Hybrid-cast Application Development and Distribution System,” NHK STRL R&D, No. 142,pp. 20-27 (2013) (in Japanese)

3.1.2 SNS and Program viewing analysis

We did R&D on technologies to make use of program related information and social networking services (SNS) to provide more attractive services that exploit the advantages of broad-casting and telecommunications.

■ Information network for broadcasting programsWe studied ways to direct Internet users from one program

to various services or other programs by utilizing broadcast-ers’ program guide information and related information and to form a community using SNSs to bring excitement to the viewing experience.

We devised a method to convert the program guide informa-tion in the Electronic Program Guide (EPG) to the Linked Open Data (LOD) format, link it with open data such as National Diet Library Authority Data and Twitter, and associate programs with program related information and social information (1). We prototyped an application to guide users from one program to related programs and information by web-based visualiza-tion (Figure 1).

For behavior analysis of users, we analyzed the results of experiments on the “teleda” social TV platform developed in FY 2011 and found that users fall into different types, such as those who prioritize their own preferences and those who are influenced by others’ behavior. Based on the analysis, we com-piled service design guidelines.

We also devised a method to estimate users’ viewing trends such as viewing times and programs viewed from their tweets on Twitter and use those estimates to categorize users. Iden-tifying influential users on SNSs based on the categorization and providing them with effective program recommendation services should promote TV viewing among SNS users.

■ Virtualization of video content productionWe studied virtualization of video content production based

on the server side video creation so that Internet users can easily produce their own video content.

We built an SNS-style CG content creation system (Figure 2) (2), in which SNS users can communicate through videos produc-tion using the servers on the network and exhibited it at the NHK STRL Open House. This system allows users to re-create and share content derived from already posted content. Users



Broadcast videoMap

SNS

Ranking data displayed

Live distribution

Figure 6. Example of 8K SHV Hybridcast services (Marathon live broad-casting)

Figure 1. Program-related information navigation application with open data

can easily produce derived content by using a graphical user interface (GUI). We conducted experiments with about 100 us-ers after we had posted sample content and demonstrated the system and method. Regarding linkage with Hybridcast, we developed a system to display sign-language CG animations on tablet devices concurrently with a broadcast and exhibited the system at the NHK STRL Open House. We also developed a simple TVML (TV program Making Language) player by using Web GL. It enables real-time runnning of TVML scripts sent with broadcasting without installing special software on the terminal.

We also worked on enhancing the functionality of the CG creation engine. We developed a new TVML player using the latest graphics library. The player can display more characters through the use of high-speed processing and improve high-quality translucent display.

■ Acquisition and analysis of viewing behaviorsWe are researching technologies for the automatic acquisi-

tion and analysis of viewing status and the estimation of view-ers’ interests, in a domestic TV viewing environment with a view to developing new services that provide program-related information matching viewers’ preferences.

We developed a method for measuring a viewer’s body movements and head pose and a machine learning system that estimates whether the viewer is gazing at the TV screen

or not. Experiments showed that gaze behavior can be judged accurately (3). We also developed a method that combines Bayesian inference with an estimation based on the intensity of facial expressions, which has improved the accuracy of fa-cial expression recognition. The research on gaze behavior es-timation and on facial expression recognition were conducted in cooperation with Tokyo City University and the University of Tokyo, respectively. Part of the research on gaze behavior estimation was conducted under a government-commissioned project from the Ministry of Internal Affairs and Communica-tions, titled “Strategic Information and Communications R&D Promotion Programme (SCOPE).”

We prototyped a viewing behavior monitoring system incor-porating the method of estimating gaze behavior (Figure 3). We used the system to study the relation between gaze behav-ior and inferred viewer interests through TV viewing experi-ments. The results demonstrated the possibility of inferring viewers’ interests by correlating gaze behavior with closed caption data that appears on screen during gaze periods. We also developed a method to selectively extract keywords from closed caption data that belong to particular classes derived from Wikipedia category information. This method enabled selective extraction of keywords identifying the names of peo-ple and places, which are often for the focus of viewer interest.

[References](1) K. Ariyasu, T. Nakagawa: “A Proposal of TV Program Information

Network Formation Method by Linked Open Data,” Proc. of FIT2013, no.2, F-035, pp. 363-364，(2013) (in Japanese)

(2) M. Douke, H. Kaneko, S. Inoue: “SNS Style CG Content Creation Sys-tem Enables Users to the Easy Chain of Re-areation”, Proc. of ITE Annual Conference, 2-3, 2013 (in Japanese)

(3) Takahashi, Clippingdale, Okuda, Yamanouchi, Naemura, Shibata: “Automatic rating of video contents based on viewer’s behaviors in living space,” SITIS2013, pp. 6-13 (2013)

3.1.3 Security technologies

To support the convergence of broadcasting and telecom-munications, we are researching integrated authentication federation technology for broadcast and telecommunications, a next-generation conditional access system (CAS), cryptogra-phy, and device-linked technology.

■ Integrated authentication federation technology for broadcast and telecommunicationsWe made progress with our R&D on an integrated authenti-

cation technology to enable secure and easy use of personal-ized services by linking license management schemes used in

broadcast and telecommunications services. Depending on the types of channels and terminals used, dif-

ferent methods are used by broadcasters and telecommunica-tions providers for access control and content protection. To view the same content in different environments, individual contract management is required for each service. To simplify this situation for subscribers to these services, we developed a license distribution management system embodying a new authorization scheme that merges different license manage-ment methods - a conditional access system for broadcast ser-vices and a digital rights management system for telecommu-nications services - in an integrated framework (1). A prototype



Figure 2. SNS-style CG content creation system

Figure 3. Example of gaze behavior estimation system in operation

system based on this method was exhibited at the NHK STRL Open House (Figure 1). This system enables content viewing rights obtained from a single user authentication to be carried over to different services of broadcasting and telecommunica-tions. To put it to practical use, we also made proposals to incorporate the next-generation conditional access system in commercial standards.

We developed the “TV note” system to provide personal-ized services securely while effi ciently collecting information on viewers’ interests through a linkage with authentication services. The system was exhibited as a service model of Hy-bridcast at the NHK STRL Open House. We also studied the specifi cations of the authorization scheme and a device link-age method in order to apply the authentication framework to actual services.

■ Next-generation CAS technologyWe are researching next-generation CAS technology for the

SHV test broadcasting in 2016. In FY 2013, we developed a veri-fi cation system and contributed to the standardization of the scrambling subsystem and associated information subsystem for ultra-high defi nition television at the Information and Com-munications Council. The scrambling subsystem uses a cryp-tography algorithm on the transmission side, and the associ-ated information subsystem uses a conventional conditional access system to manage the contract information in a way that is consistent with the access control functions of current services. For ease of maintenance and improvement of securi-ty, we also devised a downloadable conditional access system that sends the software updates to the receivers involved in processing associated information through broadcasting and telecommunications networks. These methods were refl ected in a report issued by the Broadcasting System Committee of the Information and Communications Council.

■ Cryptography algorithm for hybrid services

We researched cryptography technologies to provide secure and reliable hybrid services taking advantage of broadcasting and telecommunications. Service providers need information about viewers in order to personalize services. Meanwhile, viewers should not receive unexpected services from un-known providers. An effi cient way to meet these requirements is to store viewers’ information on a cloud. We studied an at-tribute-based encryption scheme (2) that encrypts the viewer’s information to protect his/her privacy and allows only the pro-viders meeting a policy set by the viewer based on attributes such as reliability of service provider to decrypt the informa-tion. By modifying the way of describing attributes, we found a way to use a smaller ciphertext and secret key. We evaluated the encryption on a computer. This research was conducted in cooperation with the Advanced Industrial Science and Tech-nology.

In our research on cryptography to protect content, we de-veloped a method that can trace the owner of a decryption key that has been illegally copied. To deal with possible cryptanal-ysis, we developed a way to update the scrambling method that that will work on existing as well as new receivers without increasing the transmission volume.

■ Interactive content technology for device linkage servicesWe studied technologies to enable broadcast content to be

shown on devices other than receiver equipped TVs Called “Augmented TV”, this technology captures images of the TV screen with a camera of a mobile terminal and processes the captured image on the mobile terminal. It provides a new viewing experience by displaying computer graphics (CG) of program-related characters on the mobile terminal in accurate synchronization with the TV screen.

In FY 2013, we built a system to produce and present pro-gram and associated CGs in a virtual space displayed on a mo-bile terminal. The system works in an easy and integrated way by using TV program Making Language (TVML) (3). We also achieved frame-level precision synchronization (under 0.03 s) between the TV image and the mobile terminal by using a mo-bile terminal’s camera to capture “synchronization markers,” graphic time data, on the broadcast content shown on the TV display.

[References](1) C. Yamamura, Y. Nishimoto, A. Fujii: “Development of an authoriza-

tion system for license delivery control - application of its system to CAS/DRM -”, ITE Annual Conference, 18-11 (2013) (in Japanese)

(2) N. Attrapadung, G. Hanaoka, K. Ogawa, G. Ohtake, H. Watanabe, S. Yamada: “Ciphertext-Policy Attribute-Based Encryption over Range Attributes,” SCIS, 2E4- 1 (2014)

(3) H. Kawakita, H. Kaneko, S. Inoue, A. Fujii, T. Nakagawa: “Production Platform of 3DCG Content Connected with TV Program - System Ar-chitecture for Augmented TV -”, ITE Annual Conference, 2-5 (2013) (in Japanese)

3.1.4 IP content delivery technology

With the aim of distributing programs over a network, we are researching a distributed server-based system for provid-ing broadcast programs over a network and IP content delivery technologies for future hybrid services.

■ Distributed server-based broadcasting systemWe are researching a distributed server-based broadcasting

system for time-shift viewing of past programs over IP net-works. The accumulation, reprocessing and distribution of broadcast content is handled by servers so that viewers can

view any of the previously programs as they like. In FY 2013, we upgraded the time-associated zapping sys-

tem prototyped in FY 2012 to enable program zapping by tags such as cast name, in addition to zapping by on-air time. We developed a database to manage tags associated with broad-cast on-air time and incorporated it in a distributed fi le sys-tem capable of storing and retrieving video, sound and related information of programs using broadcast time as ID. The re-sulting acquisition of associated tags in synchronization with video and audio lets viewers zap quickly without stress. We also developed a program viewing player that shows the tags

Broadcast content viewing

Telecommunications content viewing

EMM server

Tablet terminal

Receiver with built-in downloadable CAS content viewingEMM: Entitlement Management Message (Broadcast license)DRM: Digital Rights Management (Telecommunications license)

Authentication / authorization server

VOD server / DRM server

Figure 1. Integrated access control system for broadcast and telecom-munications



associated with a program in addition to information for time zapping (Figure 1) and a system for collecting viewing logs and visualizing them by video frame. We will use this system to acquire viewing data categorized by broadcast on-air time and tag.

In addition, we advanced with our development of a pro-gram viewing system that converts the resolution of programs stored in the distributed file system on the server side accord-ing to the display capabilities of the viewing terminal.

■ Hybrid content distribution technologiesWe are researching a push content delivery method to en-

able synchronized distribution over communication channels of the information to be incorporated and displayed in each video frame of broadcast content (Figure 2).

We began study in FY 2012 on a distribution method to con-trol the timing of sending information on the distribution serv-er so that the information arrives to many viewers at the same time. In FY 2013, we devised a distribution control method to minimize the difference in arrival times (jitter) of periodically distributed information in a quasi-network environment. The results of simulations showed that this method suppresses jit-ter in a large-scale distribution service (1).

■ Hybrid P2P-CDN live streaming technology

In light of the results of live streaming experiments at the London Olympics in 2012, we developed a hybrid live stream-ing system linking peer-to-peer (P2P) distribution and content delivery network (CDN) distribution (Figure 3). A distribution support server enables terminals not supporting P2P to receive live streaming video from a terminal supporting P2P. We also developed a method that uses this server to suppress differ-ences in the video reproduction timing between terminals receiving P2P distributed content and the terminals receiving CDN distributed content. We evaluated the system implement-ed on a commercial cloud and confirmed that it worked (2).

[References](1) S. Tanaka, S. Nishimura, Y. Endo: “A Study of Determining Standard

RTT in Push Content Delivery Method for Synchronization between Broadcast Programs and Push Contents,” IEICE General Confer-ence, B-6-38 (2014) (in Japanese)

(2) S. Nishimura, S. Tanaka, Y. Endo: “Development of a Hybrid P2P-CDN Live Streaming System,” ITE Winter Annual Convention, 12-9 (2013) (in Japanese)



Broadcast

Telecommunications

Player name

Hybridcast receiverPush

Push delivery by high-precision synchronization

Synthesized display on broadcast content

Program related information (Player name, etc.)

Figure 2. Example of Push content delivery method

For zapping by time

For zapping by tag

Figure 1. Program viewing player for time-shift zapping system

Live encoder Cache server Distribution support server P2P distribution server

P2P stream

Direct receptionP2P network

P2P-compliant terminalsNon-P2P-compliant terminals

CDN Distribution edge

Request transfer

Segment reception

Reception request

Distribution status monitor

Direct reception

Distribution status monitor

Figure 3. Hybrid P2P-CDN live streaming system

3.1 technologies for the convergence of … › strl › results › annual2013 › 2013...3.1...

Documents