phd research proposal of jan paul

Research Proposal

PhD Research Project ProposalBy

Manoranjan Paul

1. Title

A New Approach to Very Low Bit-Rate Video Coding Systems

2. ObjectiveTo develop a very low bit-rate video coding system (MPEG4/H.26X based architectures, algorithms, and applications) with scalable bit-rate vs. video picture quality facilities for video-telephony, videoconferencing, and video streaming to mobile phones and handheld personal digital assistants (PDA), which have limited processing and bandwidth capacity.

3. Statement of the Research Problem

IntroductionVideo is incredibly stimulating in electronic communications and multimedia applications. Video can do a great deal to enhance a presentation, illustrate a proper technique, or advertise a new product. Video files are photographic images played at speeds that make it appear to the human eye as if the images or frames are in full motion. Such files can be extremely large because of the number of images required to give the appearance of motion. Depending on the screen size of the video file, a single second of uncompressed video running at 30 frames per second may require more than 30 MB of storage space. In addition to storage, Bandwidth, the amount of data a communication channel can carry, is also an obstacle to the delivery of video.

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (1 of 13) [4/29/2003 12:25:12 AM]

Research Proposal

Delivering video over the Internet is challenging because it needs huge amount of storage capacity and a wide bandwidth.Despite these challenges, more and more multimedia applications, including digital videos, are disseminated everyday via the Internet and mobile telephone networks. In order to be used effectively however, video is often compressed for storage and transfer, and then decompressed for use. Research into very low bit-rate digital video coding face additional daunting challenges of meeting two inherently conflicting requirements—reducing the transmission bit-rate as well as increasing the image quality—that are sometimes inversely proportional due to bandwidth of communication medium and computational complexity. So video compression with reasonable image quality is a very active research topic currently. Compression depends on two factors, (1) motion estimation— a process of estimating the pixels of the current frame from the reference frame and (2) motion compensation— a process of compensation for the residual error after motion estimation. The following are three widely used motion estimation processes:

1. Optical flow equation based methods,2. Pel-recursive methods, and3. Block matching methods.

The first two methods are computationally expensive. So block-matching methods have been widely adopted by the international digital video coding standards such as H.261/263/263+/263++ [6][7][8] and MPEG-1/2/4 [3][4][5]. H.26X series are widely used for video-phoney and video-conferencing applications that require very low bit-rate to accommodate public switched telephone network (PSTN).An image is partitioned into a set of non-overlapped, equally spaced, fixed size, small rectangular blocks (known as macroblocks); and the translation motion within each block is assumed to be uniform. Although this simple model considers translation motion only, other types of motions, such as rotation and zooming of large objects, may be closely approximated by the piecewise translation of these small blocks provided these blocks are small enough. This observation originally made by Jain and Jain, has been confirmed persistently since then. Displacement vectors for these blocks are estimated by finding their best-matched same positioned block in the previous frame. The same positioned block in the previous frame is not always the exact


Research Proposal

representation of current block so pixels-by-pixels difference should be calculated for error compensation to reconstruct the current block at the decoder. In this manner, motion estimation and compensation are significantly easier than that for arbitrarily shaped blocks. Since the motion of each block is described by only one displacement vector, the side information on motion vectors decreases. Furthermore, the rectangular shape information is known to both the encoder and the decoder, and hence does not need to be encoded, which saves both computational load and side information. Most international video coding standards use a block size of area 16×16 or 8×8.Although very simple, straightforward and efficient, the block matching motion compensation technique has its drawbacks. Firstly, it has an unreliable motion vector field with respect to the true motion in 3-D world space. In particular, it has unsatisfactory motion estimation and compensation along moving boundaries. Secondly, it causes block artifacts. Thirdly, it needs to handle side information, that is, it needs to encode and transmit motion vectors as an overhead to the receiving end, thus making it difficult to use smaller block sizes to achieve higher accuracy in motion estimation.All these drawbacks are primarily due to its simple model: each block is assumed to experience a uniform translation and the motion vectors of partitioned blocks are estimated independently of each other. Unreliable motion estimation, particularly along moving boundaries, causes more prediction error, hence reducing coding efficiency.The block artifacts do not cause severe perceptual degradation to the human visual system (HVS) when the available coding bit rate is high. This is because, with a high bit rate, a sufficient amount of the motion-compensated prediction error can be transmitted to the receiving end, hence improving the subjective visual effect to such an extent that the block artifacts are not perceptually annoying. However, when the available bit rate is low, particularly and below 64 kbps, the artifacts become visually unpleasant. The assumption is that the motion within each block is uniform, requires a small block size such as 16×16 and 8×8. A small size leads to a large number of motion vectors, however, resulting in a large overhead of side information. A study by [30] indicated that 8×8 block matching performs much better than 16×16 in terms of decoded quality due to better motion estimation and compensation. The bit rate required for encoding motion vectors, however, increase significantly (about four times), which may be prohibitive for very low bit rate


Research Proposal

coding since the total bit rate needed for both prediction error and motion vectors may exceed the available bit rate. It is noted the when the coding bit rate is quite low, say, for example of the order of 20 kbps, the side information becomes compatible with the main information (prediction error) (see [31]). In spite of its drawbacks, block matching is still by far the most popular and efficient motion estimation and compensation technique utilized for video coding, and it has been adopted for use by various international coding standards. Some of the recent advancements in block matching techniques are summarised below:

Hierarchical Block Matching—A set of different sizes for both the original block and correlation window are used. The successive levels with smaller window sizes and smaller displacement ranges are capable of adaptively estimating motion vectors more locally.Multigrid Block Matching—Uses a multigrid structure, another computational powerful structure in image processing, to provide a variable size block matching. With a split-and-merge strategy, the thresholding multigrid block matching technique segments an image into a set of variable size blocks, each of which experiences an approximately uniform motion. It makes uniform motion within each block more accurate than fixed-size block matching can do.Predictive Motion Field Segmentation—To solve moving boundary problems, predictive motion field segmentation technique make the blocks involving moving boundaries have the moving field measured with pixel resolution instead of block resolution. This method saves shape overhead, and avoids side information with various techniques. Overlapped Block Matching—It enlarges blocks so as to make them overlap. A window function makes motion estimation and compensation so it achieves better performance than the conventional block matching.

MotivationAfter analysing the error surface between two successive frames in video sequences, it has been observed that there are many macroblocks, which have little or no motion. For this kind of macroblocks, there is no need to consider all of the pixels in those blocks. Considering only those intensity-changing pixels, further compression may be possible with less computational time. It has also


Research Proposal

been observed that motion vectors of a macroblock are sometimes related to its adjacent macroblock because these macroblocks may be the part of the same object. So motion vectors may be predicted from the adjacent macroblocks. Furthermore it may be also concluded from our error surface analysis, completed so far, that there exist relationships between velocity of the objects in a video sequence and motion vectors of the macroblock. Understanding such relationships may help research into estimating the motion vectors and compensation error more accurately than existing methods. During the last decade, a number of signal processing applications have emerged using wavelet theory. Among those applications, the most widespread developments have occurred in the area of data compression. Wavelet techniques have demonstrated the ability to provide not only high coding efficiency, but also spatial and quality scalability features. Some algorithms are developed by exploiting the features of the wavelet transformation such as the Embedded Zero-tree Wavelet method (EZW) (see [32]) and the Set Partitioning In Hierarchical Trees (SPIHT) (see [33]) technique.

Review of Relevant Research and Theory Some recent research partly addressed some of the issues stated in the above motivation. [1] proposed four 128-pixel patterns to represent the less moving macroblocks. For the very low motion video, the number of intensity changing pixels is not as high as 128 pixels. Otherwise for the very high motion video, pattern represent is not fruitful to represent the macroblocks. [11] proposed eight 64-pixel patterns for the same purpose. To do this, proposed method divides the macroblocks into three categories according to their moving region. The proposed patterns successfully represent the macroblock with low motion. Using a fixed number of patterns does not fit well for all kinds of video sequences because objects have different texture and movements. [12] proposed a strategy for the prediction of the motion vector from the adjacent macroblocks. Considering a moving region of the macroblocks may lead the motion vector prediction a new dimension. We know motion vectors of the macroblocks located on the boundaries of the objects are not accurately estimated. [13] successfully proposed a method for edge oriented block motion estimation. [14] proposed a method, which overcome the local minima problem of the error surface in tradition fast searching algorithm. [15] - [19] and [21] proposed various kinds of fast search strategies. Fast search strategy reduces the computational time for searching and also


Research Proposal

may reduce image quality. [23][24] proposed feature based search strategy. A new error criterion is proposed in [20]. An efficient encoding method is described in [22]. Morphological operations contribute a significant improvement in motion estimation. These kinds of operations reduce the noise in frames of video sequence. See [9]. Wavelet transformation contributes a significant improvement in low bit rate video coding and video image quality. [25] and [29] described the processes how the video image quality can be improved using wavelet transformation. [26]-[28] successfully applied the Embedded Zero-tree Wavelet algorithm to improve the coding efficiency.

Research Hypothesis Many of the existing video-coding standards, e.g., MPEG-1/2, tend to employ block-based techniques because of their implementation simplicity and also because they generally provide good results when the bandwidth requirement is relaxed. This is, however, not the case with low bit-rate block-based video coding such as in H.26X. The shape of a moving object is generally arbitrary and may not necessarily be aligned with the hypothetical grid structure created by the fixed-sized, non-overlapping rectangular blocks, termed MacroBlock (MB) in the coding standards. The typical size of a MB, being 16×16 pixels, leads to a large number of blocks, some of which will contain only static background, some will have moving objects and some a combination of the two. [1] and [11] successfully introduced four 128-pixel and eight 64-pixels patterns respectively to represent the small moving region macroblocks. In [11], MBs were classified according to the following three mutually exclusive classes: I ) Static MacroBlock (SMB)—Blocks that contain little or no motion; II) Active MacroBlock (AMB)—Blocks that contain moving object(s) with little static background; III) Active-Region MacroBlock (RMB)—Blocks that contain both static background and some part(s) of moving object(s).

By treating AMB and RMB alike, as is done in H.263/H.263+, leads to coding inefficiencies [1]. In order to improve this efficiency, block size may be reduced only to add additional information to be transmitted due to the increase in the number of blocks [10]. Both [1] and [11] successfully addressed the above issue by


Research Proposal

segmenting each RMB into two regions using a fixed number of predefined regular patterns. They respectively considered four, 128-pixel and eight, 64-pixel predefined RMB patterns. Once the segmentation process was complete, motion estimation/compensation was then only performed on moving regions. Each SMB was skipped for transmission (since they did not contain motion and could be copied from the reference frame) and each AMB was treated exactly as defined in H.263 standard, using motion estimation and compensation techniques. The non-coding of SMB patterns and limiting the number of RMBs to a prescribed set of patterns lead to an improved coding efficiency.Error surface analysis among frames may indicate a relation between movements of the objects with respect to the background. Since there are inter frames relations, error surface may be predicted from adjacent error surfaces. In this way motion vector may be predicted from the connected macroblocks. Patterns representation clearly reduces the number of transmission bits. By introducing more patterns, we can find more compression. But the problem is that to search the suitable pattern from a large number of pattern codebook, impose a large number of computational times. A quick searching algorithm may reduce the time for pattern searching. Another point is that 64/128 pixels fixed pattern cannot represent all RMBs properly. So arbitrary shaped pattern may be effective. So the following are my research goals:

Low bit-rate video coding algorithm focusing on moving region: Divide the macroblocks of a frame into some classes of little or no motion, a few motion and full motion blocks according to object movements; we can reduce the motion vector calculating time and compensational error. As the consequence we may get better compression ratio as well as better image quality.

Apply error surface co-relation in video coding: Error surface analysis explores a lot of relationships among objects’ motion, velocity, texture, shape etc. After observing the error surface features we can conclude the nature of the objects of that particular video sequence.

Apply artificial neural network in presenting arbitrary shaped moving objects: If we know the shape of the objects in a macroblocks, we approximate the macroblocks by using arbitrary shaped pattern. To find out the shaped of the macroblocks, we may use artificial neural networks to classify the macroblocks into some classes. By doing this we can reduce the computational time of motion vector estimation


Research Proposal

and error compensation.

Apply wavelet transformation, EZW, and SPIHT algorithm: Now data compression is using the wavelet transformation instead of the discrete cosine transformation. It also improves the quality of the images. There are some efficient algorithms for example, EZW, SPIHT, etc., are used to encode the image data, and the potential for exploiting this new technology will be another area that this research project will examine.

4. General Research Time-Table

First Year Second Year Third Year

Literature

review

Error-surface analysis Designing architectures & algorithms, and implementing a low bit rate video coding

system

Apply wavelet transformation, EZW, and SPIHT

Explore ANN & alternative data mining methods

Thesis writing

5. Research Already Undertaken

Literature review of error surface analysis, video and still image compression, encoding and decoding techniques, general coding technique (for example DPCM, Run length coding, Zero length coding, Huffman Coding, Arithmetic coding, dictionary coding etc.), video and


Research Proposal

still image quantization techniques, all digital video coding standard such as H.261 series, MPEG series, Wavelet and Fourier transformation techniques, Embedded Zerotree Wavelet and Set Partitioning In Hierarchical Tree algorithm, various video and still image file format and some research works related of low bit rate video coding

Error Surface analysis of frames difference for motion estimation, nature of movements of the objects, objects involvements in a particularly frame, change of pixels intensity vs. objects velocity etc.

Design a set of regular patterns for small moving region in a macroblock: Depends on the observations, already found some frequent patterns.

Develop a Variable Pattern Selection (VPS) algorithm and its Extended (EVPS) version: An algorithm, which estimates the best pattern for a particular video sequences.

Develop a Fast Pattern Searching (FPS) algorithm: It reduces the time significantly compare to other algorithms for finding the best patterns.

6. Publications

Journal Papers A Variable Pattern Selection Algorithm for Low Bit-Rate Video

Coding (In preparation for Journal of Pattern Recognition)

A Fast Pattern Searching Algorithm with Controlled Bit-Rate. (to be prepared for signal and image processing journal)

Conference papers A Low Bit-Rate Video Coding Algorithm Focusing on Moving

Regions with Variable Regular Patterns (Accepted by the International Conference on Signal Processing (ICSP-2002), Beijing, China, 26–30 August, 2002) (see attachment)

A Variable Pattern Selection Algorithm with Improved Pattern Selection Technique for Low Bit-Rate Video Coding Focusing on Moving Objects (submitted to International Workshop on Intelligent Knowledge Management Technique (I-KOMAT 2002, Podere d’Ombriano, Crema, Italy, 16-18 September, 2002) (see attachment)


Research Proposal

A Fast Pattern Searching Algorithm for Pattern Matching in Low Bit-Rate Video Coding (to be submitted to The 6th World Multi Conference on Systemics, Cybernetics and Informatics, SCI’02, July 14-18, 2002, Orlando, Florida, USA) (see attachment)

7. Applications of this research

Studio-based and desktop video conferencing

Surveillance and monitoring

Telemedicine

Computer based training

Video phone

Video over Internet

8. ConclusionsLow bit-rate in video coding has many applications, including video telephony, Videoconference, live mobile phone telecasts, etc. Especially when a device has no or little storage memory, very low bit-rate video coding may change its usefulness tremendously. My objective is the development of a very low bit-rate video coding system without sacrificing the video image quality, which will add a new dimension in the application of multimedia.

9. Ph.D. Supervision Team:

Joint-Supervisor: Prof. Laurence Dooley (50%)Joint-Supervisor: Dr. Manzur Murshed (50%)

10. References[1] Fukuhara, T., K. Asai, and T. Murakami, “Very low bit-rate video coding with block partitioning and adaptive selection of two time-differential frame memories,” IEEE Trans. Circuits Syst.


Research Proposal

Video Technol., 7, 212–220, 1997.[2] Gonzalez, R.C. and R. E. Woods, Digital Image Processing, Addition-Wesley, 1992.[3] ISO/IEC 11172, International Standard, 1992.[4] ISO/IEC 13818, MPEG-2 International Standard, Video Recommendation ITU-T H.262, 1995.[5] ISO/IEC N4030, MPEG-4 International Standard, 2001.[6] ITU Telecom. Standardization Sector of ITU, “Video codec for audio-visual services at p× 64kbits/s,” ITU-T Recommendation H.261,1993.[7] ITU Telecom. Standardization Sector of ITU, “Video coding for low bitrate communication,” Draft ITU-T Recommendation H.263, 1996.[8] ITU Telecom. Standardization Sector of ITU, “Video coding for low bitrate communication,” Draft ITU-T Recommendation H.263, Version 2, 1998.[9] Maragos, P., “Tutorial on advances in morphological image processing and analysis,” Opt. Eng., 26(7), 623–632, 1987. [10] Shi, Y.Q. and H. Sun, Image and Video Compression for Multimedia Engineering Fundamentals, Algorithms, and Standards, CRC Press, 1999. [11] Wong, K.-W., K.-M. Lam, and W.-C. Siu, “An Efficient Low Bit-Rate Video-Coding Algorithm Focusing on Moving regions,” IEEE trans. circuits and systems for video technology, 11(10), 1128–1134, 2001.[12] Gallant, M. and F. Kossentini, “An Efficient Computation-Constrained Block-Based Motion Estimation Algorithm for Low Bit Rate Video Coding,” [13] Chain, Y.-L. and W.-C. Siu, “Edge orieted block motion estimation for video coding,” IEEE Proc. –Vis. Image Signal Process., Vol 144, No. 3, June 1997[14] Chan, Y.-L and W.-C. Siu, “Reliable block motion estimation through the confidence measure of error surface,” Signal Processing 76(1999) 135-146, 1998[15] Ganbari, M. “The Cross-Search Algorithm for Motion Estimation,” IEEE Trans. On Communications, 38-7, 1990[16] Po, L,-M. and W.-C. Ma, “A Novel Four-Step Search Algorithm


Research Proposal

for Fast Block Motion Estimation,” [17] Liu, L.-K and E. Feig, “A Block-Based Gradient Descent Search Algorithm for Block Motion Estimation in video Coding,” 1051-8215(96)05194-4.[18] Lu, J. and M. L. Liou, “A Simple and Efficient Search Algorithm for Block-Matching Motion Estimation,” IEEE Trans. On Cir. And Sys. For Video Tech., Vol. 7, No. 2, 1997[19] Chow, K.H.-K and M. L. Liou, “Genetic Motion Search Algorithm for Video Compression,” IEEE Trans. On Cir. And Sys. For Video Tech., Vol.3, No.62, 1993[20] Fuldseth, A. and T. A. Ramstad, ”A new error criterion for block based motion estimation,” [21] Li R., B. Zeng, and M. L. Liou, “ A new Three-Step Search Algorithm for Block Motion Estimation,” IEEE Trans. On Cir. And Sys. For Video Tech., Vol.4, No.4, 1994[22] Kossentini, F. and Y.-W. Lee, “Computation-Constrained Fast MPEG-2 Encoding,” IEEE Signal Processing Letters, Vol. 4, No. 8, 1997[23] Chan, Y.-L and W.-C. Siu, “A Feature-Assisted Search Strategy for Block Motion Estimation,” 0-7803-5467-2/99 IEEE[24] Chan, Y.-L and W.-C. Siu, “An Efficient Search Strategy for Block Motion Estimation Using Image Features,” 1057-7149(01)06035-3[25] Fan, G., and W-K Cham, “Postprocessing of Low Bit-Rate Wavelet-Based Image Coding Using Multiscale Edge Characterization,” IEEE Trans. On Cir. And Sys. For Video Tech., Vol.11, No.12, 2001[26] Ramaswamy, V.N., K.R. Namuduri, and N. Ranganathan, “Context-based Lossless Image Coding Using EZW Framework,” IEEE Trans. On Cir. And Sys. For Video Tech., Vol.11, No.4, 2001[27] Ramaswamy, V.N., K.R. Namuduri, and N. Ranganathan, “Performance Analysis of Wavelets in Embedded Zerotree-based Lossless Image Coding Schemes,” IEEE Trans. On Sig. Proc., Vol.XX, No.Y, ZZZZ[28] Jianhua, W.U., C. Olivier, and C. Chateller, “Embedded Zerotree Runlength Wavelet Image Coding,” 0-7803-6725-1/01 2001 IEEE[29] Shipeng, L. and W. Li, “Shape-Adaptive Discrete Wavelet


Research Proposal

Transforms for Arbitrarily Shaped Visual Object Coding,” IEEE Trans. On Cir. And Sys. For Video Tech., Vol.10, No.5, 2000[30] Chan, M. H., Y.B. Yu, and A. G. Constantinides, “Variable size block matching motion compensation with applications to video coding,” IEEE Proc., 137(4), 205-212, 1990 [31] Lin, S., Y. Q. Shi, and Y.-Q. Zang, “An optical flow-based motion compensation algorithm for very low bit-rate video coding,” Proc. 1997 IEEE Int. Conf. Acoustics, Speech Signal Process., 2869-2872, Munich, Germany, 1997; Int. Imaging Syst. Technol., 9(4), 230-237, 1998[32] Shapiro, J., “Embedded image coding using zerotrees of wavelet coefficient,” IEEE Trans. Signal Process., 3445-3462, 1993[33] Said, A. and W.A. Pearlman, “A new fast and efficient image codec based on set partitioning in hierarchical trees,” IEEE Trans. Circuits Syst. Video Technol., 243-250, 1996


phd research proposal of jan paul

Documents

step search algorithm

arbitrary shaped pattern

error surface analysis

low bitrate communication

video image quality

coding bit rate

block motion estimation

rate video coding