n etworked m edia l ab. d ept. of i nfo. & c omm., k-jist video over ip as a voip extension...
Post on 29-Jan-2016
214 Views
Preview:
TRANSCRIPT
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Video over IP Video over IP as a VoIP extensionas a VoIP extension
JongWon Kim, Ph.D.
Advanced Network Conference (APAN Busan 2003)
Aug 27th, 2003
Networked Media LaboratoryDept. of Information & Communications
Kwang-Ju Institute of Science & Technology (K-JIST)jongwon@kjist.ac.kr
http://netmedia.kjist.ac.kr
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ContentsContents
Media Delivery over IP
Network-adaptive Media Deliveryo Networking Supporto Network Adaptation
Video over IP as a VoIP extension
Future of Video over IP - Toward Access Grid & Immersive Media
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Multimedia Delivery over IP
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Multimedia over IP NetworksMultimedia over IP Networks
Media BroadcastingHosted StreamingInteractive Conferencing
TNR
Internet
StreamingSender
Manager
Proxy
Proxy
Receivers
Conferencing
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Multimedia Delivery - Multimedia Delivery - TechnologiesTechnologies
Efficient Media Representation Techniques: Scalable source representation to deal with dynamically varying
bandwidth
Robust source representation to deal with high error/loss rate
Dynamic and Reliable Media Delivery Techniques: Network adaptation to map scalable and robust source
representations onto the MM characteristic-aware transport protocols
Adaptive resource provisioning/management to guarantee the desired QoS / multicast service
Universal Media Access with Security Techniques: Unified integrated media support with a scalable information
structure and a choice of efficient file format for MM
End-system and application customization with security concerns
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
0%
Pac
ket
Lo
ss
Command/ control (eg Telnet, Interactive
games)
Conversationalvoice and video
Voice/videomessaging
Streamingaudio/video
Transactions (eg E-commerce,Web-browsing, E-
mail access)
Paging, Downloads
(eg FTP,still image)
Fax
Background(eg Usenet)
5%
100 msec 1 sec 10 sec 100 sec
Zeroloss
Interactive Responsive Timely Non-critical
Delay
Multimedia Application’s QoS RequirementsMultimedia Application’s QoS Requirements
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Media Delivery – Application vs Media Delivery – Application vs NetworkNetwork
Application/Source
Network
Network Characteristics and Policies: Rate (CBR, VBR, ABR, …), Error
(random/bursty in BER/PLR/CRL), Delay/Jitter (tolerance)
Content-aware Transport
Application/Source Characteristic :
steady/burst, real-time?, loss-resilient?
Network-adaptive Delivery
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Network Adaptive Media Delivery
- Networking Support Part -
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Media Delivery – Adaptation & Media Delivery – Adaptation & SupportSupport
Media Representation(Compression)
Error Control
MediaSynchronizati
on
Inter-operableProtocol
SystemSupport
(Transport,OS,
Storage)
Congestion(Rate, Flow)
Control
DynamicNetwork
AdaptationFramework
NetworkMulticastSupport
Network QoSSupport
OtherMiddleware
Support
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Inter-operable Delivery with Inter-operable Delivery with Protocol SupportProtocol Support
Standard-based vs. Proprietary Protocols? Related issue: Standardized media content
representation (e.g. MPEG-X) vs. Others?
Security Protocols
RSVP
IPv4, IPv6
TCPUDP
RTCP RTSPSIPSAP
SDP
RTPHTTP SMTP
RTP Payload
Media Contents Conference ControlShared Tools
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
End-to-end QoS guarantee with
QoS provisioning (admission control and QoS controls) & Network adaptation
The Internet
Economical and reliable network with
Scalable multicasting & Optimized server location and load sharing
including contents routing
Paid services withCopyright management,
authentication,billing, etc
Required Network Infrastructure: Multicast, QoS,
and others
Content broadcast network
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Multicast Support for Multipoint Multicast Support for Multipoint Media DistributionMedia Distribution
Multicast streaming?Model (ASM, SSM, ALM/Overlay/P2P)
Reliable Multicast TransportMobile/Wireless MulticastResilient Multicasting of continuous media
I nternetI nternet
232/ 8232/ 8
I GMPv3I GMPv3I GMPv3I GMPv3
Source filtering Source filtering capabilitycapability
Source filtering Source filtering capabilitycapability
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
QoS-network Support for Quality Media DeQoS-network Support for Quality Media Deliverylivery
QoS Model: IntServ, DiffServ, Traffic Engineering (MPLS), …
QoS signaling
Reservation and Allocation
DiffServNetworkMIBMIB MIBMIB
WFQEF
BEWFQ
EF
BE
Console orClient User
JAVA InterfaceProgram
MIBMIB
ResourceManager-aware
Application
SLA
Resource Manager
RouteDB
Interface
Mon.
DB
K-JIST KISTI
KOREN21
Configuration
Reservationrequest
QoS monitor
BrowsingT
CP
/IP
TC
P/IP
BBTPAuth.
module
Route Info
moduleCAC
moduleMonitor-
ingmodule
Execution module SNMP module
SLA module Re source
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Toward Consolidated Support Toward Consolidated Support via Middleware?via Middleware?
Issues to be covered +++o Identificationo Directories, …o Security: Authentication, Authorization, …o …
Middleware:o Specialized networked services that are shared by
applications and userso A set of core software components that permit
scaling of applications and networks
GRID Middleware Home Networking Middleware
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Network Adaptive Media Delivery
- Network Adaptation Part:Bandwidth (rate), loss, and delay -
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Media Delivery – Adaptation & Media Delivery – Adaptation & SupportSupport
Media Representation(Compression)
Error Control
MediaSynchronizati
on
Inter-operableProtocol
SystemSupport
(Transport,OS,
Storage)
Congestion(Rate, Flow)
Control
DynamicNetwork
AdaptationFramework
NetworkMulticastSupport
Network QoSSupport
OtherMiddleware
Support
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Required Network Adaptation Required Network Adaptation FunctionalitiesFunctionalities
In case of Reliable Unicast with feedbacko Network Congestion Control => Quality Adaptation: Source Rate Control / Network Rate Adapt
ationo Error control: Error Resiliency and Error Recoveryo Quality Recovery: Post-processing & Concealment o Synchronization Control & Adaptive Playout
Media Pump
Adaptable Source
Congestion Control
(TCP-friendly)
Client
Quality Adaptation
Network
Quality Recovery
Error Control
Synchronization Control
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
TCP-Friendly Congestion Control TCP-Friendly Congestion Control / Rate Control/ Rate Control
TCP-Friendly Congestion Control: equation-based …
Toward unified congestion control handling: Congestion Manager (CM)
Congestion Control / Rate Control / Flow Control
Multicast congestion control?
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
TCP-Friendly Congestion Control TCP-Friendly Congestion Control / Rate Control (Cont.)/ Rate Control (Cont.)
Rate Control byo Source Rate Control (on-line
encoding only)o Source Rate Adaptation (or Shaping)
o Smoothing – via Bufferingo Packet Schedulingo Pre-fetching
I
B B B B B B
PP
Td 2Td 3Td 4Td 5Td 6Td 7Td 8Td 9Td
B B B B B BP P
Td0 2Td 3Td 4Td 5Td 6Td 7Td 8Td 9Td
I
Ravg
Time
Time
Transmission Rate
Transmission Rate Non-Smoothed
VBR stream
Smoothed near-CBR stream
I
B B B B B B
PP
Td 2Td 3Td 4Td 5Td 6Td 7Td 8Td 9Td
B B B B B BP P
Td0 2Td 3Td 4Td 5Td 6Td 7Td 8Td 9Td
I
Ravg
Time
Time
Transmission Rate
Transmission Rate Non-Smoothed
VBR stream
Smoothed near-CBR stream
I B B P B B P B B P B B I B B P
New connectionrequest
I-pictureStarting point
Superposed stream(existing sources)
period
(1) (2) (3) (4)
time
Forward Proxy (Delegate)
Reverse Proxy
(Surrogate)
Web Server
+ Content Distribution+ + Proxy Plug-ins
Forward Proxy (Delegate)
Reverse Proxy
(Surrogate)
Web Server
+ Content Distribution+ + Proxy Plug-ins
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Proactive and Reactive Error Proactive and Reactive Error ControlsControls
Network error controlso Proactive (FEC), Reactive (ARQ), and Hybrido Delay is key controlling factor
Network (Channel) error control vs Source error resiliency?
Quality Recovery: Error Concealment and Post-processing
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Source EncoderNetwork
Adaptation SinkDecoderNetwork
Adaptation
Network FramingApplication
FramingCoding
Error Detection
Loss Detection
Error Correction
Erasure Correction
CRC
Sequence No.
Retransmission
FEC
Quality Control(Layered Coding)
Data Structuring(Synchronization)
Resynchronization
Loss Concealment
Post Processing
Courtesy of Dr. Noh NVW 2001
Error Control and Quality Error Control and Quality RecoveryRecovery
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Synchronization Control and Adaptive PlaySynchronization Control and Adaptive Playoutout
Adaptive playouto Utilizing silenceo Utilizing time-scale modifica
tion
Synchronizationo Intra-mediao Inter-mediao Inter-client
o Tight vs Loosely-coupled
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Media Delivery – Network Media Delivery – Network AdaptationAdaptation
< Source prioritization for content-aware network adaptation >
MM Content
Scalable streamin rate/error
Integrated &
prioritizedprioritized stream
Channelstream Network
Source encoding
Source transcoding &Framing/Multiplexing
Trans-MUX &Channel codingwith FEC/ARQ
Transmission
Feedback from Host/Application
Feedback from Network
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Wired/WirelessNetworks
Layered Video Encoding Receiver/User
NetworkAdaptation
Network Adaptation &Prioritized Packetization
VideoPre-
process-ing
R-DAnalysis
Layereden-
coding
TargetMinimumBuffer Size(Receiver
Constraint)
TargetMinimumBandwidth(Network
Constraint)Frame
Complexity(Quality
Constraint)
ConstantQualityRate
Control
(SourceRate/ErrorResilience)
(only ifapplicable)
PrioritizedPacketi-zation
LayeredRPI
NetworkMonitoring
&FeedbackHandling Application Feedback
EstimatedAvailable
Bandwidth,Loss/Delay
Network Feedback
PrioritizedLayeredPackets
R-D /Corruption
Model
Network Feedback (end-to-end)
NetworkAdaptation
(NetworkFiltering)
ReceiverSelection
Media Delivery – Network Adaptation
NetworkAdaptation
(NetworkRate/Loss/
Delay)
NetworkAdaptation
(Receiver /User
Adaptation)
Corrup-tion
Analysis
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Video over IP as VoIP extension (V2oIP)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
VoIP Extension: Voice + Video + DataVoIP Extension: Voice + Video + Data
We want to achieve IP TELEPHONY (a.k.a. voice/video/data convergence): Real time voice, video and data communication over Internet Protocol (IP) networks
However, latest high-end VC system supportso H.263+ or H.263++ (w/ Annex U for background restoration)o 384 kbit/s or 768 kbit/so TDM/ISDN (H.320) or IP (H.323)o CIF (352x288) or CIF-Interlace 2x(352x288)o Capture and rendering typically interlaceo 25/30 fps, 50/60 fields per secondo Round-trip delay around 400ms +o Round-trip delay for multipoint (w/ MCU) more than one second
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
VV22oIP Building BlocksoIP Building Blocks
SIP MGCP H.323
Proxy Announcement Toolkit Multi-point Control Gatekeeper Toolkit
Service Creation Gate-
keepers GatewaysCall AgentsAnnouncement
Player Conference Bridges
Service Creation Gate-
keepers Gateways
Protocol stacks (H.323, MGCP, SIP) Switching cores (e.g. Gatekeepers, Call Agents) Media engines (Announcement, Mixing, Gateway) Service creation environment
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
MCU
Multicast Audio and Video
Unicast Audio and VideoDecentralized Centralized
POTS
xDSL
Cable
PSTN
BackendServers (services)
GKC/BEC
AdministrativeDomain A
AdministrativeDomain A
AdministrativeDomain B
GKA/BEA BEB/GKB
H.323Entity
TA
H.323Entity
TB
H.323Entity
TC
GKC1
GKA1 GKB1
Non-H.323Entity
TP
GW
BECHNon-H.323System
ClearingHouse
Signaling
PhysicalConnection
VV22oIP DeploymentoIP Deployment
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Video Quality and Function Issues (1)Video Quality and Function Issues (1)Improve Video Quality and Size
Upcoming H.264 video encodingo Product design stage now, 640x480 at full frame rate on one high-end DSPo Bit rate around 384 kbit/s yields acceptable picture qualityo Subjective quality significantly better than H.263++ w/ CIF at same bit rate
Variable, content-dependent capture rateo Linking with region-oriented (maybe combined with segmentation) encoding on top of variable f
rame skipping
Wide-screen video support o Can support video resolution like 2560x480 with single CCD camera with custom, wide-angle
lens; Multiple DSPs for distributed codec is required; 1.5 – 3 Mbps Progressive scan hardware
o No A/D and D/A losses; Better quality and good delay characteristics; 250ms round-trip delay over IP demonstrated?
Enhance error resilience with ERPS and so on
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Video Quality and Function Issues (2)Video Quality and Function Issues (2)Provide more natural, cooperative environment for multi-
site conferencing
Let’s avoid transcoding and enhance user experienceo Create all representations needed in the sending endpoint and simulcast them (in line wit
h layered multicast video) o Ideally suited for IP multicast environment; But can be with overlay (application-layer) multi
cast; o May need some innovative changes in the control and mux protocolso No transcoding artifacts whatsoevero No additional end-to-end delay for multipoint when using IP multicast
Let’s get rid of fixed environment that discourages uncooperative use of the technologyo Toward flexible meeting environment without fixed camera position, fixed background, stud
io lighting, seating
Let’s add more functionalities while keeping user interface simple and straightforward
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Automatic Automatic Broadcast Broadcast and and Recording Recording
Note takingNote taking
Remote ParticipationRemote Participation
VoIP Extension: Real-time CollaborationVoIP Extension: Real-time Collaboration Bring Voice, Screen, Computation together to improve: Real-time C
ollaboration Vision from Microsofto Rich presence and IMo PC and phone integrationo Data/screen conferencingo Information agento Enhanced Meetings
incoming incoming communicommunicationcation
to task to task listlist
to web to web phonephone
contextcontext
2003 Gurdeep Singh Pall from Microsoft Corporation
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Future of Video over IP
- Access Grid & Immersive Media -
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
The Access GridThe Access GridAccess Grid does for people what the computation
al Grid does for machinesMuch more than teleconferencing, if
possible at commodity pricesTo enable groups of people to interact with Grid res
ources and to use the Grid technology to support group to group collaboration at a distanceo Distributed Lectures and seminarso Remote participation in panel discussionso Virtual site visits meetingso Complex distributed grid based demonstrations
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Usage AreasUsage Areas The Academic and Research, Government, Private Sectors Making Remote Collaborations Work across Boundaries Check “Multi-Sector Collaboration over the Access Grid” by J. T. von H
offman (Boston Univ)”
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Access GridAccess GridComponents of an AG NodeComponents of an AG Node
Digital Video
Digital Video
Digital Audio
NETWORK
MixerControl
Computer
NTSC Video
RGB Video
Analog Audio
Video Capture
Computer
DisplayComputer
AudioCapture
Computer
EchoCanceller
Ambient mic(tabletop)
Presentermic
Presentercamera
Audience camera
Stream
TypeMax.
latencyMax. Jitter
Min. BWMax
. Loss
Multicast
Text 100 ms N/A 64 kbps 0 % No
Audio 400 ms 60 ms 64 kbps 5 % Yes
Video 400 ms 30 ms 256 kbps x 4 = 1 Mbps25 %
Yes
Requires Multicast
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
AG extended toward ACEAG extended toward ACE
Immersive Media SupportedAdvanced Collaboration Environment
Immersive Media SupportedAdvanced Collaboration Environment
Advanced Collaboration EnvironmentAdvanced Collaboration Environment
Access Grid 2.0Access Grid 2.0
Access Grid
Enable group-to-group interactions at a distance
Improve the user experience Enable complex multi-site visual and
collaborative experiences Build on integrated grid services
architecture Use quality but affordable digital IP based
audio/video
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Broadband Content Requires Broadband Content Requires More… More…
Music CDs160 Kbps
Music
Browsing144 Kbps
½ Screen Video300Kbps
TV Shows
Full Screen600Kbps – 1.5Mbps
Live Events
High Resolution 6-8 Mbps
Pay Per View
ImmersiveMedia
> 100 Mbps
What kinds of broadband contentso High speed Internet access: WWW + Mail + Messagingo Video focused: Basic TV + Pay TV + Personalized TVo Networked games + Music downloads + Online gamblingo …
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Future Broadband Contents Future Broadband Contents Service & Immersive MediaService & Immersive Media
Interactive collaboration - Collaborative design and engineering; Remote customer support; Distance learning
Remote visualization & Large-scale, multi-site computation and data mining
Shared virtual reality - Military and industrial team training / simulation; Multiplayer games; Virtual shopping malls (e-commerce); Mobile entertainment; Online tradeshows and conferences; Virtual Heritage
Any combination of the above
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Core Techniques for Immersive Media Service Integration
Multi-modal Immersive Multi-modal Immersive Media Service TechnologyMedia Service TechnologyMulti-modal Immersive Multi-modal Immersive
Media Service TechnologyMedia Service Technology
3D Audio Generation
& Reconstruction
3D Geometry Generation & Modeling
Digital Contents Management &
Protection
Multi-D Haptic Generation & Reconstruction
Immersive Media
Compression & Integration
3D Video Synthesis &
Reconstruction
Immersive MediaDelivery
Immersive Media Storage &
Query/Browsing
Contents Service (Protocol &
System)
Immersive Media Content
Adaptation
Interactive User Interface
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Thank You!Thank You!
Send Inquiries to Send Inquiries to jongwonjongwon@@kjistkjist.ac..ac.krkr
top related