research on hapto-virtual environments and applicationspetriu/unimi2008-part1b... · research on...
TRANSCRIPT
Research on Research on HaptoHapto--Virtual Virtual Environments and ApplicationsEnvironments and Applications
DISCOVER 10DISCOVER 10thth AnniversaryAnniversary
School of Information Technology and Engineering School of Information Technology and Engineering (SITE)(SITE)
University of OttawaUniversity of Ottawa
www.discover.uottawa.cawww.discover.uottawa.ca
THE UNIVERSITY OF OTTAWATHE UNIVERSITY OF OTTAWA• FOUNDED IN 1848
•RESEARCH INTENSIVE UNIVERSITY: Ranked 5th in Research Intensity (2007)
•10 FACULTIES (Medicine, Law, Engineering, Science,….)
• 37,000 STUDENTS
• 1,100 FULL-TIME PROFESSORS
• 1900 FULL-TIME and PART-TIME SUPPORT STAFF
• OPERATING BUDGET : $ 700 MILLION (2007-08)
• RESEARCH BUDGET : $ 250 MILLION (2006-07)
DISCOVERDISCOVER: : DISDIStributedtributed and and COCOllaborativellaborative VVirtual irtual EEnvironments nvironments
RResearch Laboratoryesearch LaboratoryFounded in 1997Over 50 researchersDISCOVER Laboratory (530 m2): 2002
IBM DCV System with 32-Processor P595 ServerONYX 3400 with 3 graphic pipes and 12 processorsMechdyne FLEX 3-panel screenOther VR and Haptic equipment
Professors (7):Nicolas D.Georganas, DirectorEmil M.Petriu, DirectorEric DuboisAbed El SaddikShervin ShirmohammadiWonSook LeeJochen Lang
Virtual Reality as a New MediumVirtual Reality as a New Medium
Virtual Reality or Real Virtual Reality or Real VirtualityVirtuality??Originally, the term referred to 'Immersive Originally, the term referred to 'Immersive Virtual Reality.' Virtual Reality.'
the user becomes fully immersed in an the user becomes fully immersed in an artificial, 3artificial, 3--D world that is completely D world that is completely generated by a computer.generated by a computer.
The headThe head--mounted display (HMD) was the mounted display (HMD) was the first device providing its wearer with an first device providing its wearer with an immersive experience (1965)immersive experience (1965)
Virtual RealityVirtual Reality
Desktop VRDesktop VRTwo dimensional medium , used for 3Two dimensional medium , used for 3--DD
Augmented RealityAugmented Realitythe ability to see the real world and the virtual the ability to see the real world and the virtual world at the same time. world at the same time.
Immersive VRImmersive VRthe user becomes fully immersed in an artificial, the user becomes fully immersed in an artificial, threethree--dimensional world that is completely dimensional world that is completely generated by a computergenerated by a computer
CAVECAVE
CAVE ImmersionCAVE Immersion
Distributed Virtual Distributed Virtual EnvironmentEnvironment
---------------------------------------------
--------------------------------------
-------
CVE
Real-time Interaction-Between the
customers-Between customers
and salesmen-Between customers
and goods
Shared Environment-Global coordinate system-Static background-Dynamic entities
Mutual awareness-Customers can “see” and “talk” to each other in the CVE
--------------------------------------
-------
Server
DISCOVER FUNDINGDISCOVER FUNDING
DISCOVER Operating Funds Raised 1997-2007: $13,236,662
NSERC31%
Netw. Centres of Excellence
6%
Can.Res.Chairs6%Univ.Res. Chairs
2%
Ont. Centres of Excellence
19%
ORDCF15%
CANARIE13%
INDUSTRY4%
Other4%
DISCOVER Infrastructure Funds Raised 1997-2007: $22,265,134
CFI - OIT61%
ORDCF23%
IBM SUR15%
NSERC1%
DISCOVER FACILITIESDISCOVER FACILITIES
DISCOVER Flex CAVEDISCOVER Flex CAVE
DISCOVER DISCOVER ZaxelZaxel Virtual ViewpointVirtual Viewpoint
•Virtual Viewpoint™ is not a computer graphics animation but a live video recording of the full 3D shape, texture, color and sound of moving real-world objects.
DISCOVER DIVINE System: DISCOVER DIVINE System: DDesktopesktop--IImmersive mmersive VVirtual and irtual and IInteractive nteractive NNetworked etworked
EEnvironment Systemnvironment System
DISCOVER D.I.V.I.N.E.DISCOVER D.I.V.I.N.E.
Bring in the Touch Bring in the Touch -- HAPTICSHAPTICS
““Science of applying Science of applying force feedback and force feedback and tactile sensation to tactile sensation to
human interface with human interface with computers.computers.””
BMW’s iDrive
CyberGloveVirtualHand® for
MotionBuilder
CyberForce®Tactile Feedback
System
CyberGrasp™Exoskeleton
DISCOVER DISCOVER HapticHaptic DevicesDevices
NovintNovint FALCONFALCON3D Touch Workspace 4" x 4" x 4"3D Touch Workspace 4" x 4" x 4"Force Capabilities > 2 lbs Force Capabilities > 2 lbs Position Resolution > 400 dpiPosition Resolution > 400 dpiQuick Disconnect Handle < 1 second change timeQuick Disconnect Handle < 1 second change timeCommunication Interface USB 2.0Communication Interface USB 2.0Size 9" x 9" x 9"Size 9" x 9" x 9"Weight 6 lbsWeight 6 lbsPower 30 watts, Power 30 watts,
100V100V--240V,50Hz240V,50Hz--60Hz60Hz
$250$250
DISCOVER DISCOVER HapticHaptic Devices (cont.)Devices (cont.)
SensAbleSensAble Phantom Phantom DesktopDesktop
6 DOF Input. 3 DOF 6 DOF Input. 3 DOF output.output.HighHigh--Fidelity force Fidelity force feedbackfeedbackDirect parallel port Direct parallel port connectionconnectionVery impressiveVery impressiveAbout $2,000About $2,000
DISCOVER DISCOVER HapticHaptic Devices (cont.)Devices (cont.)
SensableSensable Phantom and Phantom and the the ReachinReachinDisplayDisplay
•• 3 DOF3 DOF•• Max force of 1.75 NMax force of 1.75 N•• Translational position Translational position
resolution of 23 resolution of 23 µµmm•• Very high force Very high force
resolutionresolution•• $20,000$20,000
DISCOVERDISCOVER HapticHaptic Devices (cont.)Devices (cont.)Four MPB Freedom 6SFour MPB Freedom 6S
6 DOF6 DOFHighHigh--Fidelity force Fidelity force feedbackfeedbackConnects through Connects through HRTCC box to your HRTCC box to your computercomputer$25,000 (we have 4)$25,000 (we have 4)
DISCOVERDISCOVER HapticHaptic Devices (cont.)Devices (cont.)HAPTIC MASTER
$50,000
DISCOVER DISCOVER HapticHaptic Devices (cont.)Devices (cont.)
Immersion CyberForce/ CyberGrasp/ CyberGlove
$100,000
DISCOVER RESEARCH DISCOVER RESEARCH HIGHLIGHTSHIGHLIGHTS
Multimedia and Interactive Virtual Environments
Eric DuboisEric DuboisResearch Interests:Research Interests:
Digital Signal Digital Signal ProcessingProcessing
Data CompressionData Compression
Image Processing and Image Processing and CommunicationCommunication
FIEE, FEIC,Professor and SITE Director
AbdulmotalebAbdulmotaleb El El SaddikSaddikResearch Interests:Research Interests:
HapticHaptic Audio Visual Audio Visual Environments (HAVE)Environments (HAVE)Multimedia Multimedia CommunicationsCommunicationsMultimedia TeleMultimedia Tele--surveillancesurveillanceKnowledge ManagementKnowledge ManagementInteractive Media and Interactive Media and GamesGamesCollaborative Ambient Collaborative Ambient Intelligence Systems and Intelligence Systems and Applications (CAMISA)Applications (CAMISA)
Associate ProfessorSMIEEE University Research Chair
Ambient Intelligence & TeleAmbient Intelligence & Tele--Surveillance Surveillance
imagesDB
AudioDB
VideoDB
MultimediaWeb
Services (sensory data
& enterprise/Gov. knowledgeare stored on distributed
DB2 databases
SOA-based
Distributed Environme
nt
Media features
extractionservice
Media features FusionService
TrustService
DomainOntologyService
IntelligentdecisionService
User Query InterfaceMultimo
dal User
Interface
SOAP
SOAP
Env. Sensory
DB
Enterprise 1DB
Gov. NDB
Multimodal Interaction for Search Engines (Freeware)
HAML: HAML: HapticsHaptics Applications Meta Applications Meta Language (Open Source)Language (Open Source)
HAML
Graphic Rendering Description
Haptic Rendering Description
Haptic
API Des
cript
ion
HapticDevice Description
Haptic Data Description
Application Description
QoE
Des
crip
tion
HAMLAT: HAMLAT: HapticsHaptics Authoring Tool Authoring Tool (Open Source)(Open Source)
Blender:Blender:Open sourceOpen sourceFullFull--fledged 3D fledged 3D graphical graphical rendererrendererPhysics and game Physics and game engine engine Adaptive userAdaptive user--interface interface
PlugPlug--ins incorporated in ins incorporated in Blender source codeBlender source code
HapticHaptic panel in Blender panel in Blender GUIGUIHapticHaptic rendering rendering extensionextensionImplemented in C++ Implemented in C++ using using OpenHapticsOpenHaptics and and OpenGL OpenGL
Multimodal Multimodal Handwriting Learning and Handwriting Learning and Evaluation Tool Evaluation Tool
Multimodal: Multimodal: haptichaptic, , visual, and audio visual, and audio information for each information for each letterletter
Learning mode choice: Learning mode choice: full or partial guidance, full or partial guidance, or test mode.or test mode.
Evaluation Mode: Evaluation Mode: quantitative evaluation quantitative evaluation using dynamic time using dynamic time warping warping
Language choice: Language choice: Arabic, French, English, Arabic, French, English, Japanese and SpanishJapanese and Spanish
HapticalHaptical/Graphical Password /Graphical Password (Patent)(Patent)
Reachin Display System PHANToM™
Use graphical Use graphical passwordspasswords
Increasing the Increasing the entropy and entropy and dictionary attack dictionary attack resistantresistantEasy to Easy to rememberremember
Use personal Use personal entropyentropy
Not easy to forgeNot easy to forgeUse visually hidden Use visually hidden attributesattributes
Unobservable Unobservable and shoulderand shoulder--surfing resistantsurfing resistant
Nicolas D. Nicolas D. GeorganasGeorganasResearch Interests:Research Interests:
Ambient Multimedia Ambient Multimedia Intelligence Systems (AMIS)Intelligence Systems (AMIS)Multimedia CommunicationsMultimedia CommunicationsCollaborative Virtual Collaborative Virtual EnvironmentsEnvironmentsTeleTele--HapticsHapticsWeb Web TelecollaborationTelecollaborationApplicationsApplicationsIntelligent Internet Sensors Intelligent Internet Sensors and Appliancesand Appliances
Associate Vice-President, Research (External)OC, OOnt, FIEEE, FRSC, FCAE, FEIC, Distinguished University Professor
VIRTUAL Environment for Industrial VIRTUAL Environment for Industrial Training: A Collaborative Virtual Training: A Collaborative Virtual
EnvironmentEnvironment
Nicolas D. Georganas
DISCOVER Industrial Training DISCOVER Industrial Training PrototypePrototype
Nicolas D. Georganas
Collaborative Virtual e-Commerce
federate Afederate B
HLA MiddlewareTCP/UDP/IP Network
federate C
federate A
Nicolas D. Georganas
Collaborative Haptic Tele-Surgery (Tracheotomy)
Training Simulation
DISCOVER Tele-Surgery (Trachea) Training
Nicolas D. Georganas
Eye Cataract Surgery Training Eye Cataract Surgery Training Simulation over CA*net4Simulation over CA*net4
Cataract Surgery is the replacement of Cataract Surgery is the replacement of the natural lens with an artificial onethe natural lens with an artificial oneSummary of the surgical stepsSummary of the surgical steps
1.1. Cut the cornea Cut the cornea 2.2. Bypass the irisBypass the iris3.3. Remove the lens capsuleRemove the lens capsule4.4. Destroy the lensDestroy the lens5.5. Insert the artificial lensInsert the artificial lens
Prototype on Prototype on 2D 2D Desktop DisplayDesktop Display
Nicolas D. Georganas
Prototype on 3D Stereo Prototype on 3D Stereo ReachinReachin DisplayDisplay
Nicolas D. Georganas
DISCOVER Augmented Reality (AR) DISCOVER Augmented Reality (AR) PrototypePrototype
A trainee works with a wearable computer
Where and which virtual objects should be superimposed on the real world?
A marker with a square region and some pattern inside of the square region is used
Supports multiple trainees and trainers distributed in different locations
Nicolas D. Georganas
GesturalGestural User InterfacesUser Interfaces
Nicolas D. Georganas
Search and Retrieval by Gestures Search and Retrieval by Gestures What types of gestures What types of gestures do we consider?do we consider?
Postures, signs in which Postures, signs in which the hand shape does not the hand shape does not change.change.Dynamic gestures relying Dynamic gestures relying on hand shape variations.on hand shape variations.
We did not consider We did not consider gestures based on moving gestures based on moving postures since the tracker postures since the tracker data were not stable data were not stable enough.enough.
The gesture recognition The gesture recognition has 2 levels:has 2 levels:
Detecting gesture samplesDetecting gesture samplesDetecting sequences of Detecting sequences of gestures (online gestures (online recognition).recognition). Nicolas D. Georganas
““Minority ReportMinority Report”” InterfaceInterface
VisionVision--Tracking and Gesture Tracking and Gesture Recognition with Color GloveRecognition with Color Glove
Nicolas D. Georganas & Emil M. Petriu
Multimedia and Interactive Virtual Environments
JochenJochen LangLangResearch Interests:Research Interests:
Computer graphicsComputer graphicsImageImage--based based modellingmodellingPhysicsPhysics--based based modellingmodellingDeformable Deformable modellingmodellingComputer visionComputer vision3D sensing and 3D sensing and modellingmodellingInteractive acquisitionInteractive acquisitionNavigation systemsNavigation systemsVirtual environmentsVirtual environmentsHapticsHaptics
Assistant Professor
Interactive MeasurementInteractive Measurement--Based Based ModelingModeling for Computer for Computer
GraphicsGraphics
Interactive MeasurementInteractive Measurement--Based Based ModelingModeling for Computer Graphicsfor Computer Graphics
renderingrenderingphotographphotograph
Realistic Appearance Models for
Cultural heritageMedical simulationProduct designGames
Realistic Realistic Appearance Appearance Models forModels for
Cultural heritageCultural heritageMedical Medical simulationsimulationProduct designProduct designGamesGames
Interactive Measurement-Based ModelingInteractive Interactive MeasurementMeasurement--Based ModelingBased Modeling
Modeling the Behavior of Complete Modeling the Behavior of Complete Objects Objects notnot MaterialsMaterials
ReflectanceReflectanceReflectance
ArticulationArticulationArticulation
TranslucencyTranslucencyTranslucency
DeformationDeformationDeformation
Interactive MeasurementInteractive Measurement--Based Based Modeling Modeling
Approach:Approach:Consumer computing instead of special laboratory Consumer computing instead of special laboratory equipmentequipmentLots of visual sensing Lots of visual sensing User control of acquisition and modeling process User control of acquisition and modeling process
Projects:Projects:Image segmentation as a tool for modelingImage segmentation as a tool for modelingMobile 3D modeling based on stereo visionMobile 3D modeling based on stereo visionInteractive scanning of Interactive scanning of haptichaptic surface properties and surface properties and compliancecompliance
Raster ImageRaster Image
Color Image Segmentation Color Image Segmentation Hierarchies Hierarchies MacDonald et al. 2006MacDonald et al. 2006
Image segmentation is a common first lowImage segmentation is a common first low--level image level image processing step processing step Inherently ambiguous (context and scale dependent) Inherently ambiguous (context and scale dependent) Hierarchy allows multiple simultaneous segmentationsHierarchy allows multiple simultaneous segmentations
Application Example: Image Application Example: Image VectorizationVectorization
Vector ImageVector Image
1000 Segments1000 Segments
Vector ImageVector Image
5 Segments5 Segments
Mobile Scanning of Mobile Scanning of HapticHaptic Surface Surface Properties Properties Andrews & Lang 2007Andrews & Lang 2007
Instantaneous FeedbackInstantaneous Feedback
CameraCameraHumanHuman--Object Object
InteractionInteraction
LaptopLaptop
ForceForce--Sensing PenSensing PenModelingModeling
Prof. Prof. WonSookWonSook LeeLeeResearch Interests:Research Interests:
Computer Graphics & Computer Graphics & Animation Animation Human modeling & animationHuman modeling & animationFace Recognition Face Recognition HumanHuman--Computer Interaction Computer Interaction Virtual Reality in Health Care Virtual Reality in Health Care Medical applicationsMedical applicationsMusic Analysis Music Analysis Computer Vision Computer Vision Computer Games Computer Games Graphics related eCommerce Graphics related eCommerce
Assistant Professor
Computer Graphics and AnimationComputer Graphics and AnimationHuman Modeling & AnimationHuman Modeling & Animation
HapticHaptic GameGame
Prof. WonSook Lee
Prof. WonSook Lee
3D Human Modeling: out of single 3D Human Modeling: out of single photographphotograph
Create Create a kida kidout of adult out of adult databasedatabase
Expression Databank Expression Databank : : MPEGMPEG--4 controlled face animation4 controlled face animation
Andrew Andrew Lijia Frank
Prof. WonSook Lee
Skin bank: Skin bank: Face Transplant, Face Transplant, aging and aging and rejuvenationrejuvenation
Prof. WonSook Lee
Body Animation: Body Animation: PianoPiano PedagogyPedagogy
Prof. WonSook Lee
Feel the Virtual objects and LiquidFeel the Virtual objects and Liquid::HapticHaptic Game and Fluid AnimationGame and Fluid Animation
Prof. WonSook Lee
Emil M. Emil M. PetriuPetriu
Research Interests:Research Interests:
Interactive Virtual EnvironmentsInteractive Virtual EnvironmentsRobot Sensing and Perception Robot Sensing and Perception Intelligent SensorsIntelligent SensorsNeural Networks and Fuzzy Systems Neural Networks and Fuzzy Systems Digital Integrated Circuit TestingDigital Integrated Circuit Testing
FIEEE, FCAE, FEIC Professor and University Research Chair
CSO XYZ RGB Inc.
DISCOVER Virtual Theater projectDISCOVER Virtual Theater project
VisualFeedback(s)
Machine/ Human Interface
TactileFeedback(s)
ForceFeedback(s)
VideoSensor(s)
TactileSensor(s)
ForceSensor(s)
StructuredLight
Human/Machine Interface
Object Recognition
Motion TrackingVirtual Object Manipulation
Object Shape &Behavior ModelsObject Interaction Models
Computer Generated Models Animation Script
Pseudo-Random Encoding
VIRTUAL SCENE / THEATER (where virtual objects interact)
Head-Tracking system
Body Tracking System
Facial Emotions Recognition
Biology-Based 3D Face Modeling: over 200 facial muscles and m14,000 possible expressions
3D generic face deformed using muscle-based animation based on Expression Action Units (EAUs) in an “expression space”
defined by 7 pairs of muscles + “jaw drop”
Facial Expression Recognition
Person Dependent
Person Independent
Marius D. Marius D. CordeaCordea, A 3D Anthropometric Muscle, A 3D Anthropometric Muscle--Based Active Appearance Model for ModelBased Active Appearance Model for Model--Based Video Coding, Ph.D. Thesis, 2007Based Video Coding, Ph.D. Thesis, 2007
……
Tactile Sensor Interface
HapticRobot
InterfaceROBOT(k)
OBJ(i)
Robot Arm Controller
CyberGrasp ™ CyberTouch ™
Haptic Human I nterfaceUSER (k)
NETWORK NETWORK
…
{ [ 3D(j) & F(k,j) ], t }
AVATAR HAND ( k )
. . .
. . .
OBJ (N) OBJ (1)
3D Geometric & Elastic
Composite Model of Object{( x
p, y
p, z
p,E
p) | p = 1,.., P }
OBJ ( j )
Virtual Operation TheaterVirtual Operation Theater
Composite Haptic Interaction Vector between User (k) and Object (j)
ApplicationSpecific
InteractiveAction
Scenario
…
……
Tactile Sensor Interface
HapticRobot
InterfaceROBOT(k)
OBJ(i)
Robot Arm Controller
Tactile Sensor Interface
HapticRobot
InterfaceROBOT(k)
OBJ(i) OBJ(i)
Robot Arm Controller
CyberGrasp ™ CyberTouch ™
Haptic Human I nterfaceUSER (k)
CyberGrasp ™ CyberTouch ™
Haptic Human I nterfaceUSER (k)
NETWORK NETWORK
…
{ [ 3D(j) & F(k,j) ], t }
AVATAR HAND ( k )
. . .
. . .
OBJ (N) OBJ (1)
3D Geometric & Elastic
Composite Model of Object{( x
p, y
p, z
p,E
p) | p = 1,.., P }
OBJ ( j )
Virtual Operation TheaterVirtual Operation Theater
Composite Haptic Interaction Vector between User (k) and Object (j)
ApplicationSpecific
InteractiveAction
Scenario
{ [ 3D(j) & F(k,j) ], t }
AVATAR HAND ( k )
. . .
. . .
OBJ (N) OBJ (1)OBJ (1)
3D Geometric & Elastic
Composite Model of Object{( x
p, y
p, z
p,E
p) | p = 1,.., P }
OBJ ( j )
3D Geometric & Elastic
{( xp
, yp
, zp
,Ep
) | p = 1,.., P }
OBJ ( j )
Composite Model of Object{( x
p, y
p, z
p,E
p) | p = 1,.., P }{( x
p, y
p, z
p,E
p) | p = 1,.., P }
OBJ ( j )
Virtual Operation TheaterVirtual Operation Theater
Composite Haptic Interaction Vector between User (k) and Object (j)
ApplicationSpecific
InteractiveAction
Scenario
…
VE for Interactive Haptic Telemanipluation
NN Modelling of 3D Object Shape
51096 points, 20-10-1, 5 extra surfaces, d=0.055, 2000 epochs, 5.2 hrs.
19000 points, 14-7-1, 4 extra surfaces, d=0.055, 1100 epochs, 3.3 hrs
19080 points, 10-5-1, 5 extra surfaces, d=0.055, 1200 epochs, 2.8 hrs.
Multi Layer Feed-Forward (MLFF) Neural Network
Neural Gas NN
(er= 0.0098)
SOM NN (er= 0.0125)
Initial Point-Cloud
19080 points 14914 points 13759 points
1125 points, 42 min.
1125 points, 26 min.
875 points, 11 min.
875 points, 24.5 min.
875 points 22 min.
875 points, 10 min.
Self-Organizing Map (SOM) and Neural Gas Neural Networks
NN Modelling of 3D Object Elastic Properties from Measurement Data
3D pointcloudof data
Samplepoints
Deformationprofiles
ForceMeasurements
FeedforwardNeural Network
F
profile(f0)profile(f1)profile(f2)profile(f3)
f0f1f2f3
Neural gasnetwork
Rangefinder
Force/Torquesensor
Elastic ball usedfor experimentation.
Laser range-finder based recovery of the geometric shape in an area around the
contact point between the probe and object.
Force-torque sensor measuring the interaction at the point of
contact between the robot manipulated probe and the object.
(a) (b)
Real and modeled deformation curves using neural network for rubber under forces applied at different angles: a) F=65N, α1=10° and F=65N, α2=170°, b) F=36N, α1=25°, and F=36N, α2=155
Robotic “finger” equipped with instrumented passive-compliant wrist and “fingertip”tactile probe array.
Biology-Inspired Robotic Haptic Perception
Pseudorandom code symbols Braille-like embossed on a 3D
object surface.
Composite tactile image of four symbols recovered on the encoded object surface
Model-Based Blind Robotic Tactile Recognition of Pseudorandom Encoded Objects
Multimedia and Interactive Virtual Environments
ShervinShervin ShirmohammadiShirmohammadiResearch Interests:Research Interests:
Massively Massively MultiuserMultiuser Online Online Gaming and Simulations Gaming and Simulations TeleTele--HapticsHaptics and and TelecollaborationTelecollaboration systemssystemsMultimedia Adaptation and Multimedia Adaptation and P2P Communication P2P Communication Protocols Protocols
Assistant ProfessorSMIEEE
MMMM--VISA: Massively VISA: Massively MultisuerMultisuerVIrtualVIrtual Simulation Architecture Simulation Architecture
MMOG (Massively MMOG (Massively MultisuerMultisuerOnline Gaming) for military Online Gaming) for military simulationssimulationsApplication Layer Multicasting Application Layer Multicasting and P2P to increase number of and P2P to increase number of supported users/playerssupported users/playersZoning and VisibilityZoning and Visibility--based based Filtering (reducing traffic)Filtering (reducing traffic)Dynamic Zones (to address Dynamic Zones (to address overcrowding or overcrowding or ““hotspotshotspots””))Other novel ideas unique to Other novel ideas unique to MMMM--VISAVISA
In collaboration and with the financial support of CAE.
A Hybrid (both clientA Hybrid (both client--server and Peerserver and Peer--toto--Peer) Protocol for Peer) Protocol for supporting supporting MMOGsMMOGsCurrent commercial systems Current commercial systems –– Client/ServerClient/Server
AdvantagesAdvantagesControl of powerControl of powerEasy to administer, etcEasy to administer, etc
DisadvantagesDisadvantagesScalability issue, expensive for companies, single point of failScalability issue, expensive for companies, single point of failure ure Expensive for customers (subscription fees)Expensive for customers (subscription fees)
Hybrid systemHybrid systemCombination of responsibilitiesCombination of responsibilitiesAdvantagesAdvantages
Scalable systemScalable systemCost effectiveCost effective
DisadvantagesDisadvantagesRequire to maintain the proper collaboration states and Require to maintain the proper collaboration states and Difficult to administerDifficult to administerExchange of too many messagesExchange of too many messages
DSDS--ALM: ALM: DominatingDominating--Set Based Set Based Application Layer Multicast ProtocolApplication Layer Multicast Protocol
Best Paper AwardIEEE Collaborative Peer to Peer
Systems 2007
Media Resource Server
Compressed Domain Video Adaptation, Authentication, and
Encryption
• Compressed Domain: means all processing is done without decompression and recompression!
• Incentive: delivery of audio/video to heterogeneous user devices through the Internet.
• The first work of its kind to adapt (temporal), authenticate (the sender), and encrypt the video content based on MPEG-21 gBSD, and without cascaded decoding/re-encoding. Results published as a first benchmark in this field in IEEE Multimedia.
Adaptive Adaptive QoSQoS--enabled Audio enabled Audio Watermarking Watermarking
A A single single watermarking watermarking schemescheme for: for:
Content Content proprietorshipproprietorshipContent Content authenticationauthenticationQuality of ServiceQuality of ServiceAssessmentAssessmentCombine the Combine the watermarking scheme watermarking scheme with with adaptationadaptationseamlessly.seamlessly.Both watermarking Both watermarking and adaptation can and adaptation can be done in be done in real timereal time..
Press coverage in:• The Scotsman news paper, August 14 2007, “Beating Pirates”• Science Daily, August 11 2007, “Scuppering Pirates Improves Internet Audio”• Others …
CONCLUSIONCONCLUSION
VR and VR and HapticsHaptics are new media in are new media in Multimedia applicationsMultimedia applications
Other senses (smell, taste) yet to be brought Other senses (smell, taste) yet to be brought inin
HaptoHapto--Virtual Collaborative Environments Virtual Collaborative Environments (HAVE) have already many applications in (HAVE) have already many applications in Training (particularly education, military Training (particularly education, military and medical) and will see many more in and medical) and will see many more in the futurethe future
متشکرم