of 50 e gov universal access ahmed gomaa cimic rutgers university
TRANSCRIPT
Of 50 Of 50
EGOV Universal Access
Ahmed GomaaCIMICRutgers University
2Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
3Of 50 Of 50
Universal Access
Any MM Information Over Any Network
To Any One with varying expertise, capabilities, and preferences
To Any Device
4Of 50 Of 50
EGOV Universal Access Example
The Entrepreneur can view on the EGOV portal a set of information resources specific to his type of business on any type of internet appliance with customizable preferences based on his appliance and his own preferences and capabilities.
5Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
6Of 50 Of 50
A medical Digital Library Object
• The video must start immediately after the image has been displayed .
• The text must be displayed simultaneously with the image and the video.
[Adam et al,2001]
7Of 50 Of 50
• The digital library object is downloaded to the client machine.
8Of 50 Of 50
Implementation
9Of 50 Of 50
Implementation
10Of 50 Of 50
Object Plan as MOPN
Adam et al, 2001
11Of 50 Of 50
DL Object Plan for the above medical object
Adam et al, 2001
12Of 50 Of 50
The modified plan (with only audio and text capabilities)
Adam et al, 2001
13Of 50 Of 50
Parallel and Sequential
Adam et al, 2001
Adam et al, 2001
14Of 50 Of 50
An Adjusted plan
Adam et al, 2001
15Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
16Of 50 Of 50
Issues and Problems• Issue:
– Synchronization and User interactivity .
• Problems: *In the user interaction cases:Forward / Rewind / Pause.– How to make sure that the synchronization constraints are still
valid?– How to visualize and analyze the effect of the user interaction on
the temporal constraints?
17Of 50 Of 50
Issues and Problems
• Issue:– Automatic detection of desynchronization and minimize
the desynchronization periods automatically
• Problems:– Some of the multimedia object playbackperiod is determined on the runtime (Video). So even if there are constraints on the presentation,
desynchronization will appear. How can we automatically detect that desynchronization and put it to a minimum?
18Of 50 Of 50
Issues and Problems
• Issue:– We need to make the MM information accessible based on
user preferences, expertise and capabilities.
• Problems:– The user might need to convert some MM formats to
others based on his capabilities ( text to audio)– Personalize the Multimedia presentations based on user
preferences, expertise and capabilities. The broker server issue arise for converting to different formats.
– How to select the route for converting the MM presentation?
– What will be the cost function?
19Of 50 Of 50
Presentation outline
• Universal access definition and example• Issues and problems• Methodology• Previous work• Extensions proposed• SMIL and timing• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Conclusions
20Of 50 Of 50
Extended EGOV Universal Access
PETRI-NET Modeling for validation
and analysis
•Temporal Synchronization and User
interactivity
•Automatic detection of desynchronization
•Minimize the desynchronization periods
•Personalization and accessibility
•Temporal Synchronization and User
interactivity
•Automatic detection of desynchronization
•Minimize the desynchronization periods
•Personalization and accessibility
Implementation with
user/web friendly with flexibility
and interoperability (SMIL 2.0)•Temporal Synchronization with User
interactivity
•Automatic detection of desynchronization
•Minimize the desynchronization periods
•Personalization and accessibility
•Temporal Synchronization with User
interactivity
•Automatic detection of desynchronization
•Minimize the desynchronization periods
•Personalization and accessibility
21Of 50 Of 50
Why Use Petri-Nets?
• The Above mentioned problems can be implemented by hard coded solution. ( SMIL 2.0)
BUT• How can we identify if there is a temporal or spatial
conflict in case of normal playback or even in Forward or Rewind? How can we visualize the runtime of our presentation to prevent any deadlocks, resource constraints, or network constraints?
SO• We need a model to analyze and abstract the
problems mentioned. Petri- Net has been used in the academic field as well as the industrial field to model different systems.
22Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
23Of 50 Of 50
Server
ObletAdjusted
Object Plan
N/Wdelay
Synchronization:Fine and coarse-grain
Temporal Navigation
( User Interaction)
Client
Temporal Dynamic
presentation
ExtensionsInterpretable & easy to maintain
ObjectPlan(SMIL)
Automatic adjustment
of Desync.
AccessibilityCost
Function
24Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
25Of 50 Of 50
Why SMIL 2.01. XML Based.
2. Allows you to use clips in different locations . (Separate URL for each clip)
3. Time and control a presentation. (Different approaches for timing)
4. Lay out a presentation. When your presentation includes multiple clips, such as a Real Video clip playing simultaneously with text captions or banner graphics, you use SMIL to define the layout.
5. Content Control module provides alternate presentations ( multiple languages, or different bandwidths. )
26Of 50 Of 50
SMIL2.0 To Petri Net
Equals ( SYNC):<par dur="30s"> <img id="foo"
src="a.jpg"/> <text src="text.html" /> <audio src="audio.au" /></par> |------------| image |------------| text |------------| Audio 30s
After ( Before):<seq> <img src="a.gif"
dur="6s" /> <img src="b.gif" dur="4s"
begin="1s" /> </seq>|----------| 6 sec |---------|
4sec|--------------------------|
11 seconds
27Of 50 Of 50
Temporal relations and corresponding OCPN
[Little et al, 1990]
28Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
29Of 50 Of 50
Data Retrieving Engine
[Yang, 2001]
30Of 50 Of 50
RTSM
[Yang, 2001]
31Of 50 Of 50 Of 50 Of 50
[Yang, 2001]
32Of 50 Of 50
Fast Forward operation
Of 50 Of 50 [Yang, 2001]
33Of 50 Of 50
Pre- Fetch table
Retrieval time =
(Playback period *play rate) / Estimated BW + RDdelay
Of 50 Of 50
[Yang, 2001]
34Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
35Of 50 Of 50
Model Global time
36Of 50 Of 50
User Interaction Modeling
37Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
38Of 50 Of 50
Absolute time line
• The main deficiency in the absolute time line is the indeterminism of continuous objects.
• It might take more or less than 10 seconds for the audio file to play. – How to insure synchronization? Indeterminism problem arise.
Of 50 Of 50
39Of 50 Of 50
Indeterminism and absolute time line
Of 50 Of 50 [Layaïda et al, 2002]
40Of 50 Of 50
Indeterminism Modeling
Of 50 Of 50
41Of 50 Of 50
42Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
43Of 50 Of 50
Accessibility• How to render a different
type of media to the client? For example, a blind
person do not need to download text files, instead he needs to convert it to audio format.
Three cases:• Converters on the Server.• Converters on the Client.• Converters with middleman
server
Cost
function?
Cost
function?
44Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
45Of 50 Of 50
Future Work• Issue:
– Incorporate Temporal and spatial constraints
• Problems: – What constraints shall be added to insure presentation integrity? ( Temporal and Spatial constraints )– How to synchronize between different components in a spatio-temporal fashion?
46Of 50 Of 50
Future Work
• Issue:– We need to make the different networks transparent to the
user, and making the user feel as if he is watching a local presentation. Which we call Fine and Coarse grain network delay manipulation.
• Problems:– How to estimate the bandwidth? (Known at runtime)– How to make the network delay seamless between
different networks (wireless – mobiles- cables …) can we develop a generic model that automatically adapt for changes between different networks?
47Of 50 Of 50
Future Work
• Issue:– Adaptation of Multimedia Presentations on different
hardware and software. ( OS / MM Boards / Devices ).
• Problems:– Resource constraints and device diversity. How can we
manage that the user device will be able to handle the Incoming presentation?
– How are we going to deal with the client resources? Buffer problem? Different board latency time?
48Of 50 Of 50
Server
ObletAdjusted
Object Plan
Multimedia Boards
Audio/Video
N/Wdelay
Client
Different Devices
Future WorkInterpretable & easy to maintain
ObjectPlan(SMIL)
Automatic adjustment
of Desync.
ResourceConstraints
N/WConstraints
Access Control for Timesheets ServerServer
Synchronization:Fine and coarse-grain
Temporal Navigation +
Spatial Constraints
Spatial&Temporal Dynamic
presentation
AccessibilityCost
Function
49Of 50 Of 50
Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and
minimization of desynchronization periods• Accessibility• Future Work• Conclusions
50Of 50 Of 50
Conclusion
• Converting SMIL 2.0 to Petri-net for interoperability purposes.
• Timesheets use in SMIL2.0.• User input into the Petri net model.• Forward / Rewind / Pause in a PN model.• Presenting Indeterminism and how to minimize
desynchronization for uncontrollable objects in a PN model.
• Presenting both Spatial and temporal constraints in one Petri net model ( In progress)
• Accessibility and cost function.
Of 50 Of 50