© 2005 Avaya Inc. All rights reserved.
A Client-Side Architecture for Supporting Pervasive Enterprise
Communications
A Client-Side Architecture for Supporting Pervasive Enterprise
Communications
Amogh Kavimandan†, Reinhard Klemm‡, Ajita John‡, Doree Seligmann‡, Aniruddha Gokhale†
† Vanderbilt University, Nashville TN‡ Avaya Labs Research, Basking Ridge NJ
2© 2005 Avaya Inc. All rights reserved.
Hermes: Pervasive, Context-Aware Enterprise Communications Middleware
Hermes Enterprise Communications Middleware
Business processes need to communicate with people and connect people with each otherHermes automates people communications by enabling design and execution of communications applicationsIntegration with business processesHermes communications applications connect the right people, at the right time, via the right communication media
Communications Application
Business Process
Communications Application
Communications Application
Bob
Alice Cary
ConferencingSystems
CommunicationServers
MessagingSystems
VoiceSwitches
3© 2005 Avaya Inc. All rights reserved.
launches
Expert Finder Application
Example of Hermes Communications Application
ConferenceBridge
ManagerConference
Hermes
Alice in her home office
Browser
HermesUser Portal
logs into
bridges in
sends dialog
popsup
returns filled dialog
transmits user context
Client-SideAgent
Pervasive, context-aware enterprise communications middleware requires client-side support
– Specific to communication endpoints, media
fills out dialog
4© 2005 Avaya Inc. All rights reserved.
Client-Side Challenges for Pervasive, Context-Aware Enterprise Communications (1/3)
Pervasive, context-aware communications applications make decisions about who, when, how to contact/connect people
Effective decision-making requires efficient collection of user context:
– User presence: which communication endpoint(s) is user connected to?
– User communication activity: what communication activities has user engaged in during last x time units? With whom/what is user communicating right now?
5© 2005 Avaya Inc. All rights reserved.
Client-Side Challenges for Pervasive, Context-Aware Enterprise Communications (2/3)
– User interruptibility: can user be interrupted with request for feedback?
• Computed from user presence, communication activity, nature of request, enterprise and personal rules/policies
– User availability: can user participate in new communication session? If not now, how soon?
• Computed from user feedback, for specific communication session
– Communication device characteristics: can user participate in/continue with new/ongoing communication using current communication endpoint?
• Static and dynamic characteristics
6© 2005 Avaya Inc. All rights reserved.
Client-Side Challenges for Pervasive, Context-Aware Enterprise Communications (3/3)
– Support for application-user dialogs: dialogs for collecting user availability feedback either outside or during ongoing communication session
• Notifying user of pending dialogs or barge-in
• Rendering (multiple) dialogs
• Collecting user responses
• Dialog presence
– Session management: users may be engaged in several communications sessions simultaneously and need to receive dialogs in the right session(s)
– Transparency: user should not perceive difference in performing various communication tasks despite ongoing user context collection
7© 2005 Avaya Inc. All rights reserved.
Argus: A Client-side Architecture for Supporting Pervasive Enterprise Communications (1/2)
Argus complements enterprise communications middleware on the client-side for supporting pervasive, context-aware communications applications
Negotiation between users and communications applications through Argus agents
Enterprise Communications Middleware
Endpoint 1 Endpoint 2 Endpoint 3 Endpoint 4
Argus Agent Argus Agent Argus AgentArgus Agent
SessionMgmt.
Device Monitor
SessionMgmt.
Device Monitor
SessionMgmt.
Device Monitor
SessionMgmt.
Device Monitor
8© 2005 Avaya Inc. All rights reserved.
Argus: A Client-side Architecture for Supporting Pervasive Enterprise Communications (2/2)
Argus agents are deployed with user communication endpoints
Agents provide context gathering, propagation, dialog presentation functionalities
Device monitor tracks and reports device properties of interest – e.g. battery remaining, interaction capabilities etc.
Session management associates session information with individual communication endpoints
9© 2005 Avaya Inc. All rights reserved.
Web Browsers as Communication Endpoints
Web-based user context data collection is meaningful since employees spend a lot of time interacting with enterprise Web portals that expose a variety of enterprise applications
Web browsers are universally deployed, programmable client software of choice for many user activities
Can render multimedia contents easily
Already provide infrastructure support for uploading information, rendering interaction (through forms) and returning user feedback
Firefox Web browser extension mechanism for implementing data collection functionality
Other communication endpoints to follow…
10© 2005 Avaya Inc. All rights reserved.
Argus Architecture Argus consists of two main components:
– Communication Agent: Tightly coupled with endpoint (here: browser). Responsible for user context gathering, dialog presentation, feedback collection
– Machine Agent: Mediates between user and middleware. Maintains session information, sends device characteristics to middleware
One Communication Agent per communication endpoint, but only one Machine Agent per client machine
Web Services Client
Proxy Server
Device Agent
Per endpoint instance
Machine Agent Per device
User Device
Enterprise Communication Middleware
Web ServicesClient
ProxyServer
DeviceAgent
Communication AgentAJAX
Components
User ResponseUI
ElementsActivityMonitor
11© 2005 Avaya Inc. All rights reserved.
Communication Agent Communication Agent monitors user activity and manages dialogs
– Activity Monitor
• Context data may be the current page being presented, time spent on the page, what Web application is being used etc.
• Periodically packages observed context data and sends package to Machine Agent
– User Response Component
• Makes upcalls into the AJAX components to forward the context data
• Receives dialogs to be rendered as forms to the user
– AJAX Components
• Asynchronous components
• Create server requests from JavaScript calls, return control back to extension event loop
12© 2005 Avaya Inc. All rights reserved.
Machine Agent
Single instance of Machine Agent per device provides session management for all active sessions
– Proxy Server
• Single point of contact for Communication Agents to send user context information to
• Maintains session information, dialog graphs
– Device Agent
• Currently implemented for Windows XP but can be easily ported to other operating systems
• Collects, reports static/dynamic device capabilities (device context)
• Device context helps enterprise communications middleware to adapt dialog design
13© 2005 Avaya Inc. All rights reserved.
Dialog Rendering and Injection Synchronized Multimedia Integration Language (SMIL), designed for describing and rendering interactive multimedia presentations, is used for dialogs
– Standardized development tools
– Rendering supported in Web browsers
Dialogs are associated with session ID of target browser session
Machine Agent detects and receives (through Java/Web Services calls) list of pending dialogs for active session from middleware
Communication Agent retrieves dialogs (as SMIL scripts) through HTTP requests and renders them through User Response component in Firefox extension
Web Services Client
Proxy Server
Device Agent
Machine Agent
User Device
Enterprise Communication Middleware
Web ServicesClient
ProxyServer
DeviceAgent
Communication AgentAJAX
Components
User ResponseUI
ElementsActivityMonitor
JS Calls HTML
HTTP
XML
Java Method Calls
Web Services Calls
14© 2005 Avaya Inc. All rights reserved.
User Perspective (1/2) Communication Agent implemented as Firefox extension:
– Context gathering is transparent – no perceivable change in browsing experience
– User can turn on/off context gathering at any point
Dialogs are rendered in the form of a pop-up window which the user can fill in
– May contain audio, video
15© 2005 Avaya Inc. All rights reserved.
User Perspective (2/2)
Pro
xy s
erve
r
Com
munication A
gent
Dialogs may be presented piecemeal, where each dialog depends on the outcome of the previous user response
Can be seen as a graph maintained at Machine Agent
16© 2005 Avaya Inc. All rights reserved.
Conclusion and Future Work
We presented the client-side architecture Argus for supporting pervasive, context-aware enterprise communications
Argus gathers user context information, propagates it to enterprise communications middleware, and supports multimodal dialogs, thereby accelerating decision-making in enterprise communications applications
We plan to do several user studies for quantifying usability and measure accuracy of user context data provided by Argus
Argus has been extended to work with Thunderbird email client; we are also looking at extending it to a range of communication endpoints