tutor.com the tutor.com classroom · tutor.com the tutor.com classroom architecture and techniques...
TRANSCRIPT
Tutor.com
The Tutor.comClassroomArchitecture and techniques using Silverlight, WPF, and the Microsoft .NET Framework
Russell Greenspan, VP Technology11/1/2008
The Tutor.comClassroomArchitecture and techniques using Silverlight, WPF, and the Microsoft .NET
Russell Greenspan, VP Technology
Architecture and techniques using Silverlight, WPF, and the Microsoft .NET
2 | P a g e
Contents
CONTENTS ............................................................................................................................................................. 2
OVERVIEW ............................................................................................................................................................. 4
HOW-TO TIPS ...............................................................................................................................................................4Code Management ...............................................................................................................................................4TCP/IP Sockets.......................................................................................................................................................4Multi-threaded Programming...............................................................................................................................4Client Applications (Silverlight/WPF) ....................................................................................................................4Silverlight ..............................................................................................................................................................4
INTRODUCTION .....................................................................................................................................................5
ABOUT TUTOR.COM .......................................................................................................................................................5THE TUTOR.COM CLASSROOM..........................................................................................................................................5OTHER RELEVANT TUTOR.COM TECHNOLOGIES....................................................................................................................5
TUTOR.COM COMMUNICATIONS PLATFORM ........................................................................................................7
CODE SHARING..............................................................................................................................................................7How-To: Share code between Silverlight and WPF applications ........................................................................................ 7
MESSAGING.SOCKETS .....................................................................................................................................................7Connecting ............................................................................................................................................................8
How-To: Connect to a TCP/IP Listener ............................................................................................................................... 8Receiving Data ......................................................................................................................................................8
How-To: Specify “End-Of-Message” to a TCP/IP Listener................................................................................................. 10Sending Data.......................................................................................................................................................10
MESSAGING.CONFERENCING..........................................................................................................................................11Message..............................................................................................................................................................11ConferenceManager ...........................................................................................................................................11MessagePackage ................................................................................................................................................11IMessageListener ................................................................................................................................................11
CLASSROOM.CONTROLMESSENGERS................................................................................................................................12CLASSROOM.CONTROLS ................................................................................................................................................12
How-To: Enforce an MVC-style architecture .................................................................................................................... 12PUTTING IT ALL TOGETHER..............................................................................................................................................13
TUTOR.COM RELAY SERVER ................................................................................................................................. 14
SAFEREADERWRITERLOCK UTILITY CLASS ..........................................................................................................................14How-To: Provide automatic reader-to-writer lock upgrade............................................................................................. 14
CONNECTION MANAGEMENT .........................................................................................................................................15Ping/Pong ...........................................................................................................................................................15
CONFERENCE MANAGEMENT .........................................................................................................................................15MESSAGE STORAGE ......................................................................................................................................................15MESSAGE TRANSLATION................................................................................................................................................17
3 | P a g e
REDUNDANCY..............................................................................................................................................................17CONCURRENCY IS KEY ...................................................................................................................................................17
CLASSROOM TOOLS ............................................................................................................................................. 18
WRAPPANEL ...............................................................................................................................................................18How-To: Create a horizontal WrapPanel that is transform-aware and exposes row and column counts........................ 18
WORKSPACE ...............................................................................................................................................................19Active Item Scrolling............................................................................................................................................19Zooming ..............................................................................................................................................................19
APPLICATION SHARING..................................................................................................................................................20WPF: Screen-scraping and Image Passing ..........................................................................................................21
How-To: Screen-scrape an application via Win32 PrintWindow() API ............................................................................. 21Browser Sharing ..................................................................................................................................................21Silverlight: Image Display....................................................................................................................................21
WHITEBOARD..............................................................................................................................................................23Capturing Mouse Input .......................................................................................................................................23
How-To: Trap and process mouse movement in an InkPresenter ................................................................................... 23Flood Fill ..............................................................................................................................................................24Decorated strokes ...............................................................................................................................................24
How-To: Create a Path object from a Stroke.................................................................................................................... 24SILVERLIGHT CONTROLS.................................................................................................................................................25
Toolbar ................................................................................................................................................................25How-To: Build a Silverlight ToolbarButton user control................................................................................................... 25
ODDS AND ENDS..........................................................................................................................................................28Dispatcher Invoking ............................................................................................................................................28Internationalization ............................................................................................................................................28
SILVERLIGHT INSTALLATION EXPERIENCE............................................................................................................. 30
DETECTION .................................................................................................................................................................30INSTALLATION..............................................................................................................................................................30
Hand Holding ......................................................................................................................................................30Browser Restart ..................................................................................................................................................31
How-To: Manually install and detect for the Silverlight plug-in ....................................................................................... 31
4 | P a g e
OverviewTutor.com’s online learning system, the Tutor.com Classroom, utilizes many of the advanced capabilities of the Microsoft .NET 3.5 platform, including TCP/IP sockets and WPF, in addition to making full use of the Silverlight browser plug-in. This white paper explores some of the architecture and implementation details in the Tutor.com Classroom and how the potential of .NET was employed.
How-To Tips
Code Managemento Share code between Silverlight and WPF applicationso Enforce an MVC-style architecture
TCP/IP Socketso Connect to a TCP/IP Listenero Specify “End-Of-Message” to a TCP/IP Listener
Multi-threaded Programmingo Provide automatic reader-to-writer lock upgrade
Client Applications (Silverlight/WPF)o Create a horizontal WrapPanel that is transform-aware and exposes row and column
counts o Screen-scrape an application via Win32 PrintWindow() APIo Trap and process mouse movement in an InkPresentero Create a Path object from a Stroke
Silverlighto Manually install and detect for the Silverlight plug-in
5 | P a g e
Introduction
About Tutor.comTutor.com is the leading provider of online learning services. Every day we connect more than 6,000 students to tutors for one-to-one homework help, and in six years have conducted close to 4 milliononline tutoring sessions.
Tutors and students interact using our proprietary, online classroom. The classroom features interactive tools like instant messaging, a two-way whiteboard, and application sharing.
The Tutor.com ClassroomThe Tutor.com Classroom system consists of the Tutor.com Communications Platform (set of shared C# libraries), two front-end, Tutor.com Classroom applications (WPF and Silverlight), and the Tutor.comRelay Server (Windows Service).
Figure 1 - Architectural Overview
Other Relevant Tutor.com TechnologiesNote that a second Windows Service (the Tutor.com Matching Server, not detailed herein) is responsible for queue management and matching functions. This system uses our Presence subsystem, also not
6 | P a g e
detailed herein, to determine each connected user’s status (available, waiting, in session, etc.) to best match student requests with available tutors.
Similarly, note that the WPF classroom also interacts with a third classroom (AJAX/Flash), briefly mentioned in the Relay Server “Message Translation” section, but otherwise not detailed herein.
7 | P a g e
Tutor.com Communications PlatformThe set of shared libraries that make up the Tutor.com Communications Platform are stacked such that each library is only aware of the libraries immediately above and below it. This allows separation of responsibility and helps to dictate an MVC-style implementation.
Code SharingSince these libraries are shared between three very different applications (Silverlight Classroomapplication, WPF Classroom application, and the Tutor.com Relay Server Windows Service), subtle differences in the code are required. These differences are implemented with “#if WPF” directives and Extension Methods that mimic expected arguments (for example, Storyboard.Begin()requires athis parameter in WPF, but not in Silverlight).
How-To: Share code between Silverlight and WPF applicationsTo actually share the code, use compile-time code sharing by choosing “Add As Link” when adding project files, rather than using our source code management’s “Share”. Using the “Add As Link” technique ensures that as changes are made to shared code, successfully building the solution guarantees that the changes are valid across all three applications. Source code management “Share” requires developers to check in their changes, do a “Get Latest Version” on the shared files, rebuild the solution, make any necessary changes, repeat, etc. In the meantime, the build is potentially broken.
Messaging.SocketsThis is the layer that talks TCP/IP sockets. It is concerned with data in terms of bytes. Our SocketClient class wraps the Socket and SocketAsyncEventArgs classes from the .NET 3.5 System.Net.Sockets namespace. .NET 3.5 introduced the SocketAsyncEventArgs class, which provides a reusable, event-driven sockets architecture.
Events raised from our SocketClient indicate socket success/failure operations, such as Connected and DataReceived.
Figure 2 - Messaging.Sockets.SocketClient
8 | P a g e
ConnectingConnecting to a TCP/IP Listener (in our case, the Tutor.com Relay Server) simply involves specifying a RemoteEndPoint to connect to.
How-To: Connect to a TCP/IP ListenerUsing the Socket class, we just need to create a SocketAsyncEventArgs and call ConnectAsync():
public void Connect(string host, int port) {
//instantiate socketm_Socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream,
ProtocolType.Tcp);
//set up endpoint#if WPF
m_SocketSendArgs.RemoteEndPoint = new IPEndPoint(Dns.GetHostAddresses(host)[0], port);
#elsem_SocketSendArgs.RemoteEndPoint = new DnsEndPoint(host, port);
#endif
//set up argsSocketAsyncEventArgs args = new SocketAsyncEventArgs();
args.UserToken = m_Socket;args.RemoteEndPoint = m_SocketSendArgs.RemoteEndPoint;args.Completed += new EventHandler<SocketAsyncEventArgs>(OnConnect);
m_Socket.ConnectAsync(args);}
Notice an example of the “#if WPF” directive for subtle Silverlight/WPF differences (as described in the Code Sharing section above) like setting the SocketAsyncEventArgs RemoteEndPoint.
Receiving DataOnce connected, clients then call SocketClient.Start() to begin listening.
As data is received, Receive_Completed() is automatically called by .NET 3.5 from a thread pool thread (we set up the event listener when we created the SocketAsyncEventArgs class). We use our
public void Start(){
//begin receiving//read first 8 bytes to get message and byte array lengthsm_BufferProcessor.Reset();
m_SocketReceiveArgs.SetBuffer(m_BufferProcessor.ActiveBuffer, 0, m_BufferProcessor.BytesRemaining);
//begin receivingm_Socket.ReceiveAsync(m_SocketReceiveArgs);...
}
9 | P a g e
BufferHelper helper class to set and increment counters based on the number of bytes that have been read, and we return a BufferHelper.ReceivedBytesStatus indicating the current status of the receipt process. When all expected bytes have been read from the stream, we notify listeners via the DataReceived event. Notice that after firing the DataReceived event, we call Start() to begin the process again.
Also notice the check for e.BytesTransferred == 0; this is ONE potential way we will be notified when the remote client has disconnected. Others include SocketDisposed or ObjectDisposed exceptions.
private void Receive_Completed(object sender, SocketAsyncEventArgs e){
if (e.BytesTransferred == 0 || e.SocketError != SocketError.Success){
//close connection ...
}else{
//did we get the # of bytes we were expecting?switch (m_BufferProcessor.ReceivedBytes(e.BytesTransferred)){
case BufferHelper.ReceivedBytesStatuses.Incomplete:
//keep listening for the # of bytes we still needm_Socket.ReceiveAsync(e);...
break;
case BufferHelper.ReceivedBytesStatuses.LengthsComplete:
//good to go... processor now has # of bytes to expect//point m_Socket handler to newly created buffer and
continue receivinge.SetBuffer(m_BufferProcessor.ActiveBuffer, 0,
m_BufferProcessor.BytesRemaining);m_Socket.ReceiveAsync(e);...
break;
case BufferHelper.ReceivedBytesStatuses.DataComplete:
//we've received all the bytes in the messageif (this.DataReceived != null)
this.DataReceived(ref m_BufferProcessor.StringData, m_BufferProcessor.ByteArrayData);
//start listening againStart();
break;}
}}
10 | P a g e
How-To: Specify “End-Of-Message” to a TCP/IP ListenerSince a TCP/IP Listener is constantly in a listening state, the Listener needs to know when to act on the bytes received. To accomplish this, listen first for 8 ASCII bytes that represent two 32-bit integers specifying the number of bytes the sender is sending, and then continue listening until receiving this number of bytes. This is an easier solution to implement than using a delimiter to specify end-of-message, since you don’t have to worry about delimiter escaping, and since you can specify the exact number of bytes to listen for via the third parameter to SocketAsyncEventArgs.SetBuffer().
Sending DataSince so much of what we pass between users is string data, we allow callers to pass in both a stringand a byte[], and the SendData() function converts the string to a UTF-8 encoded byte array for sending. Note that this string is passed in by reference, so that we avoid copying it throughout the call stack.
We then simply implement the reverse of the process described in the “Receiving Data” section: wedetermine the length in bytes of the two byte arrays to send, create two 4-byte integer representations of these lengths, concatenate these 8 bytes with the actual data to send, and push the concatenated byte stream onto the socket. Note that we also have to deal with cross-thread concurrent access to the socket, so we use an AutoResetEvent that switches back to “ready” state when SendData_Completed is fired.
public void SendData(ref string Data, byte[] ByteArray){
//convert string data to UTF-8byte[] stringByteArray = Encoding.UTF8.GetBytes(Data);
//get length of our message and length of our byte array...
//copy to single byte array for sendingbyte[] finalArray = new byte[dataLengths.Length + stringByteArray.Length +
((ByteArray == null) ? 0 : ByteArray.Length)];
//wait for signaled state...
//set to non-signaled, so other threads wait to send...
//now send messagem_SocketSendArgs.SetBuffer(finalArray, 0, finalArray.Length);if (m_Socket.SendAsync(m_SocketSendArgs) == false){
//handle case where sendAsync returns true if I/O operation is pending (i.e. SendData_Completed event will be raised)
...}
}
11 | P a g e
Messaging.ConferencingThe Messaging.Conferencing library bridges the logical/physical divide, by organizing raw bytes into meaningful objects.
MessageThe Message class provides static functions to serialize/ deserialize byte streams into well-defined Message objects,which have easy-to-use properties like DateSent, MessageBody, etc.
ConferenceManager The ConferenceManager class provides methods like JoinConference() to manage logical conference membership and SendMessage() to communicate with conference members, and raises events like ConferenceJoined and ConferenceMemberAdded.
MessagePackageThe MessagePackage class compresses/decompresses a set of messages for future playback.
Figure 3 -Messaging.Conferencing.ConferenceManager
IMessageListenerTo listen for specific messages, interested parties implementIMessageListener and attach themselves to the ConferenceManager via AddMessageListener().
Figure 4 -Messaging.Conferencing.IMessageListener
12 | P a g e
Classroom.ControlMessengersClasses in this library serve as MVC-style controllers (thus the “Control” from “ControlMessengers”) for shared classroom tools, like the Avatar or Whiteboard. They also implement IMessageListener to send, receive, and process messages(thus the “Messenger” part of “ControlMessengers”), and they coordinate between tools and the ConferenceManager by listening on tools’ INotifyPropertyChanged or custom events.
As an example, consider the AvatarMessenger class, which is the controller responsible for receiving and sending avatar state changes. Figure 5 -
Control.ControlMessengers.AvatarMessenger
Classroom.ControlsClasses in this library generally derive from some derivative of FrameworkElement (e.g. UserControl), and as such are pure front-end controls (Views). They have no knowledge of the messaging system or classroom containers. They communicate to controllers and containers through dependency properties or by raising INotifyPropertyChanged or custom events. Some are custom controls implemented in code-only, and others are user controls implemented with Xaml markup.
As an example, consider the Avatar class.
How-To: Enforce an MVC-style architectureThe benefits of MVC-style architecture are numerous: testability, modifiability, simplicity of design, etc., and WPF/Silverlight Xaml “binding” provide an ideal structure to effectively detach the front-end from middle-tier business objects.
One simple way to enforce MVC-style architecture is to put classes with distinct responsibilities in separate assemblies, and add the minimal set of references only as required by each assembly. This enforces the principle of separation of concerns, since you’ve actually made it impossible for code in one assembly to be concerned with the business of another if it shouldn’t!
For example, there is no way for the Avatar class (a pure front-end View control) to communicate with the Relay Server, since the assembly it lives in does not have a reference to
Figure 6 - Classroom.Controls.Avatar
AvatarMessengerClass
Properties
AvatarConferenceMemberId
Methods
MessageReceived
IMessageListener
13 | P a g e
Messaging.Conferencing. In order to communicate, it must raise events that a corresponding Controller in Classroom.ControlMessengers listens on.
Putting it all togetherFollowing this all the way through, let’s examine the sequence when a student changes his avatar’s state:
Client #1 Server Client #21. Button click event on
AvatarHappyButton in AvatarManager is trapped and raised to its AvatarManagerController.
2. AvatarManagerControllercreates and fills a Message and sends it via ConferenceManager.SendMessage().
3. SendMessage() calls SocketClient.SendData(), which pushes the message’s bytes up the stream.
4. The Relay Serverreceives this message for the SocketClientattached to the sender.
5. The Relay Serveriterates through its list of participants in the sender’s conference, relaysthe message, and stores it locally.
6. SocketClient receives specified number of bytes and firesDataReceived event.
7. ConferenceManager receives this event and notifies all interested IMessageListener objects of the message.
8. An instance of an AvatarManager, which has previously attached to the ConferenceManager for the AvatarChanged message type, processes the message and sets the State property of the Avatarcontrol it is controlling.
14 | P a g e
Tutor.com Relay ServerThe Tutor.com Relay Server’s function is two-fold: to receive, store, and process references to incoming System.Net.Sockets.TcpClient connections, and to “relay” messages between clients in the same conference.
SafeReaderWriterLock utility classSince proper locking is the key to success when dealing with shared data structures in a multithreaded system, we created custom SafeReaderWriterLock and SafeLockCookie classes that wrap System.Threading.ReaderWriterLock and LockCookie. The benefit is that our custom classesprovide automatic lock upgrade (when a read lock is held and AcquireWriterLock() is called) and downgrade. This class is used through the Relay Server whenever shared data structures are accessed.
Figure 7 - SafeReaderWriterLock
How-To: Provide automatic reader-to-writer lock upgradeCreate a class that contains a module-level System.Threading.ReaderWriterLock (e.g. m_Lock)and wrap the AcquireReaderLock() function. Then create an AcquireWriterLock() function that returns a LockCookie wrapper (SafeLockCookie), checks to see if the read lock is held, and upgrades if necessary, and a ReleaseWriterLock()function that processes the SafeLockCookie and releases the lock or downgrades back to read:
public SafeLockCookie AcquireWriterLock(){
//if we have a read lock, upgradeif (m_Lock.IsReaderLockHeld){
return new SafeLockCookie(m_Lock.UpgradeToWriterLock(TIMEOUT_MS));}else{
m_Lock.AcquireWriterLock(TIMEOUT_MS);return null;
}}
SafeReaderWriterLockClass
Methods
AcquireReaderLockAcquireWriterLockReleaseReaderLockReleaseWriterLock
15 | P a g e
public void ReleaseWriterLock(SafeLockCookie SafeLockCookie){
//do we need to downgrade?if (SafeLockCookie != null){
m_Lock.DowngradeFromWriterLock(ref SafeLockCookie.LockCookie);}else{
m_Lock.ReleaseWriterLock();}
}
Then have calling functions request write locks by storing a SafeLockCookie when requesting the lock and returning it when releasing:
SafeLockCookie lc = m_Lock.AcquireWriterLock();try{
...}finally{
m_Lock.ReleaseWriterLock(lc);}
Connection ManagementWhen a TCP request comes in, the Relay Server receives and stores a reference to the System.Net.Sockets.TcpClient connection.
Ping/PongSince no TCP event is guaranteed to fire if the connection is unexpectedly broken, we use application-level ping messages between the client and the server; on the server side, a 30 second timer removes clients who have failed to update their ping time within the allotted time, and on the client side, we auto-reconnect to the server if we have not received ping responses from the server within 30 seconds.
Conference ManagementThe Relay Server maintains a list of logical conferences and conference participants, so that it can relay conference messages from sender to each conference participant as messages are received.
Message StorageMessages are stored in memory and flushed to a database using three cache levels. The system works as follows:
16 | P a g e
1. As messages are relayed, we write the message to a message package (List<Message>) in“Level0”.
2. When a conference ends, and for each message package in “Level0” every 60 seconds, we move the message package to “Level1” (copy to “Level1” and remove from “Level0”). This permits relayers to start adding more messages to a new Level0 package immediately.
3. We then save each “Level1” List<Message> package to the database. On success, we remove the “Level1” package, and on failure (e.g. database down), we move the “Level1” package to “Level2” and attempt to save again 60 seconds later.
Since we potentially have many users relaying messages while this process in occurring, proper locking of each message package is paramount during this process. Notice how we use our SafeReaderWriterLock class and lock both the “Level0” and “Level1” Dictionary<string, MessagePackage> objects during the transfer (step #2 above), and how we release these locks as soon as the transfer is complete. This allows relayers to begin adding messages to “Level0” as soon as possible.
Also note the use of lock (m_ConferencePackages[ConfId]) inside the lock on ConferencePackagesLock. This is because the lock on ConferencePackagesLock is used to ensure the outer collection does not change (i.e. no more conferences are added/removed), and the lock(m_ConferencePackages[ConfId]) is used to ensure the inner collection does not change (i.e. messages are not added) while we are transferring.
private void saveMessages(){
...
//move to m_ConferencePackagesL1; acquire both L0 and L1 locks here to ensure that if we can't lock both, we move on and try again later
SafeLockCookie lc = m_ConferencePackagesLock.AcquireWriterLock();try{
SafeLockCookie lcL1 = m_ConferencePackagesL1Lock.AcquireWriterLock();try{
//lock on our specific conference so more messages are not added while we are transferring
lock (m_ConferencePackages[ConfId]){
//add messages to L1...
}}finally{
m_ConferencePackagesL1Lock.ReleaseWriterLock(lcL1);}
//transfer complete; now remove from L0 packagem_ConferencePackages.Remove(ConfId);
}finally{
m_ConferencePackagesLock.ReleaseWriterLock(lc);
17 | P a g e
Message TranslationNote that as mentioned earlier, a legacy classroom (AJAX/Flash), with a database-driven relay system, is also integrated. To make this possible, the Relay Server does two things: translate messages to and from “legacy” format via classes from the Tutor.com Communications Platform (via serialization) and message re-writing, and read from and write to the database relay system.
RedundancySince each Relay Server stores its list of connections in the database as part of the Presence subsystem (not detailed herein), each server knows which server any given client is connected to. When we need to relay a message to a conference participant who is connected to a second Relay Server, the relaying server creates a server-to-server connection and relays the message to the second Relay Server, which then relays the message to the other party.
This effectively allows for an n-redundancy model, where we can easily scale the number of Relay Servers as demand grows, just by having clients call a web service to ask which Relay Server to connect to prior to connecting.
Concurrency is KeyHow much concurrency do we have? At peak time, we currently do about 500 concurrent sessions. This equates to about 16,000 messages per minute (5MB/minute), which is about 265 messages per second(80KB/sec).
}
//we've successfully moved our messages from L0 to L1 and we're ready to save from L1 to DB
lock (m_ConferencePackagesL1[ConfId]){
m_ConferencePackagesL1[ConfId].Save(m_HWHSessionConnStr,delegate //success delegate (database save complete){
//remove from m_ConferencePackagesL1...
},delegate(Exception ex) //exception delegate{
//move to failed package collection...
});
}}
18 | P a g e
Classroom Tools
WrapPanelOur WrapPanel differs from the framework WrapPanel in a few ways. First, in early builds of Silverlight there was no wrap panel, so rolling our own was the only option. Second, ours reports back the number of rows and columns that the panel is currently displaying, which we use to determine the offset to scroll to when changing the active item. Third, it is aware of any ScaleTransform transforms affecting it, so it appropriately accommodates space based on the state of its current transform.
How-To: Create a horizontal WrapPanel that is transform-aware and exposes row and column countsBegin by deriving from Panel, which is the base class for container controls, and override the MeasureOverride() and ArrangeOverride() functions. These functions are called iteratively as the layout engine determines how to distribute available visual space. Have them call a custom function that takes either the available size (from the measure pass) or the final size (from the arrange pass), and a ShouldArrange parameter to specify which pass is processing, and returns a Size.
private Size measureAndOptionallyArrangeItems(Size size, bool ShouldArrange){
Point point = new Point(0, 0);Size s = new Size(size.Width, 0);
//consider scale transform that might be affecting usdouble xfactor = 1;double yfactor = 1;if (this.RenderTransform != null){
if (this.RenderTransform is ScaleTransform){
ScaleTransform st = (ScaleTransform)this.RenderTransform;
xfactor = st.ScaleX;yfactor = st.ScaleY;
}}
double largestHeight = 0.0;
this.Rows = 0;this.Cols = 0;
foreach (UIElement child in Children){
if (child.DesiredSize.Height > largestHeight)largestHeight = child.DesiredSize.Height;
//first row?if (this.Rows == 0)
this.Rows = 1;
double desiredWidth = child.DesiredSize.Width;
19 | P a g e
//does this child cause us to wrap?if (point.X > 0 && point.X + desiredWidth > size.Width * (1 / xfactor)){
this.Rows++;
s.Height += largestHeight * yfactor;
//goto the next linepoint.X = 0;point.Y += largestHeight;largestHeight = child.DesiredSize.Height;
if (ShouldArrange)child.Arrange(new Rect(point, new Point(point.X +
desiredWidth, point.Y + child.DesiredSize.Height)));
//set our current location in this new linepoint.X = desiredWidth;
}else{
if (ShouldArrange)child.Arrange(new Rect(point, new Point(point.X +
desiredWidth, point.Y + child.DesiredSize.Height)));
point.X = point.X + desiredWidth;
//if we're doing first row, set columnsif (this.Rows == 1)
this.Cols++;}
}
s.Height += largestHeight * yfactor;
return s;}
WorkspaceThe Workspace tool is responsible for navigation of the active classroom tools. It provides animated scrolling from item to item, and a zoom-able wrap panel that allows users to get an overview of all container controls or a zoomed-in view of a single control.
Active Item ScrollingUnder the hood, the Workspace makes heavy use of our custom WrapPanel (described above) which is placed inside a ScrollViewer; to achieve the smooth animation as the active item is changed, we simply call ScrollToVerticalOffset() on the ScrollViewer using a thread pool thread timer.
ZoomingSo to achieve the zoom effect, we use a ScaleTransform on our custom WrapPanel. Note that this not only allows both an overview and a zoomed view, but also allows us to provide automatic scaling based on screen resolution. This means that we can automatically resize and scale the active tool to fill the available screen space. And since this effect is achieved using a simple transform, all tools are able to receive and process input while the Workspace is in a zoomed state.
20 | P a g e
Figure 8 - 1024x768, zoomed out
Figure 9 - 1920x1200, zoomed in (notice the size of the whiteboard relative to the avatars)
Application SharingThe ApplicationSharing tool provides desktop application sharing from the WPF Tutor.com Classroom to the Silverlight Tutor.com Classroom.
21 | P a g e
WPF: Screen-scraping and Image PassingIn the WPF application, we make calls to the PrintWindow() Win32 API to gather screen contents. We then splice the return bitmap into a 4x4 grid and compare hashes of each of the slices with the previous slices’ hashes. If a slice has changed, we package up its bytes and ship it across the wire.
There are two key components to making this process of screen scrape, comparison, and messaging fast enough to run each second without overburdening the CPU. The first is that the entire process runs on a thread pool thread via a timer, which frees up the GUI. The second is that our Bitmap uses the Format32bppArgb pixel format, which uses more memory but optimizes performance of PrintWindow() calls.
How-To: Screen-scrape an application via Win32 PrintWindow() API[DllImport("user32.dll", SetLastError = true)]static extern bool PrintWindow(IntPtr hwnd, IntPtr hDC, uint nFlags);
void processCapture(){
using (Graphics g = Graphics.FromImage(captureBitmap)){
//get window contentIntPtr hdc = g.GetHdc();try{
result = PrintWindow(m_Win32HostHandle, hdc);}finally{
g.ReleaseHdc(hdc);}
//process captureBitmap ...
}
}
Browser SharingBrowser sharing via the embedded browser (a System.Windows.Forms.WebBrowser inside a System.Windows.Forms.Integration.WindowsFormsHost) is a slightly more complicated process, since WPF and embedded Win32 controls do not play as nicely together as you’d like. For one, Win32 controls do not respect WPF container boundaries (they are always “on top”), so you do not get automatic control boundary cropping as you would expect. Secondly, we’re now screen scraping our own application, so we need to trim out everything but the web browser; however, determining exactly where the browser begins and ends is a bit tricky since it is a Win32 component.
Silverlight: Image DisplayOn the Silverlight side, we receive these bytes via our ImageDisplayGridMessenger, which then alerts the ImageDisplayerGrid it is controlling of the new image to display.
22 | P a g e
Figure 10 - WPF Classroom, screen-scraping and image passing
Figure 11 - Silverlight Classroom, image display
23 | P a g e
WhiteboardThe Whiteboard control contains an InkPresenter, which provides an easy-to-use container forStroke objects. Additionally, the Whiteboard derives from Canvas, so we can add shape objects like Rectangle and Ellipse directly to its Children collection, and position them using Canvas.SetTop() and Canvas.SetLeft().
We needed to create our own Whiteboard control instead of using the WPF InkCanvas control (which provides some similar functionality) for two reasons. First, there is no InkCanvas in Silverlight, so rolling our own was required for Silverlight support. Secondly, having more fine-grained access allows us to better manipulate objects (shapes and strokes) and corresponding whiteboard operations (resize, move, undo, redo).
Capturing Mouse InputTo process mouse input, we need to listen on the MouseDown, MouseMove, and MouseDown events fired by the Canvas we derive from.
How-To: Trap and process mouse movement in an InkPresenterIn the MouseDown event, instantiate a module-level Stroke object (e.g. m_DrawingStroke) to collect points traversed by the mouse. Fire the StrokeAdded event so that listeners can prepare for the operation as necessary. Most importantly, call CaptureMouse() to notify the mouse input engine to give you high frequency mouse event notifications.
void Whiteboard_MouseLeftButtonDown(object sender, MouseButtonEventArgs e){
//get our current positionPoint p = e.GetPosition(this);
//create a new styluspoint collection for our new strokeStylusPointCollection coll = new StylusPointCollection();coll.Add(new StylusPoint() { X = p.X, Y = p.Y });
m_DrawingStroke = new Stroke(coll);
//fire notification eventif (this.StrokeAdded != null)
this.StrokeAdded(m_DrawingStroke);
//begin capturing mouse inputthis.CaptureMouse();
}
In MouseMove, add the traversed point to the Stroke’s StylusPoints collection, and fire the StrokeChanging event to alert interested listeners.
void Whiteboard_MouseMove(object sender, MouseEventArgs e){
//get our current positionPoint p = e.GetPosition(this);
//add the traversed point to our collectionStylusPoint sp = new StylusPoint() { X = p.X, Y = p.Y };
24 | P a g e
m_DrawingStroke.StylusPoints.Add(sp);
//fire notification eventif (this.StrokeChanging != null)
this.StrokeChanging(m_DrawingStroke);}
In MouseUp, release mouse capture and fire the StrokeComplete event.
void Whiteboard_MouseLeftButtonUp(object sender, MouseButtonEventArgs e){
//release mouse capturethis.ReleaseMouseCapture();
//fire notification eventif (this.StrokeComplete != null)
this.StrokeComplete(m_DrawingStroke);}
Flood FillFlood fill is particularly challenging in a vector-based, multi-layered drawing surface like ours, since we do not have a fixed bitmap from which to scan pixels in search of boundaries. Instead, we keep an in-memory bitmap approximation of each pixel in our 800x400 drawing surface, and we update the bitmap as strokes are added. When the user then goes to flood fill, we perform the boundary search using the in-memory bitmap approximation, find the minimal set of enclosing points, and create a filled Pathinside the boundaries.
Decorated strokesSince Silverlight Stroke objects do not expose a StrokeDashArray, to accommodate dashed and dotted strokes and lines we change the drawn Stroke to a Path object and set the Path’sStrokeDashArray property.
How-To: Create a Path object from a StrokeSimply transfer Stroke stylus points to a PolyLineSegment on a Path:
static Path getPathFromStroke(Stroke str){
//make a path from this strokePath path = new Path();PathGeometry pg = new PathGeometry();
//start our figure from the first stylus point in the strokePathFigure fg = new PathFigure();fg.Segments = new PathSegmentCollection();fg.StartPoint = new Point(str.StylusPoints[0].X, str.StylusPoints[0].Y);
PolyLineSegment seg = new PolyLineSegment();
//add each additional stylus point to the line segmentfor (int x = 1; x < str.StylusPoints.Count; x++){
StylusPoint sp = str.StylusPoints[x];
25 | P a g e
seg.Points.Add(new Point(sp.X, sp.Y));}
fg.Segments.Add(seg);
pg.Figures = new PathFigureCollection();pg.Figures.Add(fg);
path.Data = pg;
return path;}
Silverlight Controls
ToolbarOne of the controls missing out-of-the gate from Silverlight is the Toolbar. There is no magic to the WPF Toolbar (it’s just a control container that auto-docks itself and auto-styles its children), but for free you get the flat button look-and-feel that users expect at the top of an application.
In Silverlight, we have to manually create this look-and-feel. Since our application’s toolbar consists solely of buttons (both normal and toggle buttons), and since the buttons in our toolbar have the “icon followed by a text description” format, we can abstract this to build the ToolbarButton user control.
How-To: Build a Silverlight ToolbarButton user controlBegin by creating a new user control. Define the styles for the Hover and Pressed states in your user control’s Xaml resources:
<UserControl.Resources><LinearGradientBrush x:Key="HoverBrush" StartPoint="0,0" EndPoint="0,1">
<LinearGradientBrush.GradientStops><GradientStop Color="White" /><GradientStop Color="#F8D28F" Offset=".3" />
</LinearGradientBrush.GradientStops></LinearGradientBrush>
<LinearGradientBrush x:Key="PressedBrush" StartPoint="0,0" EndPoint="0,1"><LinearGradientBrush.GradientStops>
<GradientStop Color="#F8D28F" /><GradientStop Color="White" Offset=".3" />
</LinearGradientBrush.GradientStops></LinearGradientBrush>
</UserControl.Resources>
Then, use a Grid to hold two Border objects; one contains the image and the text label, and the other is a half-opaque, white filled Border that we use when the button is disabled.
<Grid><Border x:Name="MainBorder" BorderThickness="1" CornerRadius="1"
MouseEnter="Border_MouseEnter" MouseLeave="Border_MouseLeave"MouseLeftButtonUp="Border_MouseLeftButtonUp" Cursor="Hand" Margin="1">
<StackPanel Orientation="Horizontal" Margin="5,1,5,1">
26 | P a g e
<Image x:Name="BorderImage" Width="16" /><TextBlock x:Name="BorderTextBlock" Style="{StaticResource
blackText}" Margin="5" /></StackPanel>
</Border>
<Border Background="#B2FFFFFF" x:Name="DisabledBackground" BorderThickness="1"CornerRadius="2" BorderBrush="#B2FFFFFF" Visibility="Collapsed" /></Grid>
Next, wire up the IsEnabled, ToolTip, ImageSource, and Caption properties, and implement the hover and click functionality. Note that these properties are backed by dependency properties on the objects we defined in our Xaml.
public partial class ToolbarButton : UserControl{
bool m_IsEnabled = true;public new bool IsEnabled{
get { return m_IsEnabled; }set{
m_IsEnabled = value;
MainBorder.Background = null;MainBorder.BorderBrush = null;DisabledBackground.Visibility = (value) ? Visibility.Collapsed :
Visibility.Visible;}
}
public string ToolTip{
get { return ToolTipService.GetToolTip(MainBorder).ToString(); }set { ToolTipService.SetToolTip(MainBorder, value); }
}
public ImageSource ImageSource{
get { return BorderImage.Source; }set { BorderImage.Source = value; }
}
public string Caption{
get { return BorderTextBlock.Text; }set { BorderTextBlock.Text = value; }
}
public bool IsToggleButton { get; set; }public bool IsToggled { get; set; }
public event MouseButtonEventHandler Clicked;
public ToolbarButton(){
InitializeComponent();}
27 | P a g e
private void Border_MouseEnter(object sender, MouseEventArgs e){
//if we're a toggle button and we're on...if (this.IsToggleButton && this.IsToggled)
return;
Border b = sender as Border;
b.Background = Resources["HoverBrush"] as Brush;b.BorderBrush = new
SolidColorBrush(ColorHelper.FromHexString("FFCA8103"));}
private void Border_MouseLeave(object sender, MouseEventArgs e){
//if we're a toggle button and we're on...if (this.IsToggleButton && this.IsToggled)
return;
Border b = sender as Border;
b.Background = null;b.BorderBrush = null;
}
private void Border_MouseLeftButtonUp(object sender, MouseButtonEventArgs e){
if (!m_IsEnabled)return;
if (this.IsToggleButton){
this.IsToggled = !this.IsToggled;
//set stateBorder b = sender as Border;b.Background = (this.IsToggled) ? Resources["PressedBrush"] as
Brush : Resources["HoverBrush"] as Brush;}
if (this.Clicked != null)this.Clicked(sender, e);
}}
Finally, in your container Xaml, use the ToolbarButton class inside a StackPanel with Orientationset to Horizontal:
<StackPanel Orientation="Horizontal">
<local:ToolbarButton ImageSource="/Resources/board.png" Caption="Add Whiteboard" Clicked="AddWhiteboard_Click" Margin="1" />
...
</StackPanel>
28 | P a g e
Odds and Ends
Dispatcher InvokingAmong the most frequent actions we take is switching from a worker thread to the GUI thread, either when a function is called-back from a thread pool thread or a timer. When this occurs, you must call Dispatcher.BeginInvoke() to post an operation to the GUI thread. The hidden CheckAccess()
helper function will let you know if code you are executing is currently on the GUI thread:
...
//forward call to GUI thread if necessaryif (this.CheckAccess() == false){#if WPF
Dispatcher.BeginInvoke(System.Windows.Threading.DispatcherPriority.Normal,(Action)delegate
#elseDispatcher.BeginInvoke(
delegate#endif
{...
});
return;}
...
Note the syntactical difference in WPF and Silverlight.
InternationalizationWe anticipated problems with datetime variable serialization, and so we serialize using the “en-US” format and deserialize using a specific ICultureInfo provider:
static System.Globalization.CultureInfo usCultureInfo = newSystem.Globalization.CultureInfo("en-US");
public Message(ref string MessageXml, byte[] MessageData){
...
//deserialize creation DateTime by specifying US cultureDateTime creationTime;if (DateTime.TryParse(reader.GetAttribute("CT"), usCultureInfo,
System.Globalization.DateTimeStyles.AssumeUniversal, out creationTime))this.CreationTime = creationTime.ToLocalTime();
...}
29 | P a g e
However, among the first problems we encountered was that some international users were throwing errors when deserializing Double variables. It turns out these users’ culture formats doubles with a comma (e.g. “###,#” instead of “###.#”), and our application was not accounting for this possibility when deseriailizing Double variables.
Rather than be forced to specify this format on each conversion from String to Double, we went with a brute-force force solution: force the application to use the “en-US” culture setting. This works for usfor two reasons: we have no need for internationalization support (i.e. we do not display any currency or date variables), and all deserialization winds up on the GUI thread (so we can directly instantiate GUI objects). We do this when the application first loads:
...
System.Threading.Thread.CurrentThread.CurrentCulture = newSystem.Globalization.CultureInfo("en-US");
...
30 | P a g e
Silverlight Installation Experience
DetectionSince we wanted to completely control the end-user experience, before sending users to the page containing our Silverlight application, we send users to a Silverlight detection page. This page does two things via JavaScript: detect for the presence of the Silverlight plug-in, and call out to third-party analytics systems (like Google Analytics). If the correct version of the plug-in is installed, we send usersto the Silverlight application page, and if not, we send them to the installation page.
InstallationThe installation page affords a completely controlled end-user installation experience. We present the need to install Silverlight, along with the benefits that installation provides.
Figure 12 - Installation Page
Hand HoldingThe ‘Get Microsoft Silverlight’ button simply links to the installer on Microsoft’s site. After the user clicks this button, we swap out the classroom graphic for a browser/OS-dependent one that shows screenshots of the exact steps a user will have to take during the installation process.
31 | P a g e
Figure 13 - Installation Page during install
Browser RestartAlthough the Silverlight installer states the ambiguous “you may have to restart your browser”, from our experience Internet Explorer users generally do not have to restart. So to avoid the browser restart and user confusion, a JavaScript timer on the installation page detects that the plug-in is installed and pushes the user to the classroom application page.
How-To: Manually install and detect for the Silverlight plug-inUse the isInstalled() function provided by Silverlight.js:
<script language="javascript" type="text/javascript">Silverlight.InstallAndCreateSilverlight = function(version) {
var retryTimeout = 1000; //the interval at which instantiation is attempted(ms)if (Silverlight.isInstalled(version)) {
...}else if (!Silverlight.isInstalled(version)) {
TimeoutDelegate = function() {
Silverlight.InstallAndCreateSilverlight(version);}
setTimeout(TimeoutDelegate, retryTimeout);}
}
Silverlight.InstallAndCreateSilverlight('2.0.30923.0');</script>