bcect huddersfield final report infokit

Final Report

Reference: WYLLELLUM good practice & innovation

WYLLN Elluminate Project 2010 07 30

Author(s): Chris Parkin, Joanne Charlesworth

Main Contact: Joanne Charlesworth

Department: West Yorkshire Lifelong Learning Network, University of Huddersfield.

Revision History

Date Version Description Changed by

20100727 0.1 Preliminary formatting. Chris Parkin

Final Report


Project Title: WYLLN Elluminate Project

2 of 26 www.jiscinfonet.ac.uk

Contents

CONTENTS ......................................................................................................................................2

ACKNOWLEDGEMENTS ...................................................................................................................4

EXECUTIVE SUMMARY ....................................................................................................................4

BACKGROUND ................................................................................................................................5

AIMS & OBJECTIVES ........................................................................................................................6

METHODOLOGY..............................................................................................................................6 QUALITATIVE ASSESSMENT .......................................................................................................................7 FUNCTIONAL ASSESSMENT........................................................................................................................8 COMPARISON TO OTHER PRODUCTS...........................................................................................................8 RAISING AWARENESS...............................................................................................................................9

IMPLEMENTATION........................................................................................................................10 CONNECTION FAILURES ..........................................................................................................................11 RESOLVING THE ISSUES...........................................................................................................................12 CASE STUDIES.......................................................................................................................................13

OUTPUTS ......................................................................................................................................17 COMPARISON TO OTHER PRODUCTS.........................................................................................................17 SUSTAINABILITY ....................................................................................................................................18

OUTCOMES...................................................................................................................................18 TO RAISE AWARENESS OF ELLUMINATE AMONGST WYLLN PARTNERS: ...........................................................18 TO EVALUATE ELLUMINATE AS AN EFFECTIVE TOOL FOR ONLINE COLLABORATION: ............................................19 TO ASSESS WHETHER THE TOOL IS SUITABLE FOR USE BY WYLLN AS A REPLACEMENT FOR FACE-‐TO-‐FACE MEETINGS:19 TO RAISE AWARENESS OF SOFTWARE TOOLS IN GENERAL AMONGST WYLLN PARTNERS:....................................20

LESSONS LEARNED ........................................................................................................................20 TOP TEN TIPS FOR HOSTING ONLINE MEETINGS .........................................................................................21

CONCLUSIONS ..............................................................................................................................22

IMPLICATIONS ..............................................................................................................................23

Final Report




RECOMMENDATIONS ...................................................................................................................24

REFERENCES..................................................................................................................................24

APPENDIXES .................................................................................................................................25 SOFTWARE EVALUATION QUESTIONS........................................................................................................25 DISSEMINATION SUMMARY ....................................................................................................................26

Reference: WYLLELLUM



Acknowledgements The ‘WYLLN Elluminate’ trial project was funded by JISC under the ‘Facilitating Collaboration’1 stream of the BCE programme2 as part of the ‘Trialling Collaborative Online Tools for BCE’ project3. JISC infoNet4 led the delivery of outputs with support from other JISC Advance Services5.

The trial project team would like to thank the following for their hard work and contribution to this trial project as well as the wider BCE agenda:

• Jacquie Kelly, Will Allen and Owen Roberts at JISC for their continued support.

• The partners and organisations of the West Yorkshire Lifelong Learning Network for their assistance and viewpoints.

Executive Summary The project has four main aims:

o To raise awareness of Elluminate amongst WYLLN partners. o To evaluate Elluminate as an effective tool for online collaboration. o To assess whether the tool is suitable for use by WYLLN as a replacement for face-‐to-‐

face meetings. o To raise awareness of software tools in general amongst WYLLN partners.

The awareness raising would be achieved through attending meetings and events organised by WYLLN over the 2009 – 2010 period and talking to attendees of those sessions about the types of software out there, with focus on Elluminate.

The evaluation would be done by encouraging WYLLN partners to make use of Elluminate, replacing a couple of their face-‐to-‐face meetings with online ones instead. This would give them the opportunity to assess whether online tools are suitable for the work they do in meetings and whether Elluminate itself is a particular useful tool. Because of the autonomous nature of the network, WYLLN relied on people volunteering to take part in meetings, rather than directing people to take part. This had an effect on the volume of results we could gather and the types of meetings we could hold. The awareness raising approach was vital to getting people interested in trialling the product. While plenty of people were interested in using it, very few could generate a purpose to do so (other than to solve curiosity).

During the course of the sessions, it became clear that there was an underlying technical problem which was impacting on the quality of the sessions held. In some instances, people were connecting but had issues with the quality of the connection, experiencing lag or sound drop outs. In other cases, people could not connect to the meeting at all.

1 http://www.jisc.ac.uk/whatwedo/programmes/bce/stream2.aspx 2 http://www.jisc.ac.uk/whatwedo/programmes/bce.aspx 3 http://collaborativetools4bce.jiscinvolve.org/ 4 http://www.jiscinfonet.ac.uk 5 http://www.jiscadvance.ac.uk




A process of troubleshooting followed, initially looking at whether the client software was being downloaded and installed correctly and then moving on to whether the attendee’s institutional firewall was blocking the connection. In the end it seemed neither was the case and that the initial installation of the server software at the University of Huddersfield was to blame.

We discussed the matter with our Elluminate Account Manager who indicated that our installation was unique in that we were hosting our own server. The only other institution who was doing something similar was the Open University. Rather than continue trying to resolve the issues causing this, it was decided to switch our licences over to a hosted solution, using Elluminate servers to hold meetings.

The switch over came late in the day and we still encountered issues with lag. By this point most of our partners were unable or unwilling to continue to partake in the project. We were successful in promoting online collaborative tools, with particular regard to Elluminate but perhaps failed to convince our partners on the reliability of the products. Awareness of alternatives such as Skype was already present in some corners, particularly amongst those involved in technical work.

Background The West Yorkshire Lifelong Learning Network is a partnership between HE / FE institutions and other organisations across the region, committed to providing quality vocational progression for learners to progress into higher education. The partnership works to create and promote learning opportunities, particularly ones where there is an employer demand for the skills it will provide.

Much of the work undertaken by the Lifelong Learning Network is collaborative in nature, with meetings being the driving force behind much of the activity. WYLLN are involved in a number of regional and national projects and frequently meet to consult on activity relating to these. On some of these projects, WYLLN acts as regional champion – actively promoting it to other institutions in the region who may wish to get involved. Notable amongst these projects are the national XCRI-‐CAP project (initiated by the University of Bolton) and the ECIF Funding project.

Elluminate is an online collaboration tool that suits particular types of meetings. It is not a pure internet chat application, though it does integrate this functionality into its package. For the purposes of this project Elluminate will not be offered as an alternative to attending a meeting which others are attending in person. This is due to the additional conferencing hardware required in order to facilitate this. If an opportunity to work with a partner hosting this kind of meeting arises, then it will be incorporated into the project.




Meetings held in Elluminate tend to fall into 2 broad categories – Collaboration, Presentation. Collaborative meetings can be thought of as many-‐to-‐many meetings, where all participants are able / allowed to speak and be involved with the session. Presentations are one-‐to-‐many, with one or two people controlling and directing the session, perhaps with time allowed for feedback. The two different meeting types will provide entirely different results in terms of user experience. A person passively watching a presentation will use the software in a different fashion to someone actively discussing a topic or sharing an application.

Aims & Objectives The overall aim of the project was to evaluate the effectiveness of Elluminate as a collaborative tool, inline with the work performed by the West Yorkshire Lifelong Learning Network and its partners. A secondary aim was to identify and raise awareness of the benefits of using online meetings as opposed to those held in person.

It is important to state our objective was not to promote Elluminate itself as a solution (in fact for many of our partners, Elluminate’s pricing structure would make it impractical). Rather, Elluminate was chosen purely because it is a common application found in academic institutions across West Yorkshire and was available to WYLLN to use. Participants in the project were encouraged to focus less on the branding and the product itself and more on the functionality on offer and the benefits gained from using it.

Methodology It was planned that the project would work with the West Yorkshire Lifelong Learning Network partnership to hold Elluminate sessions. This would provide the project with a large and varied user base, varying from experienced users at partner institution (including the University of Huddersfield) to non-‐users in non-‐academic positions.

The approach would be to break the project down into four separate work streams (shown in figure 1 below).




Figure 1: Project work streams.

The analysis work stream was based on qualitative feedback and a functional assessment which occurred during the case studies which took place.

Qualitative Assessment

Most of the findings of this project were drawn from the user experience. This aspect of the assessment is based on how users felt about the software, ease of use, preconceptions and experience.

Users who successfully held or partook in a session were encouraged to either provide direct feedback to the moderator (if hosted by WYLLN) or to complete an online questionnaire. Most chose to give direct feedback either during or following the meeting, when it would not intrude on proceedings.

The questionnaire was available in online format, with the URL sent to users either prior to or immediately following an Elluminate session taking place. Users were required to answer a short number of questions, providing numerical scores for different aspects of the session.

Project

Purchase Headsets / Webcams

Distribute Headsets / Webcams

Purchase Server Licences

Email WYLLN Partners

Articles for WYLLN

Newsletter

Produce and Present

PowerPoint presentations

Blog

Produce Questionnaire

Hold Sessions

Setup Awareness Activity Analysis

Collect Results

Obtain feedback

Produce Report Distribute

Questionnaire




The assessment also covers use of Elluminate’s features – such as application sharing – that may be used in a meeting. Not all functionality was used in each session and it is important that users have the option to indicate ‘not applicable’.

For the most part, questions required users to indicate their perceived quality of a function or feature (e.g. how easy was it to ask a question) with answers ranging from 1 (poor / difficult) to 10 (good / easy). In total only 10 results were collected using the survey from the various sessions held.

An aspect of the project hard to quantify is how comfortable users were with the technology. This was for two main reasons – one is that a person’s own perception of how difficult something is to do is subjective. Another is because participants rarely had to do something once they were in the session as the moderator handled most actions. Users invited to take part in the study were offered free training on the product in order to be comfortable using it before any meeting takes place. A couple of users volunteered for this and were taken through the application in a separate meeting. Broadly this was generally only taken advantage of when there were users who were going to be using a non-‐standard feature of the communication software (e.g. loading a PowerPoint presentation or showing media). Most users appeared to be comfortable with the standard features, even if they weren’t familiar with the layout of the controls for those features. They expected a minimum standard of functionality.

Functional Assessment

This looks at the features and functions of the software involved, how well it does the job required (and whether it is actually required and used). The purpose is to assess how well the software meets the needs of the individuals using it. In conjunction with the qualitative feedback, it gives an overall picture of the level of technical complexity a software can have in order to meet the requirements of the WYLLN network.

This assessment also looks at any technical feedback from users and moderators with particular regard to any operational problems encountered in using the system. The areas covered by this include installation problems, details on operating systems and machine specs, statistics on sessions and lag. Any tangible information that could be provided (including error messages) contributed to this.

Comparison to Other Products

It would be unfair to focus on a single product to determine its suitability for a purpose, without comparing it to other available products that might be used for the same task. Not all meetings held in Elluminate require its collaborative tools – many are just vocal meetings which raises the question whether other software / communication means would be more viable. A key element of Elluminate is that meetings are planned, whereas in real-‐life a great deal of communication is spontaneous.




The comparison to other products phase studied the licensing costs of Elluminate in relation to other products, comparing the functionality that comes with each application. It would also be of interest to calculate the cost of a phone call in order to determine whether a licensed VOIP application is truly value for money – the deciding factor for many users. Therefore Elluminate was compared to products which either competes with its communication aspects (e.g. Webex), the results of which can be found in the Outputs section.

Raising Awareness

In order to generate interest in the project and to get partners involved at an early stage several appearances were planned at organised events and meetings. These included Sector Officer meeting’s, Partner Meetings and WYLLN sponsored events.

A typical presentation would run through the drawbacks of holding face-‐to-‐face meetings, focusing on such aspects as travel, time and cost:

Figure 2: Slide indicating Travel times to Leeds.

Figure 2 above shows the typical driving and train times to Leeds from key towns across West Yorkshire. This slide was used at the Open Education Resources event, held at Leeds




Metropolitan University on the 26th of January 2010.

The presentations aimed to get people thinking about how they could use collaborative software – not just Elluminate – to improve the way they work and interact with their colleagues.

Figure 3: Suggested Elluminate Meeting Types.

Many partners, particularly those based at Leeds Metropolitan University (such as NTI Leeds) were familiar with Elluminate as a Presentation tool, but had not considered it for smaller scale uses. By doing the presentations, it got groups thinking about how they could utilise the software for their own needs.

Implementation The University of Huddersfield hosts an Elluminate server as part of their own application software suite. This was implemented prior to WYLLN’s involvement in the project and therefore the configuration and platform choice was not within the control of the project.

WYLLN purchased six licences to use the software, which included a dedicated server area on which meetings would be held. This area would be kept separate from meetings occurring as part of the usual UoH use of Elluminate, reducing the impact of system strain.

Use of the software by WYLLN partners would be enabled by a member of WYLLN staff organising the meeting and acting as moderator. This would require them being provided with the contact details of other users who would require moderator privileges during the meeting. Standard users would just be provided with the URL of the meeting room.

A basic requirement for attending the meeting was to have access to a headset (microphone & earpiece). Some users who were already regular users of online communication tools already had access to these. However to ensure that there was as much uptake as possible, WYLLN purchased and distributed USB headsets to partners who had expressed an interest in partaking in the project. In total 20 headsets and webcams were distributed to partners for use with the project.




A number of technical issues occurred which impacted upon the project plan and had to be dealt with. They had a significant impact on the project and its deliverables.

Connection Failures

By far the biggest problem encountered was that some participants were failing to connect to the session. There was seemingly no pattern to this -‐ some users could connect whereas others would fail. To investigate the cause we had to run through various areas of potential causes. Some of these highlighted flaws with the software which limits its usefulness as a collaborative tool.

Setup Elluminate operates on a server to client(s) basis. Each user logging in to the meeting connects to the server and not to the other participants. The client software is downloaded to the participant when they click on the meeting URL. No previous installation is required. This is both a benefit and a drawback of the system. While it appears no installation is required, clicking on the link starts a download of a Java application which is temporarily installed and runs in the browser. It requires the Java platform to have already been installed and configured correctly.

This adds to the preparation that needs to be done prior to the meeting taking place. The moderator must make sure that each user has Java installed. This usually means setting up a test meeting in advance of the live meeting and sending out a link to that. Other forms of online communication usually require that a client already be installed (e.g. Skype). Having a client pre-‐installed removes some of the configuration required (as it will do it upon install).

A second requirement is that Java is configured to allow the saving of temporary files locally on the pc. This should be just a simple matter of adjusting a check box in Control Panel. However, given most HE and FE institutions apply rigorous IT policies; it is often far from simple. If Java is installed then quite often the user does not have permission to configure it, most likely because altering it may affect how another application works.

Quite often, getting Java installed and configured on a participant’s computer involved liaising with their IT support. This led to a third meeting being booked to test the setup (the second meeting having expired when it became apparent that the user did not have permission to configure their pc).

Firewalls When Java was no longer suspected of being the underlying cause of so many connection failures, focus shifted to firewalls. This was based on information found on the Elluminate website, which indicated that an incorrectly configured firewall could potentially cause problems. This appeared to make sense as it explained why some people could connect




while others couldn’t.

It was assumed that because UoH already had an Elluminate licence and used it reasonably regularly that there was not an issue with the firewall at Huddersfield. This was partially correct as it turned out.

The Elluminate website contains some useful information about how a firewall should be configured to allow Elluminate connections to be made. This information was duly sent out to the relevant ICT support teams for participants who were still failing to connect. In most cases it made no difference.

The Liverpool City of Learning group were willing to investigate the potential firewall issue for two of their users who were trying to connect to a WYLLN meeting. They could find no issues and worked through the help documentation that they were provided with. They even checked whether Elluminate was trying to bypass their proxy server (which would result in no internet connection) but this also was found not to be the case. Compounding the issue further was the fact that Liverpool City of Learning had Elluminate licences themselves and had no issues running meetings. The problem, it seemed, lay with the University of Huddersfield.

Resolving the Issues

In order to ensure that anyone connecting to the meeting did not suffer from lag, certain steps had been taken to maintain quality:

1 – Meetings were scheduled to occur during periods of ‘light’ network activity. This meant either first thing on a morning, Wednesday afternoons when no courses were taking place or late Friday afternoons.

2 – Web cameras were not used for meetings with more than 2 participants.

3 – Users were encouraged not to be running other applications in the background.

4 – Users were encouraged to have tested their connection prior to the meeting using a test meeting.

5 – All audio equipment was to have been configured prior to the meeting.

Despite all these measures quality issues were still encountered. Some of these affected specific individuals and some were specific to the meeting itself and any functionality of Elluminate being used. Sometimes using specific functionality (e.g. desktop sharing) would impact upon the overall quality of the meeting.

We approached Elluminate about our issues with the system and they happily arranged to have our account manager come in and visit (an attempt earlier in the year to get an account manager in yielded nothing). During the course of the [face-‐to-‐face] meeting it became clear that the way we had been set up for our licence was not optimum. In fact, it




transpired that other than the Open University, no other institution in the UK was hosting its own Elluminate server.

It was decided that in order to continue using Elluminate we would have to switch our licence package to use their system. This was organised within two weeks. The new licence gives us access to a newer version of Elluminate, thought the core functionality is broadly the same.

Early testing of the software suggested that it would be better, as the connectivity problem seemed to have dissipated with the firewall no longer being an issue.

There is a noticeable difference in connection quality between the two different versions of Elluminate used by this project. When using the Huddersfield hosted version, when people could connect the audio was very good. This was not the case with the Elluminate hosted version. Audio was delayed significantly (not just by a few milliseconds – it took in some instances up to 15 seconds for sound to be relayed). This led to some very confusing conversations, with participants responding to questions posed earlier.

These issues occurred even when running the software in a scaled-‐down mode, with no cameras and no other software running on the computer.

Case Studies

Liverpool City of Learning XCRI Project (3 Sessions) A consortium of representatives from various national institutions working with Liverpool City of Learning attempted to use XCRI to avoid having to meet physically. Representatives from the universities of Huddersfield, Nottingham, Bolton and Liverpool were sent invites for a meeting (including an invite to a test meeting prior to the main meeting).

The test meeting was scheduled for a couple of days in advance of the main meeting. Invites were sent out to all participants and they were requested to have IT resources available to troubleshoot any difficulties. Only a small number of invitees used the test meeting (2 out of 8) and both of these were at the same location. Therefore it wasn’t possible to determine whether or not Elluminate was going to work for the meeting. The two that connected did have difficulty but their IT services believed the issue was with them and would make some adjustments to their firewall in time for the main meeting.

As there was a risk the meeting software would fail to deliver, the time people could connect to the meeting session was extended so that more testing could take place. This time a number of participants did try to connect and it was found that there were still connection issues. In response to this, an email was forwarded suggesting a backup plan be devised:




“It seems some people are having issues connecting to the Elluminate session

because of configuration issues to do with Firewalls / Proxies. Hopefully we'll get these resolved before 11am, but we should probably devise a Plan B just in case. In the meantime, I've set the meeting space to be available from now onwards if you want to try and test your connection. The link to the meeting is:

https://elluminate.hud.ac.uk:443/join_meeting.html?meetingId=1267778459003

Anyone having problems should read the attached document to check their

configuration.”

Figure 4: Extract from Email Invite to Elluminate Session.

In the end it was decided to use Skype, which had the functionality required. Documents for the meeting were emailed separately, rather than downloaded as they would have been in Elluminate. This was not an issue, though it is more beneficial to have Elluminate manage the document sharing. Anyone joining the meeting late automatically gets sent the documents, whereas now the moderator has to stop what they’re doing and send out the documents manually.

However, sound issues were occasionally experienced using Skype. Figure 1 shows an excerpt from a Skype meeting which was held in place of an Elluminate session. Elluminate was abandoned because of problems with people connecting.

In the context of that particular meeting, one user was coming across as silent / distant to a number of other users on the call. However, not all users were hearing the same thing and she sounded fine to some others. This is down to the way Skype works compared to how Elluminate works:

Skype is peer-‐to-‐peer software which means each user makes a connection to every other user on the call. As such, there is no host / moderator.

Figure 5: Skype Text




Elluminate differs as each user makes a single connection to a central server. The server relays the data to each other user in the group.

AXIA IAGNow Support (3 Sessions) Axia provide technical support for one of the applications used by WYLLN. The application (IAGNow) is a custom design, created specifically for the work carried out by WYLLN and is constantly under review. Normally these are carried out at Axia’s offices in Batley but it was decided on one occasion to use Elluminate.

As two participants were based at WYLLN and two at Axia, it was assumed that there would be no issues with connectivity. Connection was made but found to be lag

XCRI Project – WYLLN (5 Sessions) Two members of the same project, working from two separate UK locations needed to collaborate on a software application, running locally on a pc in Huddersfield. The application was launched and Elluminate was set to share the application. The meeting had already been running for some minutes before the application started. There were no connectivity issues other than a slight lag in sound.

Ideally, desktop sharing would have allowed both participants to contribute to the design of

Figure 6: Skype Peer-to-Peer.

Server

Figure 7: Elluminate Server Model.




some code which was open on the application. Enabling the desktop sharing was straightforward and the host ensured no other applications were running (other than Elluminate). The participant who was following the host was able to see the application clearly and could read the code. However the participant encountered issues as soon as the host moved the mouse and scrolled down the page. On his computer the application appeared to be jerking around the screen, making it impossible to follow what was going on. On the hosts computer this was not apparent.

Both users checked their settings were set to the optimum level for their connection type (LAN for the host, broadband for the participant) and that they had available free memory so as not to be maxing out their computers. It was decided that there was an unavoidable networking issue that was causing the issue (there being nothing apparently wrong with the setup of either machine) so desktop sharing was stopped.

Learning without Borders Project – Leeds City College (2 Sessions) A project group from Leeds City College, with participants spread across Greece and Poland, trialled the Elluminate-‐hosted version of the software for meetings. 4 Participants connected remotely from locations in Greece and 1 in Poland.

Participants used webcams, though they had some difficulty getting these to work. The session was set to support 6 cameras. When video was running, they only used it for the first 2-‐3 minutes of the session.

Audio was good, but there was an echo. After a short while it became apparent that there was some lag occurring. This led to the cameras being switched off to see if it made a difference.

Overall they were unimpressed with the software, but countered that collaborative software in general would be beneficial to them. They are actively looking at other products on the market to continue the project.

ECIF Project Drop-In Meetings (5 Attempted Sessions) WYLLN won HEFCE funding for the Economic Challenge Investment Fund – a pot of money to invest in workers facing or at risk from redundancy. The project is coordinated by WYLLN with regular meetings being held with each of the institutions that were involved in the allocation of funds.

The Project Manager decided that there was an opportunity to utilize Elluminate by holding a weekly drop-‐in session, to be held every Thursday morning. Those involved in the project that had an issue to discuss could connect to the session and chat to the PM informally. The benefit of doing it this way as opposed to just being called by phone is that multiple people




could drop in, making the Project Manager’s use of time more efficient as she was not normally at her desk.

Each project member was sent a link to a recurring meeting that they could join. They were provided with headsets for the purpose. Test meetings were set up to determine whether or not each user could connect. At least 7 test meetings were held in the end with 3-‐4 people attempting to connect at each session. No successful connections were made.

Outputs

Comparison to Other Products

For this to be a fair assessment of online collaborative tools, we could not restrict our analysis to Elluminate completely. Consideration to other products had to be given so as to identify other factors which may encourage / discourage their take-‐up. A key consideration was price – was the licensing of Elluminate in line with other products or was it atypical? WYLLN’s work is mainly done through small groups, typically under 10 per meeting so the main question is whether or not collaborative tools are a cost-‐effective alternative to these types of meeting?

Typically, online collaborative tools have scalable licences. Usually there are three tiers of price with each increase in tier offering more functionality. Some versions offer a basic free version as their lowest tier, with restrictions on the number of connected users.

The below table summarises the more common applications against their tiers (US prices adjusted to GBP):

Tier 1 – Free or Cheap

Tier 2 – Monthly Cost

Tier 3 – Yearly Cost

Elluminate 3 seats connect for free.

£33 for 50 seats. £217 for 50 seats.

Webex £30 for 25 seats (but add on 11 for voip / tax).

£288

DimDim 20 for free £16 for 50 seats. £150 for 50 seats.

Adobe Connect £0.20 per user per minute

£36 for 50 seats. £355 for 50 seats.

A product which was frequently turned to when Elluminate was not working was Skype. As Skype does not classify itself as a collaborative tool it has not been included in the above list. It’s an audio / visual communication tool, but doesn’t have any of the additional




functionality that would allow it to compete. However, it shouldn’t be discounted, purely on the basis that most meetings that were held in it did so without requiring the additional functionality.

As an indication as to how popular Skype is, when it was suggested to that Skype be used in place of Elluminate, all participants already had Skype user id’s. This was despite it conflicting with the Computer Policy of most institutions.

Where there was a shortcoming, a workaround could usually be found:

Elluminate Function Skype Workaround

Document sharing. Email documents to participants.

Breakout rooms. Participants disconnect from the main group and hold their own call.

Multimedia presentations. Could be sent to each participant or displayed on a third party website (e.g. www.youtube.com).

Recording Not possible.

Desktop Sharing Possible on a 1-‐1 basis.

Follow-‐me Internet sharing. Could be done through desktop sharing.

Whiteboard Could be done through desktop sharing.

There is also a third-‐party add-‐on for Skype called Innerpass. This application adds some of the more common collaborative functionality onto Skype at a price that is competitive with the other applications. The functionality added includes improved desktop sharing and shared folders / document sharing. The cost of this is £3.95 for 5 users.

Sustainability WYLLN is an organisation that exists for the duration of its funding (4 years). This year marks the start of its fourth year, when both staff numbers and resources will be greatly reduced. Any continuation of the work initiated by this project will not be done by WYLLN itself.

The WYLLN partnership may wish to continue with collaborative tools, particularly once the support structure that WYLLN offers has been removed.

Outcomes

To raise awareness of Elluminate amongst WYLLN partners:




WYLLN’s partnership is a wide and diverse network, with some based at education centres and others located in the private sector. Those working in the private sector or for industries which have few ties to education were unfamiliar with Elluminate. However they did show familiarity with some of the other products on the market (such as Skype) and the technology in general.

Through a mix of talks at both Sector Officer meetings, Sector Group meetings and events, there has been success in raising awareness of Elluminate. Many groups based at colleges in the area had not considered using the application despite them having a campus licence to do so (it was primarily used for teaching). However, they were generally aware of it, even though they hadn’t seen it in use.

The WYLLN network was given demonstrations (simple 1-‐1) of how the software worked in order to improve their understanding of what it could do for them. This often resulted in people commenting that they hadn’t considered it beyond its function as a means for distance learning or presentation delivery.

To evaluate Elluminate as an effective tool for online collaboration:

Elluminate is a widely used tool in education for distance learning and presentations. While this project has had some success in promoting the use of collaborative tools, overall it has failed to raise the profile of Elluminate. Partly this is down to the technical issues encountered, but mainly it’s down to Elluminate’s competition in this area. Free-‐to-‐use software that is readily available for download from the internet has proved far more popular as a communication tool than Elluminate ever will.

However, there were those that found they needed a middle ground between the high pricing structure of Elluminate and the functionality it could offer and have sought after products in the same category (licence-‐based). The Learning without Borders Project is one such group that need a mix of communication and access to features such as whiteboards and desktop sharing. However, being a small group they aren’t happy with having to pay a large licence fee for the software. They have been looking at alternate products (Skype does not have enough functionality for them) and may consider a pay-‐as-‐you-‐go pricing structure.

To assess whether the tool is suitable for use by WYLLN as a replacement for face-to-face meetings:

There has been a partial success here. The key success factor is the willingness of WYLLN partners to embrace the technology. Partners have been keen to try the technology for some meeting types. However, there is still the (understandable) reluctance to use it for formal meetings. This doesn’t appear to be down to any lack of faith in the product, more a tradition that these types of meetings occur face-‐to-‐face. Their perception of the reliability of these types of products was not aided by the issues which arose through our use of Elluminate, but this did not dampen any interest in using the application.




One observation about these meetings is that there tends to be a lot of ‘water cooler’ moments where board members discuss other issues, off-‐record, over a cup of coffee before or after the main meeting. These are almost break-‐out meetings that are opportunistic in nature. With a virtual meeting environment, these types of meetings could not take place. The facility to hold them is there (breakout rooms are a standard feature) but it’s often the fact that you’ve seen someone that triggers the thought that you need to speak to them about something. In many ways these unofficial meetings are often more important than the main meeting itself. In particular, meetings which are scheduled to end at or around lunchtime often have food provided. Many attendees remain for the food and have informal chats with other attendees. The food almost acts as an incentive to mingle (or as one delegate put it, the food is the incentive to turn up to the meeting in the first place!).

To raise awareness of software tools in general amongst WYLLN partners:

There was generally good enthusiasm for other tools by WYLLN’s partners. During many of the presentations, this slide was shown:

Figure 8: Web 2.0 Software (Ludwig Gatzke).

The slide above (taken from Flickr under a creative commons licence -‐ http://farm1.static.flickr.com/14/93136022_25afa7e458_o.jpg) shows common Web 2.0 software that is widely available on the internet. The purpose of most of this software is to share or collaborate. Certain logos jump out at viewers of the presentation as they are well known. Others simply raised curiosity. It got people asking questions – “we’re looking to do…” – where they hoped the solution might be hiding among the logos. The slide had the effect of bringing about discussion on the possibilities of using certain types of software to resolve certain types of issues.

Lessons Learned




Reliability is an issue that must be addressed in order for institutions to have faith in online collaborative tools. There are too many reliability issues and if the amount of organisation required is greater than that of the alternate real-‐world meeting then there is no benefit. Holding an online meeting is supposed to simplify the meeting process. However, from the experience of this project it seems to replace one organisational requirement with another. You don’t require an office manager to loiter when you use a meeting room, however it seems you do need to have IT support on hand to get a virtual meeting room to work! Each institution joining the meeting must be prepared by having their local IT support to hand.

An increased amount of preparation is required (in comparison to real-‐world meetings) just to ensure the technology works. This is against the point of using the software in the first place, which is to make holding meetings and working together simpler. If Elluminate was a car, then it’d require you to retake your driver’s test and tinker with the engine before each journey. And you may lose a passenger or to along the way.

Elluminate is geared toward the repeat user. To get the most out of it as a moderator requires experience and just getting it configured to run smoothly as a participant has a learning curve. Unfortunately most WYLLN partners are not repeat users, which limited the benefits they could gain.

Top Ten Tips for Hosting Online Meetings

1. Use software that everyone is comfortable with. If you have a licence for an application that no-‐one has heard of or ever seen then there is going to be a learning curve for all participants upon joining the meeting.

2. Being the chair does not mean you have to provide the software. This seems to be an automatic reaction – you are calling the meeting therefore you are responsible for providing the facility to host the meeting. This is not required – you can be granted moderator rights in most meeting software. If someone has a better software option, use it and ask to be granted moderator rights.

3. If you are chair, don’t try to moderate large meetings yourself. In a real world meeting, the chairperson does not take minutes, nor are they responsible for ensuring that everyone has the right documents and that presentations are loaded prior to the meeting. The same is also true of virtual meetings. If they are formal or structured, appoint someone else to be moderator or co-‐moderator.

4. Get users to test their connection prior to the meeting. Obviously resolving any potential connection problems is better done before the meeting rather than during.




5. Respect people’s privacy – part 1. Elluminate has a ‘Supervisor’ mode in which any inter-‐participant text chat that does not involve the moderator is still transparent to him. Participants are unaware when this mode is switched on (it is off by default). If you are going to activate this mode, ensure your participants know (or at least have some thick skin).

6. Respect people’s privacy – part 2. Another privacy busting feature of Elluminate is the option to record meetings. This facility may be a little more obvious to users as there is an icon displayed at the bottom of the screen indicating when recording is taking place. However, it is good practice to inform participants prior to the meeting (and not as they join) that you intend to record some or all of the meeting. They may have issues with this. Or they at least will not make remarks that they would not want repeated outside of the meeting. If the software allows the recording to be exported (Elluminate does not) then users should be informed of where it will be posted and for how long, as well as who will have access to it.

7. Add extra time to the meeting. Although a benefit of using online meeting tools is that more efficient use of time can be made, it may not always be beneficial to do so. In real world meetings people do not [aim] to arrive just as the meeting starts, the get there early to network and hang around after to discuss other matters. Buffer times can usually be added to meetings – let your participants know what the buffer times are so they can take advantage of it.

8. Have technical support available. Most institutions have an IT Support desk to help with issues. Holding a meeting online can be a technical challenge so ensure support is available and don’t schedule the meeting to occur during any planned system downtimes. Although it would be unlikely to affect the quality of the meeting, it just means that IT Support will likely be busy dealing with that and may not be as available.

9. Be flexible. If a feature stops working or becomes unusable because of lag, adapt the meeting to get round it rather than postponing. Not everyone is guaranteed to have the same quality connection, so if one person is having problems following a PowerPoint presentation on the server, send them a local copy to use.

10. Practice! Familiarity with the functions and features of Elluminate (or any software) will make meetings run smoother and more fluid.

Conclusions




It is clear that the policies of FE/HE institutions in regards to IT act to constrain online collaboration, not encourage it. Some of the difficulties encountered in getting an Elluminate meeting to take place have been compounded by restrictive policies.

Java is a common programming language, used in a variety of applications on a variety of platforms (phones, computers, DVD players etc). It is therefore surprising to encounter institutions which have banned it from being installed on their computers. This was the case at Leeds City College and in the end the participant had to provide their own private computer to be able to use the application.

Elsewhere we have encountered computers with their USB ports blocked to prevent external devices being attached. This meant the USB headset could not be connected. This was possibly more surprising than the lack of Java. Preventing external hardware from being attached could possibly have been to prevent data being removed from the computer, but as they were still attached to the internet this seems unlikely. It may be to prevent damage to the computer.

With the Virtual Meeting having almost as many organisational requirements as its real-‐world equivalent, one wonders if it will be a short-‐lived function. There is a desire for HE and FE staff to use alternative meeting methods, but these should be fluid and easy. The concept of a meeting space – whether it is virtual or real – is outdated.

Elluminate does not lend itself well to the formal meeting, despite the fact that this is the very concept it is modelled on. Elluminate is requires a moderator / chair person to organise and lead the meeting. It’s a very structured process which perhaps is more suitable to structured events – such as presentations and training – where there is a clear agenda to be followed. It doesn’t suit dynamic meetings where the ‘lead’ switches from person to person (imagine a meeting where everyone contributes to a whiteboard discussion). It can do those meeting types, but not well. Only 6 people can have open microphones and webcams at a time which makes chipping in and contributing more difficult. People work off facial expressions. For instance, in a meeting it’s often quite clear when someone is waiting for someone else to finish speaking before making a statement. In Elluminate, if you’re not one of the six people with an open microphone, you have to ‘raise your hand’ to get the moderator to allow you to speak. It is not a constructive process.

Implications WYLLN has a finite lifespan and as such will shortly no longer be the central hub for its partners. This removal of WYLLN means that there is no central organisation to facilitate collaboration (in the context of Elluminate) amongst its partners. In essence this is the removal of a formal requirement for the partners to come together but they still may wish to collaborate in some form in the future.

The removal of WYLLN as a central structure has actually increased the need to identify new methods of working together and sharing resources. The partnership met and worked using




resources paid for and provided by WYLLN. This allowed them to undertake activities sponsored by WYLLN, but also to pursue their own interests. Granting them access to Elluminate has allowed some of these partners to recognise that they could continue to get together in some form, without the need for the overheads that are normally covered by WYLLN. While Elluminate itself is not a cost-‐effective solution for partners not based at educational institutions, it has acted as a catalyst for exploring alternative solutions.

Perhaps as an indication as to how meetings will occur in the future, The University of Huddersfield is soon to roll out a new desktop communication tool called Office Communicator. The two key features of this product are instant messaging and video-‐to-‐video calling. This will enable staff to (internally) communicate face-‐to-‐face without leaving their desk, reducing the demand for meeting rooms. The policy for using other software for external communication (e.g. Skype, Instant Messaging software) remains unchanged.

Recommendations A number of lessons were learned from the project, both from a technical and management perspective.

Identify in advance how you intend to collaborate – what does it mean for you as an organisation? For WYLLN, collaboration occurs in meetings in the form of discussion. If you rarely work in a certain way, look for alternate free ways of achieving that rather than paying for functionality that will be underused. Elluminate is a feature-‐rich application, but most of these were un-‐used in the sessions that were held. Functions such as desktop sharing, break-‐out rooms and recording were among these.

Pick a product that is scalable to the size of your organisation or network and has a decent margin for growth before the next pricing tier. Take into consideration the price change in pricing at the next tier. Also take into consideration the number of employees that actually require the ability to create meetings.

Know your institution’s IT policy prior to committing to purchasing any solution and discuss the software with your ICT team to identify any potential barriers to it working properly. Often technical tools are utilised by non-‐technical people and it’s easy to misunderstand a system requirement or recommendation when the system is implemented. In WYLLN’s case the system was implemented to a standard that was not recommended by Learn Central, with the server being locally hosted at UoH. This was done prior to the initiation of the project.

References




www.wyllnproj.wordpress.com – blog for promoting WYLLN technical projects, including the Elluminate project.

Appendixes

Software Evaluation Questions The following questions were available online on Survey Monkey for users of collaborative software to leave feedback about the version they used.

• Your Name? • Name (and version) of the software used? • Did this software require you to create an account in advance? • Did the software require installation? • How easy was it to:

o Install the Software? o Create the account? o Join the meeting?

• Was the version of the software used: o Free (no licence required)? o Free but with a purchasable version? o Free under a trial licence? o Meeting host has licences? o Other?

• What software features are available in this version of the software: o Webcam Support o Headset Support o Text chat o Document Sharing o Document Distribution o Desktop Sharing o Breakout meeting rooms o Voting o PowerPoint presentations o Multimedia support o Other features

• Which features were used and how easy was it to do so? o Webcam Support o Headset Support o Text chat o Document Sharing o Document Distribution o Desktop Sharing o Breakout meeting rooms o Voting o PowerPoint presentations o Multimedia support

• What features does this product not have that you would have liked?




• Would you use this application again? o Yes o No o Depends

• Did you find this software suitable for the type of meeting held? o Yes o No

• Is this application better or worse or about the same as Elluminate? o Better o Worse o About the same

• Rate this software out of 10 (10 being best, 1 terrible).

Dissemination Summary Presentations about the project were made at:

• Internal WYLLN Sector Officers meetings (held bi-‐monthly). • The Open Educational Resources Event (Rose Bowl, Leeds) – Jan 26th, 2010. • The Digital Industries Sector Meeting. • Aspire-‐I / WYLLN XCRI Awareness Event – Bradford.