report final draft 1.3.1

8/13/2019 Report Final Draft 1.3.1

1/52

Page 1of 52

ASSESSMENT BRIEF - COVER SHEET

Course: BSc Computing Year: 1 CSY1021 - ComputingMathematics

TCA No: 2 Title: Practical Applications of Statistical Techniques

Date due out: Date due in:

21/04/2013

Extension date: Extension agreed by:

Group Name:

Group: 2 (Team Caffeine)

Group Member Names Student ID's

Dennis Wilson (Group leader) 11424939

Jonathan Nicholls 10321515

Josh Phipps 12408771

Jonathan Raine 12406672

Chris Adshead 12417844

Adam Lang

Chris Abiodun 12408576

Rhys Hall 12412314

Tutor: Mark Johnson

ASSESSMENT FEEDBACK:RATING SCALE Excellent Good Satisfactory Needs some

more work

Needs much

more work

Formulated Aims and

Objectives (10%)

Problem Domain Elicitation

Activities and Findings

(10%)

Evaluation Questionnaire -

Quality of Design and

Deployment Mechanism

Utilised (15%)

Collection and

Representation of Study

Data (15%)

Statistical Analysis of

Results

(15%)

Study Conclusion and

Recommendations

(10%)

Client Presentation (15%)

Report Quality

Presentation, format and

use of English (10%)

Specific aspects of your assignment that the

marker likes:

Specific aspects of your assignment that need more

work:

Tutors Signature: Date: Grade:


2/52

Page 2of 52

ContentsAbstract ................................................................................................................................................... 3

Introduction ............................................................................................................................................ 3

Background and rationale ................................................................................................................... 3

Aims and Objectives ............................................................................................................................ 3

Research Methodology ........................................................................................................................... 4

Problem Domain Review ......................................................................................................................... 5

Identification of the Problem Domain ................................................................................................ 5

Current System Review ....................................................................................................................... 8

Comparable Virtual Learning Environments ..................................................................................... 10

Key Stakeholder Interview Summary Findings ................................................................................. 12

Questionnaire Development ................................................................................................................. 13

Data Collection & Statistical Analysis of Results ................................................................................... 14

Conclusion and Report Recommendations .......................................................................................... 25

Report recommendations ................................................................................................................. 26

Future work ....................................................................................................................................... 29

Critical evaluation ............................................................................................................................. 29

References ............................................................................................................................................ 30Appendix ............................................................................................................................................... 31


3/52

Page 3of 52

AbstractAcademic Virtual Learning Environments (VLEs) have become an essential part of the modern

university, allowing students to access learning material to enhance their learning experience and is

now not just essential for disabled students, but essential for lecturers and students alike.

But because of the importance of VLEs in Universities many different systems have been developed

without any leading system within the field.This has created confusion with many universities whoare unsure with which system to choose, as an incorrect decision could hinder student learning and

henceforth, the reputability of the institution.

This report investigates if the Northampton Integrated Learning Environment (NILE) is suitable for

the University of Northampton and most importantly is effective at supporting the Universities

learning environment. The report shows the problem domains identification and review, which

includes an evaluation of NILE and comparable systems and details the development and results of a

student questionnaire.

It then shows the analysis of data collected, identifying that most problems are caused by a lack of

training rather than flaws with the system itself and listing appropriate solutions to this.

Introduction

Background and rationale

The University of Northampton received a number of formal complaints about their Virtual Learning

Environment called the Northampton Integrated Learning Environment (NILE). As ensuring student

satisfaction with NILE is essential, an investigation into these complaints was started to assess if the

NILE meets a minimum level of satisfaction with all stakeholders and is suitable for purpose. And if

not, to find reasonable solutions to any problems found.

The potential benefits of this research include:

Increased user satisfaction. Increased University reputation. Increased user productivity through higher efficiency. Higher retention of students and increased attraction to future potential students. Reduction of costs.

Aims and Objectives

The central aim for this project is to assess NILE to ensure it is fit for purpose and meets a minimum

level of satisfaction for all stakeholders, and to give a recommendation on possible solutions to any

issues identified. To achieve this several objectives have been developed, these are:

Identification of essential requirements for a Virtual Learning Environment in a University. Review of current NILE functionality. Exploration and evaluation of alternative systems. Investigation into current training and support users receive. Investigation of current user satisfaction with NILE.


4/52

Page 4of 52

Analysis of feedback to identify problematic areas. Analysis of problems to identify solutions. Show conclusions in both a report and presentation to the senior management team.

Research MethodologyTo ensure NILE is suitable for purpose, the following research methodology has been formulated:

1. Formulation of the aims and objectives.2. Investigation into the problem domain.

i. Analysis of the current software.ii. Research and analysis of alternative systems.

iii. Development of a questionnaire aimed at students to gain primary data of studentviews.

iv. Distribution of questionnaire to a sample group to test suitability and effectivenessof the questionnaire.

v. Analysis of sample survey feedback and implementation of improvements from theanalysis.

vi. Distribution of improved survey to the student user group.vii. Interview to gain primary data of IT managersview.

3. Analysis of investigation.4. Conclusion and report recommendations.

Formulation of the aims and objectives is essential to ensure the investigation remainsclear and

focused. Analysis of the current system is needed to be able to gain an understanding of NILE to

guide the investigation forward. Research into alternative systems will provide a comparison to NILE

to determine if NILE is the most suitable VLE for the University of Northampton. Using the data

gathered from the current system and alternative systems review, a questionnaire will be created to

access student satisfaction get primary data from the student stakeholder group. A sample survey

will be sent to a small group of the student stakeholders to increase effectiveness of the survey. The

interview with the IT manager will be used to gain more primary data from the stakeholders. Using

all data gained from the stakeholders, the problematic areas will be identified and solutions to these

problems will be created.

A student survey is used to gain a broad range of views, interviews could be held with the creators of

the formal complaints but this would skew the view heavily in their favour and this data would not

necessarily represent the average stakeholders view.


5/52


6/52

Page 6of 52

new/old users?

Does the system need to have consistent pagesor can each page be different?

If not, how different can each page be? When are prime times for user login? Do extra resources need to be spent during

these times?

Where can users access NILE? From campusonly, home?

Do smartphones need to be compatible withNILE?

What file types need to be uploaded? Will there be an upload speed cap? What is the capacity for files?

This information could be obtained by: A review of all web-board support

manuals/paper-based documentation/online

documentation.

Interviews with product suppliers. Interviews with relevant technical experts (e.g.

the IT support staff that set-up and configured

the system for use.)

Current Staff [Leaders of modules]

What experiences do the current staff havewhen using NILE?

What features of NILE do the current staff like /use?

What features of NILE do the current staffdislike / dont use?

What features would the current staff like tosee added to NILE?

What training do the current staff receive? Do the current staff use NILE to its full

potential?

Do the current staff actively use and promotethe tool to their students?

Have the current staff used another VLE before,and if so what is their opinion of it compared

with NILE?

Would the current staff accept a new system?This information could be obtained by:

Interview carried out with a broad selection ofstaff including other organisations such as CfAP,

through interviews


7/52

Page 7of 52

Once a provisional understanding of the user-groups attitudes towards the web-board had

been established, a more extensive review of

staff attitudes could be achieved via a large

scale distribution of a well-crafted

questionnaire

Current students

What experiences do the current student usergroup have?

What features of the current system do theylike or use?

What features would they like to see added? What features do they dislike or dont use? Have they had difficulties using or

understanding NILE?

Does the user group use NILE to its fullpotential?

What training do the current students have? Is this training good enough for the current

students to understand and use NILE to its full

potential?

This information could be obtained by:

Interviews with the users who have expresseddissatisfaction with NILE

Once a provisional understanding of the user-groups attitudes towards the web-board had

been established, a more extensive review of

student attitudes could be achieved via a large

scale distribution of a well-crafted

questionnaire.

The key system stakeholder

A review of the underlying aims and objectivesof the university management team (specifically

towards the current web-board) and their

rationale for having a web-board system within

the university.

The key stakeholders expectation for any web-board system employed within the university.


Interviews with key project stakeholders.

Comparable systems

A review of other comparable software systemson the market leading to a comprehensive

understanding of the features and functionality

common to virtual integrated environment

To establish if other systems provide better orimproved functions than NILE, including but not


8/52

Page 8of 52

limited to:

Do alternative products have features NILEdoes not have?

If so, are these features important / needed? Does our product have features they do not? If so, are these features redundant? Are these alternative products easier for the

user, such as simpler to understand and use?

Are these products better technically such asfaster downloads?

Do these products have better reviews?This information could be obtained by:

Background reading (developerliterature/company web-sites) outlining

product specifications. Possible interviews with alternative product

technical and marketing teams.

Relevant legislation

A review of any relevant legislation relating tothe use of the software within the university

(Disability Discrimination Act, Data Protection

Act, etc.)


Possible interviews with relevant subject andlegal experts either from the University of

Northampton or hired independently.

Current System Review

NILE is provided for by Blackboard and has been built for a wide range of Universities, it includes

several embedded tools which create modules and pages for the modules and a collection of

features for academic institutes. Blackboard provides an extension program which allow certain

clients to create and customise NILE with add-ons, these are shared so can be used by other

intuitions. The catalogue of add-ons is large and range from support and helpdesk software to voice

authenticators although some of these add-ons require payments.


9/52

Page 9of 52

The current system features (Blackboard, [n.d]) include:

A Calendaro This function allows the user to track important dates within the university semester or

add their personal appointments or obligations.

Online grading and assessmentso This feature allows the user to receive feedback on modules via NILE providing fast

response times for grades given to them by module tutors.

Student activity reportso This allows staff to monitor student activity by being able to see what areas of NILE

certain students are accessing ensuring they are keeping up to date with the work.

Social networko Users are able to create a profile in order to interact and stay in contact with their peers.

Survey toolso This lets university staff or students to create one or more surveys that can be

distributed online to other university staff or students.

Institutional hierarchyo This allows the University to separate NILE into different subsections which allows staff

members to have different levels of administrative control as well as the ability to

customise each subsection to its specific need.

Blogso Students or staff may create a blog related to their course, assignments or university life.

Doing this allows other students to keep informed about several matters.

Announcements / Notificationso Module leaders are able to post notifications on NILE. When their students access the

related subject they are initially greeted with the notifications section in order to see

any important changes or updates.

Individual customisations (for both student and course)o Users are able to personalize their own NILE interface. This includes being able to

prioritise different sections by placing them in different areas on the screen. Topics that

the user deem more important can be placed at the top of their page.

Mobile and Smart Device applicationso There is a NILE application available for iOS and Android, this has a variety of features

including a satellite navigation system for the university, access to grades, access to

timetable of students, a news feature, limited library features and listing for events in

the near future.


10/52

Page 10of 52

Email updateso Automated emails sent daily to users of NILE to inform users of new announcements.

Users are able to opt out of this by unsubscribing to this feature.

In-site support featureso Allows the University to add its specific unique support features into NILE.

Wikio Module leaders are able to create a wiki related to the course. Students are able to edit

and update this, contributing to the data stored.

Search Systemo The system can be searched for relevant modules.

The University of Northampton has provided several text, PDF and video tutorials which are

available from the Help tabfor both guests and users. These tutorials include:

Logging into NILE Changing your password and personal information. Customising your NILE Homepage Downloading files from NILE Printing files from NILE Sending email through NILE Getting started with NILE Using the tools in NILE Assessment in NILE ePortfolios in NILE Other useful guides

Comparable Virtual Learning Environments

An investigation into comparable VLEs was started to potentially find a more suitable VLE than the

current system. This investigation would provide information to be used to compare and contrast

against the current system to allow a more accurate assessment of NILE.


11/52

Page 11of 52

Table 2Comparable systems features list(Moodle, 2011; Sakai [n.d.]; SourceForge [n.d.])

Feature NILE

(Blackboard)

Moodle Sakai Bodingtons

Calendar Yes No No No

Online grading and

assessments

Yes Yes Yes Yes

Student activity reports Yes No Yes No

Social network Yes No Yes No

Survey tools Yes Yes Yes Yes

Institutional hierarchy Yes No No No

Blogs Yes No No No

Notifications Yes No No No

Individual customisations

(for both student and

course)

Yes Only the

course page

can be

customized

No No

Mobile and Smart Device

Applications (Apps)

Yes No No No

Email Updates and

Notifications

Yes No Yes No

Announcements/News Feed Yes Yes Yes No

In-site Support Features Yes No No No

Search system Yes Yes Yes No

Wiki Yes Yes Yes No

Group work No Yes Yes YesPeer feedback tool No No No Yes

Instant Messaging No Yes Yes No

Course file search No Yes No No

Voice tools No No No No

Course glossary No Yes No No

Journals No Yes No No

Podcasts No No Yes No

Question creations No Yes No No

Detailed user profiles No Yes No No

Research suggested that if it is viable to replace the current system, then the VLE called Moodle

would be heavily considered. This is for a number of reasons; firstly being that it is open source. This

will cut costs down significantly along with the need for additional technical support staff. Secondly,

the features in Moodle are more refined and developed because it is open source and this means

that the features offered are made specifically because there is a need for it. Referring to the table,

it is seen that Moodle has integrated features which are more refined and sporting more modern

tools such as instant messaging and discussion forums for users to exchange ideas and to discuss

topics. Thirdly, it is very commonly used amongst colleges and some universities so its functionalityis proven. Other comparable systems will also be considered, such as Sakai. This is a less commonly


12/52

Page 12of 52

used system which is used by other colleges and universities. It also sports integrated features

similar to Moodle, but research suggests that it is more influenced by social networking websites

than being academic in nature. In addition, it isnt open source and it has to be funded which could

take its toll on the budget.

The last VLE to be considered was called Bodington. This features similar tools as the systems

described above such as instant messaging, user logins and forums. It is one of the first VLE to bedeveloped, but as of today it is no longer in development. Henceforth this system cannot be

seriously considered any further. No development of any system means that it will become outdated

quickly and will be exposed to security issues.

Key Stakeholder Interview Summary Findings

An interview with the IT manager was conducted to gain primary data from the IT, the following

statements are extracts from the interview transcript that were deemed the most important to the

investigation:

Q2) Have the staff received the same training for the use of NILE as the students?

The training that we give at the moment depends on the school that the members of staff are in.

We, as a university policy offered all staff the opportunity to come to some training sessions which

were quite comprehensive to show them more than just announcements which is the general

information that is available at the moment. So we set up a situation where schools could nominate

staff members to come to the training. Unfortunately, certain schools didnt take up our offer so

there is a real difference between the amount of training that certain schools did not send their

staff, and I personally think that that may reflect on the quality of service that they are giving

because they do not know about the opportunities, but yes, we did try to train them.

Q4) What is the most common problem students have?

I thought youd ask this question so I spoke to the frontline support staff and the most common

problem that students have is not knowing how to use the tool, so people come to us, and people

dont actually understand what to do, so for example, we had a student who wrote and official

complaint saying that they couldnt access their grades, when in fact that wasnt true. Their grades

were available, but they didnt know where to look in the tool to find their results. It is usually

misinformation where students complain because they dont really know how the tool works.

Q5) Are there constant updates on NILE or just at request?

Blackboard is the company that actually we pay for their service, and we get updates from them on a

sporadic basis, if they have dont some new updates or added new functionality then theyll tell us

you pay for this service, in your licence agreement, well send you the updates. We ourselves in

the IT team dont do any adjustments to their code, so we use the tool as they provide it, but yes,

there are updates and it is very sporadic, sometimes, in the peak period, had one time where there

was about 3 updates in a month, typically, there is one update every 6 months, but it does vary and

it is entirely down to them and their developers in the team, who tells us when updates are

available.


13/52

Page 13of 52

Q6) What are the main objectives of NILE?

For us, NILE ticks two boxes. It complies with the legal regulations that we have including about

providing the material in an electronic format for all people including people with disabilities, so we

have a legal requirement under the equality act and the provision of information in higher education

that is part of the equality act, previously called the disability discrimination act, so it is important

that we comply with that and for me, the most important one, the second one, is providing a good

quality of service for students. Ultimately, we want to make sure that students enjoy their time here

at the University of Northampton that students can find the information they need, and its

absolutely imperative that I believe that NILE is used to its full potential in order to meet the second

criteria of student satisfaction and experience.

Q11) The University of Northampton is a business organisation. Would you consider that to

improve NILE you would have to invest money in blackboard as a business? In the market rightnow there are open source systems that can be improved, are there facilities in the university that

could do that?

Absolutely, there is already the two sums of money that I have previously mentioned which show

that we are committed. The 250,000 for support and 180,000 for the licence show that we take it

very seriously. If you prove that what we are offering at the moment isn't good enough then I would

press for at the top levels such as the vice chancellor to ask for more money, I think that it is very

important. If you can prove that NILE as it is isn't working then 100% you have my guarantee that I

would take it to the management top level requisites of this institution and tell that that because it

isn't working, we need another solution. So I am open, even though it would be very hard as wehave had a relationship with blackboard for a long time, if you could prove with your investigation

then you will have my commitment to making changes.

Questionnaire DevelopmentUsing the problem domain review, a list of questions to ask students was created and a basic paper

based survey was made. As online survey would improve efficiency by increasing distribution and

analysis, potential online survey tools were investigated.

KwikSurvey was chosen as the survey tool used as this allowed logical sequencing and unlimitedquestions. Logical sequencing allows more specific questions to be asked without asking questions

that are not applicable to the user, and unlimited questions was needed as most survey tools limited

the questions to ten when an estimated twenty questions was needed.

The survey was distributed to six students as a sample to confirm the survey effectively gathered

feedback. Five completed it with no issues and had no suggestions or feedback to give and the sixth

student gave feedback of:

Question 23 was probably the most difficult to fill out as some features would not affectme.

Didn't really notice them[the heading text] until they were pointed out.


14/52


15/52

Page 15of 52

11 Is there a relationship between the

number of training hours a student

receives and the percentage of

students who have not had technical

difficulties

Correlation R = 0.683088407 N

12 Is there a relationship between the

number of training hours a student

receives and percentage whomanaged to resolve the issue

Correlation R = 1 But data is too

small to draw a

definitiveconclusion

N

To find the relations, certain questions had to be converted into weighted averages as they used a

non-numerical scale. All of the weighted averages use the same; 2 is strongly agree with the

statement, 1 is agree, 0 is neutral, -1 is disagree and -2 is strongly disagree. Below are all the graphs

of the relations:

Graph 1Training hours compared with Average number of problems a year

0

0.5

1

1.5

2

2.5

3

0 1 2Averagep

roblemsayearperstudnet

Training hours

Training hours vs Average problems a

year


16/52

Page 16of 52

Graph 2Training hours compared with satisfaction

Graph 3Type of training compared with satisfaction

0.75

0.8

0.85

0.9

0.95

1

1.05

0 1 2

Satisfac

tion

Training hours

Training hours vs Satisfaction

0.92

0.94

0.96

0.98

1

1.02

Talk from lecturer IT Specialist Video Web-based

Satisfaction

Type of training

Type of training vs Satisfaction


17/52

Page 17of 52

Graph 4Training hours compared with easy to use

Graph 5Average number of problems a year compared with satisfaction

0

0.2

0.4

0.6

0.8

1

1.2

None 1 hours 2 hours

Easeofuse

Hours of training

Training hours vs easy to use

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3 4 5 6 7 8 9 10 11 12 13

Satisfa

ction

Number of problems

Average number of problems a year vs

Satisfaction


18/52

Page 18of 52

Graph 6Average number of problems a year compared with if NILE helped the students learning

Graph 7Year of study compared with satisfaction

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 2 4 6 8 10 12 14

NILEhelpedmylearning

Average number of problems a year

Average number of problems a year vs Nile

helped my learning

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1 2 3 4

Satisfaction

Year of study

Years of study vs satisfication


19/52

Page 19of 52

Graph 8School compared with Average number of problems a year

Graph 9School compared with NILE usage every week

0

2

4

6

8

10

12

14

School of the

Arts

Northampton

Business

School

School of

Education

School of

Health

School of

Social Science

School of

Science and

Technology

Averagenumbero

fproblmsayear

School

School vs Average number of problems a

year

0

1

2

3

4

5

6

School of the

Arts

Northampton

Business

School

School of

Education

School of

Health

School of

Social Science

School of

Science and

Technology

NIleusageeveryweek

School

School vs Nile usage every week


20/52

Page 20of 52

Graph 10Type of training compared with average number of problems a year

Graph 11Nile weekly usage compared with average number of problems a year

0

2

4

6

8

10

12

14

Talk from lecturer IT Specialist Video Web-based

Typeoftraining

Type of training

Type of training vs number of problems

0

2

4

6

8

10

1 2 4 5 6 or moreAveragenumberofprblemsayear

Weekly useage

Nile weekly usage vs Averge number

of problems a year


21/52

Page 21of 52

Graph 12Nile weekly usage compared with satisfaction

Graph 13Alternative VLE statements

-1.5

-1

-0.5

0

0.5

1

1.5

0 1 2 4 5 6 or moreSatisfaction

Nile weekly usage

Nile weekly usage vs satisfaction

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

Nile is easier to use Nile has better

functions

Nile has helped me

more

Levelofagreementwiththestatement

Statements

Alternative VLEs statements

Moodle

Other


22/52

Page 22of 52

Graph 14Year of study compared with average number of problems a year

Graph 15School compared with satisfaction

0

2

4

6

8

10

12

14

1 2 3 4Averagenumberofproblmesayear

Year of study

Year of study vs Average number of

problems a year

0

0.2

0.4

0.6

0.8

1

1.2

1.4

School of the

Arts

Northampton

Business

School of

Education

School of

Health

School of

Social Sciences

School of

Science and

Technology

Satisfcation

School

School vs Satisfaction


23/52

Page 23of 52

Graph 16Type of student compared with satisfaction

Graph 17Potential features compared with agreement with statement

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

International student Non international student

Satisfcation

Type of student

International vs satisfication

0 0.5 1 1.5

A section dedicated toCourse file searchQuestion creation

Peer feedback toolCourse blogs

Course glossarySurvey tools

Instant messagingJournals

Social networkingPodcads

Detailed user profilesVoice tools

Weighted Average

Featuresw

anted

Which features would you like to see

added to Nile?


24/52

Page 24of 52

Graph 18Current features compared with percentage of users who dont use the feature

Graph 19Current features compared with agreement with statement

0 20 40 60

Module annoucements

Email notifications

Personalized pages

SmartPhone app

Discussion boardsCourse blogs

Calendar

Nile search engine

Course wikis

Percentage of users who don't use the feature

Feature

Percentage of users who don't use

feature

0 0.2 0.4 0.6 0.8 1 1.2

Module annoucements

Email notifications

Personalized pages

SmartPhone appDiscussion boards

Course blogs

Calendar

Nile search engine

Course wikis

Weighted average for the feature

Feature

Which features are liked, disliked or

not used?


25/52

Page 25of 52

Graph 20Training hours compared with NILE helped improve my learning

Graph 21Training hours compared with percentage of students who have not had technical

difficulties

Conclusion and Report RecommendationsThe following information was gained from the IT manager:

Certain members of staff do not accept training Students typically misunderstand the tool Blackboard is updated on average every 6 months sometimes with new features The IT team do not change the code of Blackboard

0

0.2

0.4

0.6

0.8

1

1.2

0 hours 1 hour 2 hoursNILEhelpedimrpove

mylearning

Training hours

Training hours vs NILE helped improve

my learning

0

20

40

60

80

100

120

0 hours 1 hour 2 hoursPercentwhohavenothadtechnical

difficu

lties

Training hours

Training hours vs Percentage who

have not had technical difficulties


26/52

Page 26of 52

Correlations have been found between the following:

Training hours and average problems a year Training hours and satisfaction Training hours and if users found NILE easy to use How often a user has a problem with NILE and their satisfaction with NILE Average number of problems a year and if users felt NILE helped their learning Students weekly usage of NILE and their satisfaction Students year of study and their average number of problems a year Number of training hours a student receives and if they thought NILE helped improve their

learning

Report recommendations

Recommendation summary:

Increase training hours to two hours as a minimum Train two times a year for students of all years Make training compulsory for both teachers and students When there is a large update for blackboard update the lecturers with any new features of

that update

Create a video tutorial and use this for the training, as well as sending certain schools extrasupport

Initial Analysis of the results show the number of problems students experienced have had a direct

impact on both performance and enjoyment as shown by the direct correlation between averagenumbers of problems a year, compared with student satisfaction and if students found NILE helped

with their learning. This could negatively affect student performance and the reputation of the

University.

Further analysis of the results indicates that many issues students have with NILE are

misunderstandings of the product rather than flaws in the system itself. This is supported by 90.9%

of students being able to resolve their issues with NILE, the strong correlations between training

hours, compared with satisfaction and ease of use, as well as the strong negative correlation

between training hours and number of problems. Because of this and statements two and fourfrom

the IT manager suggesting this, improvements to training is a reasoned response to improve usersatisfaction with NILE.

The results show a greater increase of satisfaction and ease of use from zero training hours to one,

than one training hours to two training hours. This suggests that even a small amount of training is

required for students to gain a satisfactory understanding of NILE. Although an increase of zero

training to one hour training shows an increase in the number of problems, this is likely because

students are using more advanced features of NILE which have not been fully explained to them and

this would explain why a large reduction in the number of problems is found at two training hours.

This suggests that training should be compulsory and two hours is required as a minimum for

students to fully understand all features of NILE that are available for them to use.


27/52

Page 27of 52

The relations between training hours compared with satisfaction and ease of use have a logarithmic

regression of y = 0.1267ln(x) + 0.866 and y = 0.1702ln(x) + 0.8254 respectively, this equals a

weighted average of 1.0052 and 1.01238 respectively for 3 training hours. Although this is an

increase, the benefit is quite small and will likely be affected by diminishing returns which will

decrease the benefit further. Therefore two hours training should be sufficient for students.

Graph 22 - Logarithmic regression for training hours compared with ease of use

Graph 23 - Logarithmic regression for training hours compared with satisfaction

The student survey results show that the most effective at reducing average number of problems a

year is video training and video training shares the highest satisfaction. This would suggest the mosteffective and efficient way would be to create a universal video tutorial for students by using the

0

0.2

0.4

0.6

0.8

1

1.2

None 1 hours 2 hours 3 hours

W

eightedaverage

Training hours

Prediction for Training hours vs easy to use

0

0.2

0.4

0.6

0.8

1

1.2

None 1 Hours 2 Hours 3 Hours

WeightedAverage

Training hours

Prediction for Training hours vs Satisfaction


28/52

Page 28of 52

current video tutorials for students to use during training and shows all features which is currently

lacking from current student support.

The School of Health showed the highest level of average number of problems followed by the

Northampton Business School and School of Education, and so these would greatly benefit from

additional support such as an IT Specialist.

The results show a high correlation between year of study and average number of problems and no

correlation has been found between weekly usage of Nile and number of problems as well as year of

study and satisfaction. This would suggest that training is required for all levels as most problems are

not resolved easily with experience, but are not flaws with NILE because 90% of all problems were

resolved. Training twice a year and for all years will allow users to be updated for new features and

features more suited for later years.

Five features had over 35% not used, although some features are likely not used due to personal

preference, most are likely not used because staff do not promote those features therefore students

do not use them. Together with the IT managers statement twoin which he criticized certain

members of staff not accepting training, training should be made compulsory for staff members to

ensure they utilize NILE to its full potential. In addition to this, the IT managersstatement five

stated that the updates to NILE where on average every 6 months, as such to ensure staff are

knowledgeable about new features, small training sessions for staff should employed after every

large update.

As the NILE usage increases, so does satisfaction and the number of problems, this indicates that

NILE has good features which users want but these features have not been fully trained and so

problems occur. This would indicate that more features are not required but could further increase

satisfaction and performance. And as 76% of students who completed the survey agreed or strongly

agreed that NILE helped their learning NILE is mostly suitable for the University of Northampton.

Only three features are widely wanted by users; a section dedicated to group work, a question

creation and a course file search. The features are offered by Moodle but these features wouldnt

justify the cost and effort of changing to a different system.

Moodle was the most highly considered comparable VLE but the student survey results shows that

most students who have used Moodle in the past prefer NILE. And as the University of Northampton

does not employ coders as stated by the IT manager answer five, Moodles open source strength

would not be fully utilised. Because of this, Moodle is not a suitable replacement to NILE.

The Sakai and Bodingtons systems which were the other comparable VLEs investigated into, were

also found to be unsuitable for the University of Northampton.This is because both of the systems

do not offer anything unique, and as no student who answered the survey had used either system,

no appraisal could be made.

Because of this, no other VLE has been found to be more suitable for the University of Northampton

than NILE.


29/52

Page 29of 52

Future work

Although no other VLE was found to be more suitable, students did slightly prefer an unknown

system, the survey tool did not allow an other text field to be used so this system was not

identified. Further investigation would be needed to identify and assess if the unknown system is

more suitable for the University of Northampton than NILE.

Although training should improve user satisfaction with NILE, it may not be the best system available

for the University of Northampton. Further investigation is needed to identify the other VLE that

students preferred over NILE, as this might be a more suitable system.

Increased survey distribution for more accurate and reliable results.

Three areas of the problem domain were not investigated into; the key system stakeholder, the

current staff and relevant legislation. To increase the recommendations accuracy an investigation

needs to be started for each of these domains. This could be achieved by modifying the student

survey and using it to gather staff feedback, a legal expert to be hired either from the University or

independently who would ensure that the system is legal in the UK and interviews with the key

system stakeholder to ensure there requirements corresponds with the recommendations solution.

International satisfaction was found to be over half the satisfaction of non-international students,

further investigation is needed to ensure NILE is also suitable for international students.

Critical evaluation

The student survey which forms the main support of the conclusion has a very low participation ratefor the size of the University. For more reliable results 10% of the student population would be

needed which is an estimated 1039 (justcourses.com, n.d) students. This has caused potentially

anomalous results such as with the best type of training where only one person replied with video

causing it to have a skewed result.

The student survey did not ask the type of problem or more information about the problems users

had with NILE. A potential error with the questioning of the survey occurred where several students

answered they had technical difficulties with NILE yet later they also stated they never have had

problems with NILE. This limited the analysis and accuracy of the recommendation. Other problems

with the survey included:

Several grammatical errors Several general errors Weak scales Several important questions are not required Didnt account for students with disabilities No questions about support documentations

The distribution method would also need improving, as method used was to get as many results as

possible but at the cost of a proportioned distribution e.g. between gender and schools. This could

be accomplished by simply targeting certain areas of the campus which are specific to the criteria.


30/52

Page 30of 52

Blackboard (the creator and provider of NILE) has several different licences which have different

levels of features depending on the price. Because of the sensitive nature of NILE the licence was not

disclosed as stated by the IT manager on statement Q11. As such , the investigation was not able to

differentiate between features the current system has and features which required a higher licence.

The investigation into the comparable systems was limited to only the internet because of the short

time period of the research. As such, alternative investigation methods such as direct contact withmarketing teams and direct use of comparable systems were unable to be employed which limited

the information gained and used. And the research into bodingtons was weak because it has been

discontinued and little documentation has been created for it.

To reduce research costs, a free survey tool was used, this meant that certain features were unable

to be used in the survey such as the other text field. This caused weakness in the survey such as

when asking for the student which alternative VLE they had used.

The investigation was not allowed to contact University staff, and as the investigation didnt have

the funding to hire an independent legal expert, the current staff, key stakeholder and relevant

legislation problem domains were not fully explored which has restricted the reports

recommendation.

Background reading of relevant legislation could have been employed but this would have not given

the sufficient accuracy needed, as such a subject expert would be required.

Research into both NILE and the comparable systems was insufficient, more detail should have been

investigated and large sections should have been separated into subsections such as with the

assessments, which could include online assessments, quizzes and into features for students and

features for staff. Research into old documentation caused an error where certain features where

doubled so NILE both had and did not have the feature.

References

Just courses. [n.d.] University guide University of Northampton [online]. Sine nomine. Available

from:http://www.justcourses.com/Universities/University_of_Northampton.html[Accessed on 3rd

of April 2013]

Moodle. (October 2011) Features MoodleDoc [online]. MediaWiki. Available from

http://docs.moodle.org/20/en/Features[Accessed on the 3rd of April 2013]

Sakai. [n.d.] Sakai features[online]. Sine nomine. Available from:

https://sakai.win.tue.nl/library/content/gateway/features.html[Accessed on the 3rd of April 2013]

SourceForge. [n.d.] Bodington[online]. Sine nomine. Available from:

http://sourceforge.net/projects/bodington/[Accessed on the 3rd

of April 2013]

Blackboard. [n.d.] Blackboard Learn | Blackboard Learn Platform Feature Showcase [online].

Blackboard. Available from: https://www.blackboard.com/Platforms/Learn/Products/Blackboard-

Learn/Features.aspx[Accessed on the 7th of February 2013]

Blackboard. [April 2010] Blackboard Learn, Release 9.1 New Features [online]. Blackboard. Available

from:http://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdf[Accessed on the7th of February 2013]
http://www.justcourses.com/Universities/University_of_Northampton.htmlhttp://www.justcourses.com/Universities/University_of_Northampton.htmlhttp://www.justcourses.com/Universities/University_of_Northampton.htmlhttp://docs.moodle.org/20/en/Featureshttp://docs.moodle.org/20/en/Featureshttps://sakai.win.tue.nl/library/content/gateway/features.htmlhttps://sakai.win.tue.nl/library/content/gateway/features.htmlhttp://sourceforge.net/projects/bodington/http://sourceforge.net/projects/bodington/https://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttps://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttps://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttp://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdfhttp://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdfhttp://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdfhttp://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdfhttps://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttps://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttp://sourceforge.net/projects/bodington/https://sakai.win.tue.nl/library/content/gateway/features.htmlhttp://docs.moodle.org/20/en/Featureshttp://www.justcourses.com/Universities/University_of_Northampton.html


31/52

Page 31of 52

Appendix

Item 1 - Questions for the IT Manager

How many complaints have been made in the last 6 months? What complaints have students been making? Has staffs made any complaints? If so typically what complaints have staffs been making? How often on average every week does someone contact the IT department requesting help? What are the most common problems students have? What web-board training do IT services provide for students and staff using the software? What support documentation is available for the software? Is this support documentation enough? I.e. does it actually help students solve problems? Do students/staff know about these documents? Do students/staff often use these documents? What opinions of the current system do the IT staffs have? Would the IT staff like to see a new software system utilised? Do the IT staffs have any recommendations of a new VLE to use?

Item 2 - Complete IT Manager transcript

Q1) Would there be the possibility to allow downtime during the transitions?

Any products that we have, have to be available to our students pretty much 24/7, 365. So, if there

was to be a new development that involved a transition then we would have to take steps, possibly

parallel implementation where the new system is brought in over the old one, but we really cannot

afford to drop our quality of service. If there was to be any downtime, then it would likely be, with

the students in mind, I would suggest it being on Sunday, at 3 in the morning, in the off-peak times

where we are unlikely to experience any traffic. But the long and the short of the answer is that wewould expect that the service would be available 24/7, 365.

Q2) Has the staff received the same training for the use of NILE as the students?

The training that we give at the moment depends on the school that the members of staff are in.

We, as a university policy offered all staff the opportunity to come to some training sessions which

were quite comprehensive to show them more than just announcements which is the general

information that is available at the moment. So we set up a situation where schools could nominate

staff members to come to the training. Unfortunately, certain schools didnt take up our offer so

there is a real difference between the amount of training that certain schools did not send their

staff, and I personally think that that may reflect on the quality of service that they are giving

because they do not know about the opportunities, but yes, we did try to train them.

Q3) What is your role in the management of NILE?

I am the top level manager and I have a brief from the institution that ultimately, I have to provide

quality of service to students throughout the year. I have a budget for which is currently a 250,000

which pays for the IT staff who support NILE. So the 250,000 is for servicing NILE and does not

include the cost of actually buying the product, and us purely for the IT support staff who work on

NILE. Again, I am not sure that were are really getting value for money at the moment, and I am

really interested to find out the data that youve got to say whether we should be going in a

different direction, but ultimately, the book stops here when it comes to NILE.


32/52

Page 32of 52

Q4) What is the most common problem students have?

I thought youd ask this question so I spoke to the frontline support staff and the most common

problem that students have is not knowing how to use the tool, so people come to us, and people

dont actually understand what to do, so for example, we had a student who wrote and official

complaint saying that they couldnt access their grades, when in fact that wasnt true. Their grades

were available, but they didnt know where to look in the tool to find their results. It is usually

misinformation where students complain because they dont really know how the tool works.

Q5) Are there constant updates on NILE or just at request?

Blackboard is the company that actually we pay for their service, and we get updates from them on a

sporadic basis, if they have dont some new updates or added new functionality then theyll tell us

you pay for this service, in your licence agreement, well send you the updates. We ourselves in

the IT team dont do any adjustments to their code, so we use the tool as theyprovide it, but yes,

there are updates and it is very sporadic, sometimes, in the peak period, had one time where there

was about 3 updates in a month, typically, there is one update every 6 months, but it does vary and

it is entirely down to them and their developers in the team, who tells us when updates are

available.

Q6) What are the main objectives of NILE?For us, NILE ticks two boxes. It complies with the legal regulations that we have including about

providing the material in an electronic format for all people including people with disabilities, so we

have a legal requirement under the equality act and the provision of information in higher education

that is part of the equality act, previously called the disability discrimination act, so it is important

that we comply with that and for me, the most important one, the second one, is providing a good

quality of service for students. Ultimately, we want to make sure that students enjoy their time here

at the University of Northampton that students can find the information they need, and its

absolutely imperative that I believe that NILE is used to its full potential in order to meet the second

criteria of student satisfaction and experience.

Q7) If it comes back that in the research that there is an alternative system that would be bettersuited to the universitys needs, would you be open to exploring other systems? And if so, are

there any constrains on that such as budgets and features?

If you could prove to me that NILE as it is, is not fit for purpose, then I would be open to

recommendations to take the product forward to either get a new product, or improve what we

have got for more services with blackboard and extra information, then yes, I would consider

changing system. It would be difficult because obviously, we have used NILE for a long time, weve

used blackboard, but I think ultimately, if you could prove that student satisfaction was really poor,

that wasnt something to do with training or it genuinely is a poor product, then we would. In terms

of the second question, which related to budgets, we have the 250,000 which is for the support at

the moment, and we pay 180,000 every 24 months to blackboard. At the moment, we are tied in

until September this year (2013), so any funds if you suggested another approach, we obviouslywouldnt renew the contract and that 180,000 could be used for a new product.

Q8) As IT manager, when complaints are made, and improvements are suggested, though you are

a key figure within the domain, are you aware of the problems and do you suggest solutions

yourself, or do you have a specific team dedicated to do so?

The truth of the matter is that I do my best to keep up to date with complaints that come in, so I

regularly talk to the support teams to ask how things are going, ask what the student reactions to

upgrades are and I am very conscious of the first few weeks when the upgrade has been

implemented to see whether there is any negative feedback. Truthfully though, there are probably

occasions where I dont get told, but I try my best to keep abreast of whats coming in and I wouldlike to think that if complaints continually came in, which is the reason why you are carrying out this


33/52

Page 33of 52

study, and we have got a problem, that I would respond to that and I would use my resources (the

250,000 and the 180,000 for NILE itself) in a different scenario, so I do my best.

Q9) Is there flexibility for new features to be added to NILE? If the plug-in was not available by

blackboard, how would you go about adding one in?

There is yes. The package that blackboard offer us at the moment (the 180,000 has the options to

add plug-ins for additional services. I suppose the down side is that for any new plug-ins that we add,

it would cost us more in terms of an extra renewal fee, but yes, there is a possibility to add more ifyou felt that the product at the moment doesn't offer a good enough service. We don't have staff in

the IT team who are competent enough to recode the product. So if there is a feature that we would

like to include that doesn't come from blackboard then at the moment we would be stuck and that

could lead on to looking at a different product or changing our contract and look for a different

provider such as Moodle or the others out there, so yes we can't make adjustments ourselves, we

rely on blackboard and what they come up with.

Q10) Would you consider reforms of policies associated with NILE that apply to lecturers possibly

making NILE training compulsory so that they can comply with your standards and your legislation

about electronic material being available would that be a possibility?

In the past, we have always tried to be nice by offering the training as support to them, but iflecturers don't take advantage as you know, it could obviously impact on the service. If you could

prove to us, possibly by looking at patterns in satisfaction that training makes a difference to the

student experience, I could possibly take that to the vice chancellor and go about making the

training compulsory. So the answer to your question is yes, but I would need some strong evidence

so the results that you would get to us in a report would be very important.

Q11) The University of Northampton is a business organisation. Would you consider that to

improve NILE you would have to invest money in blackboard as a business? In the market right

now there are open source systems that can be improved, are there facilities in the university that

could do that?

Absolutely, there is already the two sums of money that I have previously mentioned which showthat we are committed. The 250,000 for support and 180,000 for the licence show that we take it

very seriously. If you prove that what we are offering at the moment isn't good enough then I would

press for at the top levels such as the vice chancellor to ask for more money, I think that it is very

important. If you can prove that NILE as it is isn't working then 100% you have my guarantee that I

would take it to the management top level requisites of this institution and tell that that because it

isn't working, we need another solution. So I am open, even though it would be very hard as we

have had a relationship with blackboard for a long time, if you could prove with your investigation

then you will have my commitment to making changes.

Q12) What kind of support documentation is available for the software, and how available is it, so

do the staff and students know whether it is available?I've mentioned the support team that is funded by the 250,000. In that team we have a couple of

learning technologists, and they are responsible for documenting NILE in terms of the features that

we offer to staff and students and providing support material. So if you go onto NILE at the moment,

anywhere that it says or gives you a link to help that support documentation has been created by us.

So absolutely, we want to make the training and support as good as possible, not only do we do the

training during induction for staff and students, where they were invited, but we also produce lots

of online training material and the last piece of material that my learning technologists produced

was a tutorial for students on how to use the blog in NILE, again whether staff and students are

using this information is a different question, but we are absolutely committed to producing so

quality support.


34/52

Page 34of 52

Q13) How do you manage usage overload on NILE?

As part of the agreement we have, the 180,000 service agreement, our servers are paid for, and

believe it or not, are based in the Netherlands, and when we paid for the service, we predicted the

amount of traffic that we had and then added an extra 20% capacity, so theoretically, we have

enough server capacity to cater for the existing student body plus spare, so if there was a massive

intake of students, so if 2000 more people came to the university next year, we've still got enough

server capacity to cope with that. It would be extremes that we would have to pay extra money for

to not be enough, and hopefully, I'd like to think that students don't have difficulty logging on andthat availability for NILE has been good, and that regardless of how many people are online, we

don't have any noticeable degradation of performance of the system.

Q14) Is there any functions for areas of NILE which you believe could be added or improved on?

I take the advice of two parties here. Our IT staff who deal with problems when students raise them,

and I have to listen to their recommendations. And I also like to keep abreast of all the information

that comes though from blackboard themselves, so they have a research team and a team that

develop the software, and if they tell us that they could add a new feature to NILE then I listen to

that, but decisions on upgrades and where we go in and how we deal with the product basically

come from staff experience when dealing with students and from NILE blackboard

recommendations.

Q15) In terms of training for both students and staff, is it externally sourced or do you create it?

And if there is a problem, would you be willing to revise the way that it is provided?

Training at the moment was initially supplied by blackboard. Once our IT team were trained,

everything has been serviced by us, and we don't pay blackboard for training. In terms of if it was

found to be inadequate, I would 100% put everything we have into resolving that whether that

would mean making a new policy within the institution about how people are trained and I would

support any recommendations that you make.

Q16) If NILE had a refurbishment, would there be any parts of the website that you would like to

keep that same?The interface for logging on at the moment, we would like to keep. Even if the services would

internally change, we would like to keep the portal the same, and the reason why I say that is

because we have done a lot of work recently in linking up NILE and the sites to our grading system so

tutors are supposed to use the tool within NILE called grade mark and put the grades that have been

achieved by students in there, and the fact that we have linked our portal to our university system

for grading, we wouldn't want to change. So to go back, the one thing that we would need to keep

the same would be the overall infrastructure of the access portal, but we would be prepared to add

new features inside.

Q17) The initial concepts of NILE originated from helping educational organisations in different

aspects such as organising documents in one easily accessible area, when complaints are made,does this hinder the product or does it still succeed?

I really try to take notice of what the student experience is and whether they are happy or not and I

ask my IT staff to log a number of complaints even if they don't tell me the exact details. Over the

last 6 months, remembering that we have around the region of 9000 full time students and another

3-4000 distance learners, we've had complaints about QOS in the region of around 2/3 a month. The

stats are updated regularly and we have peak periods of dissatisfaction what is usually around the

start of the year where people have problems with registration and that it probably goes to about

20, but there are months where there are not complaints at all, so based on that, I feel that the

service is reasonably successful.


35/52

Page 35of 52

Q18) Would you tie down the common problems that students get to the training that is provided

to them?

One of the things that we have been trying to do is when common problems materialise, we try to

write help or tutorial aimed at those problems. The numbers that I have quoted earlier are written

complaints. There is a possibility that there is dissatisfaction, but it may just be where people have

complained and IT services have solved the issue and not logged it. So it is a possibility that there are

more complaints that previously quoted, but the just don't go any further than the student phoning

in and getting the issue solved.

Q19) As there are a lot of students on NILE every day, do you ever experience any performance

issues?

We are really happy that the servers back-up system that we have for data such as lectures and

material as we have never lost any data or unable to access it for anything longer than maybe 24hrs

and that is in all the years that we have used the software since the early 2000s. Generally, we are

happy with the service from a technical perspective.

Q20) Are those that have disabilities of a lower priority than those without when they have issues

with the software?

No, that's wrong. The disabled students are actually a high priority because of the legislation that thegovernment has put down. If we were seen to be ignoring people who were disabled, not only would

it give a bad impression for people outside of the institution, but we could be fined and liable to

prosecution. So you will find the opposite. We are actually really concerned about that region. There

are situations where we have done our best to provide a good service, we actually have a blind and

partially poor hearing joining us next year where we have already done the learning

recommendations for their disability, so we do our absolute best to deal with each disabled student

by looking at their needs and trying to make sure that generally, that NILE is compliant with the

legislation and we do our best.

Q21) Are you happy with the service that youve got at the moment?

A: The service can be put into two parts, Blackboard releases updates, and technical informationavailability of service I am happy .In terms of my satisfaction with the way the products being used,

no. I get very frustrated

Q22) I mean exactly with NILE, not with our university. We are paying money for our university to

manage it and for the product (NILE).

A: Yes. Im happy with what weve got at the moment, the value for money that we get. For

180,000 I think it does represent a good figure when you look around the market and you look at

the service we get.

Q23) You said that the login site is important to you, the way it is at the moment. Are there any

other features that you would particularly like to see?A: Yes. The login is absolutely essential because its tied in with student IDs and grades, we couldnt

change that dramatically. In terms of the other features within NILE, I think what weve got isnt bad

but its important Im ready to listen. If you do a study and find that 95% of students think we need

an extra feature and we havent got it then I would be willing to invest extra money. Or alternatively,

looking at another product if it is essential.

Q24) Theres probably people that have problems with NILE and dont make a complaint.

A: I really believe in the training. Like you say, there are possibly people out there that are not

happy, who dont complain and just put up with what theyve got. I would advocate training earlier

off.


36/52

Page 36of 52

Q25) Has there been any research like this done since the system was introduced?

A: The research that weve had has been only a collection of the complaints. The project that youre

doing at the moment is the first time weve commissioned software engineers to look into this issue.

Q26) We were told that if people are unsatisfied with the service what is the process to change it?

You are the manager, its your decision to change it. What happened there?

A: As a team with me as the lead weve come up with a policy for effectively service and what the

service includes are announcements, grade marks etc. Theres a written document for what webelieve weve got. If we wanted to change that, the first thing we would need to do is based on your

recommendation wed give a reason for the change. If I wanted to do it then it would go to the top

level management university executive which includes the pro vice chancellor. I would put forward

my case and explain that based on the evidence we need to change this, and then he would either

give me the go ahead, if it required extra funding then that would be on a table to say we need x

number of pounds. He would then say no and we would have to leave it as it was but basically I take

the recommendation forward to the management assuming theyre happy and would make the

change.

Q27) When you chose Blackboard did you choose it for the applications the universities already

had or for the applications blackboard had?A: The infrastructure weve got as a university was a consideration when we took them on-board.

We wanted to include things like compatibility with our existing legacy database for the library and

things like that. What Ive found now though is that Blackboard is usually the standard and then

were changing our technology to comply with that. Initially we were worried about compatibility

but now were more than willing to change.

Item 3- Basic paper draft

Intro:

Basic info:

Gender:

Male [ ] Female [ ]

Which school do you study within?:

Arts [ ] Sciences [ ]

Are you fulltime distance?:

Yes [ ] No [ ]

Are you an international student?:

Yes [ ] No [ ]

Year:

1st [ ] 2nd [ ] 3rd [ ] 4th [ ] Above [ ]


37/52

Page 37of 52

Usage:

How often do you use Nile on a weekly basis?:

Multiple times daily [ ] Daily [ ] Multiple times weekly [ ] Less [ ]

Do you find Nile easy to navigate?:

Strongly agree [ ] Agree [ ] Neutral [ ] Disagree [ ] Strongly disagree [ ]

How happy are you with Nile overall?:


Have you ever had any issues with Nile previously?:

Yes [ ] No [ ]

If yes, what sort of issues?:

Did IT support help solve your issue(s)?:

Yes [ ] No [ ]

Features:

Which features do you use, and how do you like them?:

Feature

Like

Dont like

Don't use / Didn't know


38/52

Page 38of 52

Would you like to see AB included as a feature of Nile?:

Yes [ ] No [ ]

How satisfied are you with the features of Nile?:


End

Have you got any other comments about Nile?:

Any comments about this survey?:

Ending statement/thanks

Item 4- Complete questionnaire used

The full questionnaire can be found at the following link:

http://kwiksurveys.com/app/rendersurvey.asp?sid=g3nl352xm2u20yc108222&refer=
http://kwiksurveys.com/app/rendersurvey.asp?sid=g3nl352xm2u20yc108222&referhttp://kwiksurveys.com/app/rendersurvey.asp?sid=g3nl352xm2u20yc108222&referhttp://kwiksurveys.com/app/rendersurvey.asp?sid=g3nl352xm2u20yc108222&refer


39/52

Page 39of 52

Item 5 - Raw results from questionnaire

Total Male 24

Total Female 28

Total 52

First Year 24Second Year 8

Third Year 17

Fourth Year 3

Fifth Year 0

Degree 45

Foundation 2

HND 3

Post-Grad 2

Full-Time 52

Part-Time 0

School of the Arts 15

Northampton Business School 11

School of Education 6

School of Health 4

School of Social Sciences 5

School of Science and Technology 13

Northampton 51

Moulton College 1

Tresham 0

Are you international

Yes 6

No 46

Did you receive training for Nile?Yes 30

No 22

Do you believe this training help?

Yes 23

No 7

Do you believe this training was enough

for you to use NILE well?

Yes 21

No 9


40/52

Page 40of 52

How long was the training?

1 Hours 27

2 Hours 3

3 Hours 0

4+ Hours 0

What type of training was it?

Talk from Lecturer 21

Talk from IT specialist 10

Video 1

Web Based 2

How often on average do you use Nile

on a weekly basis?

0 1

1 6

2 4

3 9

4 13

5 18

6 or more 0

What are your thoughts on the

statements below?

Strongly

Agree

Agree Disagree Strongly

Disagree

Overall I am very statistfied with Nile 8 37 6 0

Nile is easy to use 10 34 6 1

I would like to take another course using

NILE

3 15 26 7

Nile helped improve my learning 3 36 9 3

Have you ever had any technical

difficulties with NILE?

Yes 22

No 29

Did you manage to resolve the issue?

Yes 20

No 2

What would/did you use to help resolve

technical difficulties with NILE?

IT help desk 7

NILE help tab 4

A member of staff or another student 10

Other 5


41/52

Page 41of 52

Roughly how often on average do you

have problems with NILE?

Never 4

Once a year 8

Once a month 10

Once a day 0More than once a day 0

Have you ever used another virtual

learning enviroment (VLE) before such

as Moodle, Frog or Sakai etc

Yes 17

No 34

Which VLE have you used? (You may

tick more than one answer)Moodle 11

Frog 0

Sakai 0

Boddingtons 0

Other 7

What is your opinion on the statement

below?

Strongly

Agree

Agree Disagree Strongly

Disagree

Nile is easier to use than other VLEs I

have used in the past

3 8 5 1 17

Nile has better functions that other VLEs

I have used in the past

3 10 4 0 17

Nile has helped me more than other

VLEs I have used in the past

4 6 7 0 17

Which features do you like, dislike or

don't use?

Strongly

like

Like Dislike Strongly

Dislike

Don't use

Personalized pages 4 31 3 0 13

Email notifications 11 27 6 1 6

Module annoucements 15 29 6 0 1

Calendar 3 25 3 1 19

SmartPhone app 6 24 1 2 18

Nile search engine 3 23 5 0 20

Discussion boards 2 30 4 1 14

Course blogs 2 26 3 0 20

Course wikis 2 16 5 1 27

Which features would you like to see

added to Nile?

Would

strongly

like added

to NILE

Would

like

added to

NILE

Would

not like

added to

NILE

Would

strongly

not like

added to

NILE

Wouldnt

Use


42/52

Page 42of 52

A section dedicated to group work 22 22 2 0 5

Peer feedback tool 8 34 4 1 4

Social networking 10 21 8 1 11

Survey tools 13 22 8 0 8

Course blogs 9 29 3 0 10

Instant messaging 14 20 6 2 9

Course file search 19 27 3 0 2

Voice tools 8 10 14 1 18

Course glossary 10 30 5 1 5

Journals 14 21 5 3 8

Podcast 10 19 8 0 14

Question creation 17 23 6 0 5

Detailed user profiles 8 19 9 1 14

Item 6Presentation

The following are images of the presentation to the management team, but due to the animations

certain slides have elements which overlap as such these images are not representative of the

presentation.


43/52

Page 43of 52


44/52

Page 44of 52


45/52

Page 45of 52


46/52

Page 46of 52


47/52

Page 47of 52


48/52

Page 48of 52


49/52

Page 49of 52

Item 7 - Presentation transcript


50/52

Page 50of 52

Hello and welcome to team 2s presentation!

[Next slide]

The team was hired to investigate the Northampton Integrated Learning Environment to ensure it

meets a required standard. This presentation shows the results and conclusions of this investigation.

[Click]

Before we start let us quickly explain why you should care. Our investigation found that there is a

direct correlation between the average number of problems a student has compared with

satisfaction and if they thought NILE helped their learning. Based on this information it is likely that ifNILE is unsuitable or not the best product on the market then NILE is damaging the University's

reputation and costing you money.

[Next slide]

We started out the investigation by creating aims and objectives to keep the research focused.

[Click]

And then we created a list of objectives required for us to fully access and evaluate this aim.

After we had identified the problem domains we began investigating three areas; Starting with the

current product specification

[Click]

Here we have a list of NILEs features, but we also investigated the Embedded tools which includedthe HTML content editor, Help and support mechanisms mostly provided by the University which

include PDFs and videos, and finally NILEs Software Expansion Potential where add-ons can be added

to customise NILE.

[Next slide]

The second problem domain was the comparable systems

[Click]

We initially found three popular virtual learning environments

[Click]

Then by researching there features and comparing them to NILE we created this features table

which has been provided in your hand out.

The final problem domain investigated was the IT Support Help Desk Staff. We did this by

interviewing the IT manager

[Click]

Here we have the most important statements he provided us with.

Using the problem domain review, a list of questions to ask students was created and a basic paper

based questionnaire was made. As online survey would improve efficiency by increasing distribution

and analysis, potential online survey tools were investigated.

[Click]

KwikSurvey was chosen as the survey tool to use as it allowed unlimited questions unlike other tools

researched into.[Next slide]

After internal testing of the questionnaire had finished, we used a small sample of students to

confirm the questionnaire effectively gathered feedback. To do this we asked them to complete the

survey and then asked them several basic questions about the questionnaire.

[Click]

The sample consisted of six students and only one had any significant feedback, which is as shown,

one was about the text explaining the reasoning of the question the other was about an option for

one of the questions.

[Click]

Using this feedback the questionnaire was improved and was then ready for complete distribution.

[Next slide]To distribute the questionnaire we chose high foot traffic areas in each campus and moved around


51/52

Page 51of 52

these areas using laptops and tablets to get the largest range of feedback possible. To increase

student participation we used incentives to students who completed the survey.

The next relation we looked into was between training hours and average number of problems a

year, although there is an increase in problems going from no trainings hours to one, we believe this

is explained because students use a higher number of features.

[Click]

Here we investigate training hours against several other areas such as satisfaction. Thequestionnaire asked how strongly they agreed with several statements and this was converted into

weighted averages on a scale where 2 is strongly agree, 1 is agree 0 is neutral, -1 is disagree and -2 is

strongly disagree.

As you can see there is a significantly higher satisfaction, ease of use and improved learning with

higher training hours.

[Click]

Here we have created a prediction for three training hours using a logarithm regression line. As you

can see while there is an increase, the benefit is small and will likely be reduced further by

diminishing returns.

Because of this information, we believe the first change should be to make training two hours as a

minimum as one training hours is insufficient and three is to large[Next slide]

As shown by these graphs the most effective training is is video training as it has the lowest number

of problems and shares the highest satisfaction. This would suggest the most effective and efficient

way would be to use the existing video tutorials to create a complete video tutorial which students

would watch during training and that would teach them all features.

[Click]

As shown by this graph, the school of health had a much larger average number of problems

compared with other schools, in addition to the video, this school would greatly benefit with extra

support such as n IT specialist.

Five features had over 35% not used, although some features are likely not used due to personalpreference, most are likely not used because staff do not promote those features therefore students

do not use them. Together with the IT managers stating that certain members of staff do not accept

training, we believe training should be made compulsory for staff members to ensure they utilize

NILE to its full potential. In addition to this, the IT managers stated that the updates to NILE where

every 6 months, as such to ensure staff are knowledgeable about new features, small training

sessions for staff should employed after every large update.

[Next slide]

The results show a high correlation between year of study and average number of problems and no

correlation has been found between weekly usage of Nile and number of problems as well as year of

study and satisfaction. This would suggest that training is required for all levels as most problems are

not resolved easily with experience, but are not flaws with NILE because 90% of all problems wereresolved. Training twice a year and for all years will allow users to stay up-to-date for new features

and features more suited for later years.

As the NILE usage increases so does satisfaction and the number of problems, this indicates that

NILE has good features which users want but these features have not been fully trained and so

problems occur. This would indicate that more features are not required but could further increase

satisfaction and performance. And as 76% of students who completed the survey agreed or strongly

agreed that NILE helped their learning NILE is mostly suitable for the University of Northampton.

[Click]

Only three features are widely wanted by users; a section dedicated to group work, a que

report final draft 1.3.1

Documents