howtousethe$jisc studentdigital experience)tracker ·...

How to use the Jisc Student Digital Experience Tracker

Jisc Student/Learner Digital Experience Tracker http://bit.ly/1WJkMPS

1


Overview This document provides information about: 1. How to access your BOS (Bristol Online Surveys) account 2. How to customise your survey using the BOS design interface 3. How to launch your survey 4. How to review the data collected during and after the survey is live

1. Accessing your BOS account Once you have applied for and been notified of a place in this pilot project, your unique surveys will be uploaded to your Bristol Online Services (BOS) account and your named contact will receive an email to say they are ready.

If you do not already have a BOS account, your email will include details about how to create an account and log in. If your institution already has a BOS account, you will log in with your existing details and click on 'Jisc Prototype Services' in the top menu. If you have lost the email inviting you to create a BOS account, please go to the main sign in screen and click on the ‘lost invitation?’ link.

You are now ready to start customising your survey.


Jisc Student/Learner Digital Experience Tracker http://bit.ly/1WJkMPS 2

2. Customising your survey

You should have read through and discussed the Guide ‘How will you customise the Tracker?' before you begin editing. If you have not already done so, you can access this from the guidance page. The following section tells you how to edit and delete questions in BOS: the customisation guide helps you decide what changes you will actually make and why.

Please DO NOT CHANGE any options other than the ones outlined here. In particular, please do not change the survey name you have been allocated, or the design and background including the Jisc logo, or any features of the survey lay-‐out.

1. If you are not already logged in, log into BOS at https://www.onlinesurveys.ac.uk and click on the 'Sign in' button, top right. Sign in using the details you have set up.

2. You will see your BOS dashboard. If you already have a BOS account, you will have to select 'Jisc prototype services' from the drop-‐down menu at the top to access the Tracker area.

3. Now you need to UNTICK the box next to 'JUST MY SURVEYS' so you can see the survey(s) we have created for you.

4. Your own institutional survey(s) should be visible. Each survey title is unique, so yours will not be the same as the one shown. If you cannot see your survey, please email [email protected] for help.

Under the survey title, click 'Design' to open an editable version of your survey.



5. You are now in the survey editing space. You will see page 1 of your. Notice that most questions cannot be EDITED or DELETED. The only questions you can change are on pages 7 and 8.

Scroll down or use the 'Jump' button to reach page 7.

6. You will already have decided which questions on page 7 you are going to DELETE. For each of these questions, you need to delete both the title of the question (e.g. ‘Digital skills’, in blue), and the question itself (everything under the title, shown in grey). Click on the cog icon top right of the title, select ‘Delete’, and confirm. Do the same with the cog icon in the grey area. This will delete the whole question with all of its parts.

7. If you are including the question about use of the VLE or online learning environment, you can EDIT the question header to replace the generic term with one your learners will recognise, i.e. the name you use for your learning environment. Click on the pen icon to the right of the header and type over the given text in the Edit window. Don’t forget to ‘Save’ the revised header when you have finished. (Note that the online version is shown here.)



8. You can use the last question on page 7 to ask learners about up to five issues that are unique to your institution. If you have decided not to use this question, DELETE both title and question as before. If you are using this question, you should already have decided on the title and question statements you will use and ideally piloted these with a group of learners. Click on the pen icon at the end of the heading (in blue) to add your own heading for the question. You can DELETE the heading if you prefer.

9. Now click on the pen icon above the column beginning ‘Option 1’ (in grey). We advise you not to change the stem of the question (‘How much do you agree…?’) or the response options (‘Agree’, ‘Disagree’ etc). Enter your chosen question statements into the box called ‘Row items in first column’, pressing return after each item. We strongly recommend you do not use more than five statements (options). Please keep these as clear and short as possible. If you need to provide ‘more information’ to clarify or exemplify your statements, you can do this under the ‘Advanced Options’ menu. ‘Save’ your whole question using the green button at the bottom.



10. Now you have finished editing page 7, scroll down to page 8. You will leave the first two questions on this page as they are (the first three in the online Tracker). You will EDIT the other two questions. Note that they will be numbered differently depending on how many questions you have used on page 7. You will already have decided how to use these two questions by working through the guide ‘How will you customise the Tracker?’

11. Click the pen icon to edit the first question. Notice that with this and the following question you can leave the default ‘Question text’ and provide your own ‘Answer options’ OR you can change the ‘Question text’ as well. To edit, simply click and type over the default text. To add a further answer option, type in the white box at the bottom of the list and click 'Add another answer option'. To delete an answer option, hover over it and click on the blue dustbin icon. We strongly recommend using no more than 5 or 6 answer options for these questions. Click the green 'Save' button to finish.

12. Edit and save the second question in the same way. (Note that your example answer options may differ from the ones shown here.) Make sure you have deleted any instructions that we included in the survey to help you.



13. You can now preview the survey as your learners will see it by clicking on the 'Preview survey' button in the authoring pane top left. You can do the same by clicking on the ‘eye’ icon from the main dashboard. If you want to circulate a copy of the survey for other people to preview (who may not have access to BOS), click on the ‘cog’ icon top right within the preview screen and select ‘Export survey as PDF’.

14. If you need to make further changes to a question, return to the 'Design' screen and click on the pen icon again. Remember that once the survey is launched you will not be able to make any changes, so double check everything now. Then return to your dashboard (top left 'Dashboard' button) ready to set the launch date.

Preview in design space:

Preview from dashboard vie eye icon:

Print as pdf to distribute:

15. We don’t currently include the facility to collect participants’ emails. If you want to offer a prize draw incentive, you can give each participant a unique ‘completion’ code (see example right) and collect these instead. To turn on this option, click on ‘Survey settings’ and tick the box ‘Show completion receipt’. In your instructions to participants you will need to tell them to copy this number from the screen and submit it to you (by email, using a linked googleform etc). Pick prize winners randomly from those you receive.



3. Launching your survey

1. If you are not already logged in, log into your BOS dashboard and locate your institutional survey as before. Click on 'Distribute' to access the Launchpad.

2. From the left hand blue block, select 'Distribution settings'.

3. By default the opening time is set in the past to allow you to launch the survey whenever you want. Notice that the closing date is set to 31 March 2017 (a different date to the one shown). For this pilot, all surveys will close at the same time to allow Jisc to finalise the benchmarking process and report back on the summative data. No-‐one can see or complete the survey until you send the link to them.

4. By default the URL is taken from the long survey name shown on your dashboard. We have created a shorter 'public survey name' for you, which students will see on the first page of the surveys. Please do not change the public survey name. You can however change the



'survey short name' to create a shorter URL. We recommend the following format: your institution name or initials dash tracker dash 2017. For example: https://jisc-‐beta.onlinesurveys.ac.uk/BreeUni-‐tracker-‐2017.

5. When you have set and saved the dates and URL, click back to the survey launch pad (under 'Distribute').

6. Check all the details, and make sure your communication channels are ready to share the URL with students. Then click the 'Launch survey' button. Now you need to distribute the URL to your target students, for example via emails, Twitter, Facebook, desktops, the VLE and/or web pages. Th Don’t forget to send reminders to students whilst the survey is live. The higher your response rate, the more reliable your findings.



4. Viewing, downloading, analysing and benchmarking your data

1. You can access and browse your results at any time by logging on to BOS, locating the survey, and clicking on the 'Analyse' tab.

2. Whilst the Tracker is live you can regularly check in and see your data plotted live.

3. By default, BOS presents you with a summary of your data in the form of a bar chart comparing percentage responses within each question. BOS also shows you the raw score in bold and the percentage score (in brackets) for each response. With some Tracker questions the raw score will be more useful for analysis.

4. Once you have your complete data set, you can log in at any time and download it. However, you will not have access to complete data for benchmarking until the formal close date in March 2017.

You download data using the ‘Export’ button on the right at the top of your ‘Analyse’ screen. 'Download PDF' replicates the graphical summary you see in the 'Analyse' area of BOS. 'Export response data' lets you



download a file of the raw data to a secure location so that you can open and view it in a spreadsheet or data analysis program such as Excel, NVivo, or SPSS.

5. You can also partition your data. That is, you can view the data for different groups of learners as separate data sets for analysis and comparison. If you are downloading data into another analysis tool then you can simply download the full set and partition your data in that system. To partition your data in BOS, use 'Filter based on responses' which is at the bottom of the blue panel on the left of your 'Analyse' screen. Click on '+ Add filter’.

6. You are now in the filter editor. You will see a drop down menu showing the Tracker questions. Use the down arrow to scroll to the question you want to use for partitioning your responses. This may well be one of the questions on page 8 (‘About you’), which you customised exactly so that you could group your learners at this point in the process.



7. Select the first of the options available under the question you have chosen. A tick box will appear. This indicates that you have set up a filter to extract all the sets of data where the respondent has given this particular answer. Click on 'Save and enable' and return to the 'Analyse' screen.

8. In the blue panel you will see that you have set up a filter and applied it. You can now view the data 'Summary' for just this group of learners, or you can choose to 'Download' the data for just this group of learners. Repeat for all the possible responses (i.e. all the different groups of learners) in this question. You can also partition your data using further questions in the same way. Note that when you log into BOS you will be asked whether you want to view the filtered data that you last viewed, or start afresh.

9. The option to benchmark your responses to a particular question is also available within the BOS system. This means that you can compare your data with the average from other institutions using the Tracker.

From the 'Analyse' screen, click on 'Benchmark survey' in the 'Advanced' menu.



10. Choose 'Jisc benchmarking' as the comparison group from the drop-‐down menu and press ‘Select’.

11. Click on 'Benchmark' and wait for BOS to calculate your comparative data. This can take a while, as there may be thousands of separate responses to calculate.

12. For each question you will now see all the responses from your survey (column one) set alongside all the responses from learners in your sector but not at your institution (column two), and finally both sets of data combined to give you the sector as a whole, including your survey (column three). This allows you to visually compare your data with the rest of your sector. (Note that these examples are based on a dummy survey with very little data.) Comparisons are helpful, but beware of making assumptions without checking whether any difference between two figures is statistically significant. To statistically compare your data with the benchmarking group, check out the information in the Guide to Analysing and Understanding your responses.



5. If you need help

1. The BOS help pages are very useful if you want a better understanding of these and any other features of the software. If this Tracker is rolled out as a full Service, users will need to rely on these guidance notes and the 'Help & Support' available in BOS. We therefore politely ask that you use these as your first port of call to ensure the pilot is as authentic as possible.

howtousethe$jisc studentdigital experience)tracker ·...

Documents