eair-2013-organisation-and-practical-use-of-nse-paper
TRANSCRIPT
1 Nationale Studenten Enquête
Nationale Studenten Enquête
The organisation and practical use of the National Student Survey (NSE) in the Netherlands
Paper presented in track 4 at the
EAIR 35th Annual Forum in Rotterdam, the Netherlands
28-31 August 2013
Name of Author(s)
A.V. (Lex) Sijtsma Drs. W.H.M. (Wil) Kwinten
MSc T.C.C. (Theo) Nelissen
Contact Details
A.V. Sijtsma Foundation Studiekeuze123 P.O. Box 2290 Utrecht, 3500 GG The Netherlands E-mail: [email protected] Key words National survey, study choice, quality care, application of NSE
2 Nationale Studenten Enquête
Abstract Nationale Studenten Enquete
The National Student Survey (NSE) is a nationwide survey among almost all students in higher
education in the Netherlands. Students offer their opinion concerning study programme and
institution. The NSE is a unique survey, not in the least by its collaborative approach. The results
are used by a growing number of organisations.
This paper elaborates on how the NSE is implemented, gives practical examples of its use and how
it’s used within an institution of further education.
3 Nationale Studenten Enquête
Nationale Studenten Enquête
The organisation and practical use of the National Student Survey (NSE) in the Netherlands
Table of Contents 1 Introduction ................................................................................................................................................................ 5
2 What is the NSE? ......................................................................................................................................................... 6
2.1 Authors and organisations ................................................................................................................................. 6
3 Application of the NSE ................................................................................................................................................ 8
3.1 Choice of studies ............................................................................................................................................... 8
3.2 Quality ............................................................................................................................................................... 8
4 The questionnaire ..................................................................................................................................................... 10
4.1 Panel of Experts ............................................................................................................................................... 10
5 The process ............................................................................................................................................................... 11
5.1 Collecting information ..................................................................................................................................... 11
5.2 Field research .................................................................................................................................................. 12
6 Communications ....................................................................................................................................................... 13
6.1 Channels .......................................................................................................................................................... 13
6.2 Promotional materials ..................................................................................................................................... 16
6.3 Meetings .......................................................................................................................................................... 17
7 Results ....................................................................................................................................................................... 18
7.1 Publicly available benchmark file .................................................................................................................... 18
7.2 Institution file .................................................................................................................................................. 19
7.3 Fact sheet ........................................................................................................................................................ 19
7.4 Key file ............................................................................................................................................................. 19
7.5 NSE Online ....................................................................................................................................................... 19
7.6 Research justification ...................................................................................................................................... 20
8 Case study: Avans University of Applied Sciences .................................................................................................... 21
8.1 Vision ............................................................................................................................................................... 21
8.2 Quality management and management model .............................................................................................. 21
8.3 NSE input into Avans processes ....................................................................................................................... 22
8.4 Valuable information ....................................................................................................................................... 23
8.5 Sample project ................................................................................................................................................. 26
9 Conclusions ............................................................................................................................................................... 27
10 Appendices ........................................................................................................................................................... 28
10.1 Invitation text .................................................................................................................................................. 28
10.2 Introducing Intomart GfK ................................................................................................................................ 29
10.3 Response development ................................................................................................................................... 29
10.4 Participating institutions NSE 2013 ................................................................................................................. 30
4 Nationale Studenten Enquête
Figures Figure 1 – Scores awarded by students .............................................................................................................................. 8 Figure 2 – Web site for institutions ................................................................................................................................... 13 Figure 3 – NSE Bijblijver .................................................................................................................................................... 14 Figure 4 – Registration of e‐mail questions ...................................................................................................................... 14 Figure 5 – Response report ............................................................................................................................................... 15 Figure 6 – Response report for full-time version of programmes by location .................................................................. 15 Figure 7 – Response by institution .................................................................................................................................... 16 Figure 8 – NSE Banner ....................................................................................................................................................... 16 Figure 9 – Poster ............................................................................................................................................................... 16 Figure 10 – 2013 benchmark file ....................................................................................................................................... 18 Figure 11 – Fact sheet for Avans, 2013 (detail view) ........................................................................................................ 19 Figure 12 – NSE Online ...................................................................................................................................................... 20 Figure 13 – PDCA cycle ...................................................................................................................................................... 22 Figure 14 – Scheduling Infographic ................................................................................................................................... 25 Figure 15 – Daily development of response ..................................................................................................................... 30
Tables Table 1 – 2012 colour board (detail view) ........................................................................................................................ 24 Table 2 – 2013 NSE participants ........................................................................................................................................ 32
All hyperlinks were checked 17 July 2013.
5 Nationale Studenten Enquête
1 Introduction
This paper introduces the largest Dutch survey to measure student satisfaction with higher
education: the Dutch National Student Survey (Nationale Studenten Enquête). Not only the
organisation of the survey is discussed, but also its participants, quality management, the end results
produced and the aspects that amplify its practical use and value. In the context of one institute of
higher education, the specific ways in which the NSE is applied and leveraged are demonstrated.
This paper offers stakeholders the tools to better leverage student satisfaction data and help improve
the quality of the educational system.
6 Nationale Studenten Enquête
2 What is the NSE?
The National Student Survey (NSE: Nationale Studenten Enquête) is a nationwide survey
researching students’ satisfaction with the course programme they pursue at their university or
university of applied sciences. Students are asked to assess both their institution and their course
programme.
Up until 2010, educational institutions each measured the satisfaction of their students via internal
surveys, in addition to which a nationwide survey was conducted. Since 2010 these have been
united into a single survey: the NSE.
Almost all publicly funded institutions participate, as well as a large number of privately funded
institutions. Over 72 institutions participated in the 2013 survey and more than 264,000 students
(a response of 39.3%) completed the questionnaire – the highest figures to date. This makes the
NSE a prime example of effective collaboration between Dutch institutes of higher education.
A unique and valuable aspect of the NSE is that its results can be used for benchmarking:
institutions and course programme committees can compare their results to those of their peers.
Since 2010, the questionnaire has remained largely the same, allowing for the analysis of trends. In
addition, parties outside educational institutions, such as commercial publishing houses and the
Dutch government, make use of NSE results for ranking institutions and course programmes and
setting performance goals.
2.1 Authors and organisations This paper was written by:
A.V. (Lex) Sijtsma, Studiekeuze123, [email protected]. Lex Sijtsma is the NSE
programme manager and supervises its annual execution.
Drs. W.H.M. (Wil) Kwinten, Avans Hogeschool/University of Applied Sciences,
[email protected]. Wil Kwinten is senior advisor for research, quality management
and education development at Avans.
T.C.C. (Theo) Nelissen MSc, Avans Hogeschool/University of Applied Sciences,
[email protected]. Theo Nelissen, researcher at Avans, performs analyses on NSE data
specifically relevant to Avans’ purposes.
2.1.1 Studiekeuze123
Stichting (foundation) Studiekeuze123 provides independent and reliable information on higher
education study programmes and institutions without being influenced by commercial interests.
The Board consists of delegates from the representative bodies of the higher education institutions:
Vereniging Hogescholen (Higher Education Council), VSNU (University Association), and NRTO
(Dutch Council of Training and Education), as well as the student organisations LSVb and ISO.
Studiekeuze123 is financed by the Ministry of Education, Culture and Science (OCW) and
implemented by SURF, the higher education and research partnership for ICT in the Netherlands.
Studiekeuze123 carries out the annual NSE on behalf of the Ministry of Education, Culture and
Science and in collaboration with institutes for higher education.
2.1.2 Avans Hogeschool
Avans Hogeschool (University of Applied Sciences) serves 26,000 college students on the level of
higher vocational education and employs approximately 2,200 employees. Avans University
provides higher education in eight sectors and at seven locations in Breda, 's‐Hertogenbosch and
Tilburg. Education is organised in 19 schools and is offered in about 55 different substantive course
7 Nationale Studenten Enquête
programmes (to the levels of associate degree, bachelor and master) in the form of full‐time, part‐time and work‐study programmes. For several years in a row, it is chosen by national rankings, as
the best large-scale Dutch university of applied sciences, with the most study programs in top
positions and above-average student satisfaction.
8 Nationale Studenten Enquête
3 Application of the NSE
The NSE serves two important goals in the field of education:
1. Informing students selecting a course programme by collecting the opinions of enrolled
students, benefiting prospective students’ ability to imagine the nature and quality of the
course programmes offered;
2. Improving the quality of course programmes offered by institutes of higher education.
Below, each goal will be explained further.
3.1 Choice of studies
30 to 40% of Dutch students will in the first year of their university career switch to another course
programme or drop out altogether: a worryingly large number, which is recognised as a societal
issue. The NSE offers an instrument to strengthen the basis on which students choose a course
programme and strives to aid students in making these choices.
On the Web site www.studiekeuze123.nl objective and independent data is collected on almost all
of the course programmes available in Dutch higher education. For each course programme, the
average score awarded by students currently enrolled in the course programme is listed. Figure 1
shows an example of the Business & IT Management programme at Avans University of Applied
Sciences. Scores are plotted along a Likert scale from 1 to 5, illustrated below by the “Acquired
general skills” scores of 3.9 and 3.6 (for full‐ time and part-time respectively), where 5 stands for
“very satisfied”.
Figure 1 – Scores awarded by students1
3.2 Quality Another goal of the NSE is to help advance quality management within institutes for higher
education. Many institutions use NSE data alongside their own research to spot issues brought out
by conspicuously low or high scores on certain themes. Their own internal research then enables
them to investigate any issues in greater detail.
One university of applied sciences, for example,2 scored low on time taken to score tests, as well as
9 Nationale Studenten Enquête
other items of contact and information (such as accessibility of the lecturers) and the feasibility of
deadlines. Another university of applied sciences scored significantly higher. What could have
caused this discrepancy?
Further research and discussion brought to light a connection between the two scores, as there
appears to be a link between students’ academic self‐ discipline and teachers’ professional self-
discipline. The lesson herein was that these low scores should not only be attributed to students
succumbing to procrastination, but also to the behaviour of teachers.
The higher scoring university of applied sciences turned out to have a clear-cut “a deal is a deal”
policy: If it was agreed that tests were scored and results posted within 10 working days, teachers
who failed to perform were held accountable by their superiors as well as their peers. The lower-
scoring institution turned out to lack such a policy. Agreements were seldom met and often
adjusted; test scoring times were not standardised. As a results, students often found themselves in
the dark.
This serves as an example of general trends made visible through NSE data and issues that
institutions can tackle with additional research as a basis for effective measures.
10 Nationale Studenten Enquête
4 The questionnaire
The questionnaire was put together in collaboration with institutes of higher education. In its current
form, the questionnaire is largely the same as it was in 2010; however, it was modified slightly in
2012 to comply with requests from the institutions.
The questionnaire comprises about 100 questions, divided into the following categories:
Background information
General assessment
The content and organisation of teaching
Acquired general skills
Acquired scientific skills/skills for applied research
Lecturers in your study programme
Academic guidance/counselling
Testing and assessment
Information provided by your study programme
Programme schedules
Study load
Contact hours
Group size
Internships
Study facilities
Quality care
Other facilities and study environment
Living arrangements
Studying with a handicap
International students
The Ultimate Question
Wishes/ideas for improvement
The questionnaire is available at http://www.studiekeuzeinformatie.nl/nse/over-nse/vragenlijst
in English, Dutch and German.
Not all questions are relevant to every student, which is why participants are routed to see only the
questions that pertain to their situation. Dutch students, for instance, see no questions from the
“International Students” section.
4.1 Panel of Experts
The survey’s quality – in terms of representativeness, validity, reliability and applicability – is
assessed by a panel of experts for Studiekeuze123, ensuring scientifically sound research. In
addition, Studiekeuze123 employs a Data Quality Officer.
11 Nationale Studenten Enquête
5 The process
The process of conducting the NSE survey starts September and ends in May. It comprises, roughly,
three stages:
Aggregation of data (September – December)
Field research (January – March)
Delivery of results (March – mid‐May)
Studiekeuze123 supervises the NSE; that is to say, Studiekeuze123’s role is to oversee and direct
the execution of the survey. The operational realisation of the survey is in the hands of the
specialised research institute Intomart GfK, based in Hilversum, the Netherlands.
5.1 Collecting information
A first step is the collection of information sent in by participating institutions.
The NSE is kicked off with an introductory meeting, so as to explain the procedure and the time
schedule for the coming year.
Studiekeuze123 sends all publicly and privately funded institutions in the Netherlands a written
invitation to participate. This includes a contract to be signed and returned to Studiekeuze123.
Following this, Intomart GfK contacts participating institutions asking them to send in the following
data:
1. Student data. Institutions are asked to send in specific information on their students,
including student ID number, e‐mail address, age, course programme, course year etc. This
information is used to invite students to participate and is given back to the institutions after
the survey. Institutions are to submit information about all of their students: selecting certain
groups or individuals is not allowed. Certain categories are left out, however, such as
students enrolled in a pre‐master programme.
2. Extra questions (optional). Participating institutions may add a limited number of their
own questions, pertaining for instance to specific local circumstances. This way they can
forego organising their own research. These questions can also aid in encouraging students
to complete the questionnaire. The extra questions are added to the questionnaire and posed
to students after the general questions. Answers are made available to the institution only.
Intomart GfK can assist institutions in formulating their own questions and performs a
number of standardised checks to ensure, for instance, that students’ privacy is safeguarded.
3. Extra labels (optional). To aid in the processing of the resulting data, institutions may add
custom labels to the questionnaire; for example, in the case of course programmes that are
the result of a collaborative effort by two institutions. In this case, the name of the
collaboration can be added as a custom label. Labels are visible to the institution only.
4. Invitation text (optional). (See also Appendix 10.1.) Institutions or course programme
committees may add their own afterword, as well as an additional line of text to be added to
the standardised invitation sent on behalf of Studiekeuze123. These texts may even be
student‐specific.
Intomart GfK makes a secure uploading environment available to institutions. During uploading,
some preliminary, automated checks are done: course programme names and institutions, for
instance, are checked against public registries (BRIN and CROHO respectively).
In addition, Intomart GfK carries out their own checks on these data, such as checks for missing
course programmes. The data are subsequently added into the database on which the field research
will be performed.
12 Nationale Studenten Enquête
5.2 Field research
The field research phase takes 8 weeks from 2014 on, before which it took a total of 10 weeks, and
starts out in mid-January. The NSE survey is offered online at www.nse.nl. For the 2014 NSE,
research is underway into how the questionnaire can be comfortably filled in on a smartphone.
Expectations are that this will increase response figures owing to the greater ease of use.
All students are invited via e-mail. For every student, institutions may enter up to two e-mail
addresses; commonly, a university address and a personal address. All e-mail addresses that are sent
in are used, which has led to the impression that the use of personal e-mail addresses yields higher
response figures and is thus more effective than using a student’s institutional address.
Students may complete the questionnaire in instalments: the system will remember what parts are
yet to be completed. In like manner, it is possible to switch from a PC to a laptop or tablet mid-use.
Until students have completed the entire questionnaire, they receive a reminder every fortnight. In
the last week of this phase, a so-called “last shot” reminder is sent.
Institutions may track development of the response via response rate summaries. For more
information on this, refer to section 6.1.7.
13 Nationale Studenten Enquête
6 Communications
Adequate communications around the process of the NSE survey are of major importance.
Studiekeuze123 keeps in touch with their contacts at institutions and with participating students via
various channels of communication. Studiekeuze123 also supplies participants with promotional
materials and organises a number of events around the NSE.
6.1 Channels
6.1.1 Web site www.nse.nl
The survey is offered online at www.nse.nl.
6.1.2 Web site www.studiekeuzeinformatie.nl/NSE
A Web site is maintained for participating institutions, offering information on the survey as it is
underway: for instance, background information on quality assurance and research justification.
Studiekeuze123 is firmly committed to keeping the entire NSE process open and transparent.
Figure 2 – Web site for institutions
6.1.3 Social media
Studiekeuze123 regularly publishes tweets (hashtag: #nse2013) and enables students to send a tweet
or publish a Facebook post to say they have completed the NSE survey.
6.1.4 NSE Bijblijver
NSE Bijblijver is a digital opt-in newsletter that is sent out as needed over the course of the research
phase. In 2013, 8 of these newsletters were sent out. An archive of past newsletters is available via
http://www.studiekeuzeinformatie.nl/nieuwsbrief/archief-nieuwsbrief-en-bijblijvers.
14 Nationale Studenten Enquête
Figure 3 – NSE Bijblijver
6.1.5 E‐mail: [email protected] and [email protected]
Participating institutions as well as students can get in touch with the NSE organisers via e-mail.
The address [email protected] is available for institutions and students can use the address
[email protected]. All incoming e-mail is automatically given an identification number and entered
into a call handling system. This system is developed for the handling of questions and incidents
and can be manned by multiple employees if necessary. As a call is handled, a record of all actions
is kept.
This allows for quick and adequate reactions to questions and ensures no questions remain
unanswered, even in busy periods.
About 2,000 e-mails were handled in 2013.
Figure 4 – Registration of e‐mail questions
15 Nationale Studenten Enquête
6.1.6 Telephone number: +31 35 625 76 68
The Intomart GfK team can be contacted via +31 35 625 76 68.
6.1.7 Response rate
Participating institutions may track development of the response in real time. As soon as a student
finishes the survey, the questionnaire is added to the response rates. Response rate summaries are
available to institutions both in overview form and in several more detailed views. Please see Figure
5 for a look at the response reports available.
Figure 5 – Response report
The example below shows one of the views available for Hogeschool Avans, which offers full-time,
part-time and work-study programmes. In this example, only the figures for the full-time versions of
several course programmes are visible, split by location.
Figure 6 – Response report for full-time version of programmes by location
There are also overviews for institutions who wish to compare their own response rates to those of
others. See below for an example of an institution-level overview.
16 Nationale Studenten Enquête
Figure 7 – Response by institution
Finally, Studiekeuze123 informs all stakeholders on the development of response rates on a weekly
basis.
6.2 Promotional materials Studiekeuze123 supplies participants with promotional materials in the form of banners, flyers and a
poster. The building blocks for these materials are also made available, so that institutions can add
their name or other information. The materials are available in English, Dutch and German.
Please refer to http://www.studiekeuzeinformatie.nl/nse/traject-van-nse/promotiemateriaal for these
materials and more information.
Figure 8 – NSE Banner
Figure 9 – Poster
17 Nationale Studenten Enquête
6.3 Meetings
Studiekeuze123 funds and organises several events for and in collaboration with participating
institutions: to inform them on NSE, but also for the benefit of networking. The following meetings
are organised:
1. Introductory and closing meetings. All institutions are invited to these two events, which
are held at the start and the end of the survey period.
2. Knowledge Sharing Workgroup. Participating institutions can benefit greatly from sharing
information, such as best practices, tips and tricks, and have shown great interest in learning
from one another. To this end, a knowledge sharing workgroup was established among
institutions. Each year, three meetings are organised to present and discuss a variety of
topics. In 2013, the workgroup discussed how institutions can spread the results across their
organisations; also, the commercially oriented party CHOI (www.choi.nl) explained how
they make use of NSE data in producing their Keuzegids (Course Programme Guide). In
addition, the workgroup sets up part of the agenda for the introductory and closing meetings.
Three meetings were held in 2013.
3. Focus group. Institutions may share their opinions on future plans for the NSE through this
focus group, which Studiekeuze123 assembles from a selection of participants. These
institutions are asked about their experiences with recent developments within the NSE and
a full evaluation of their NSE participation is performed every year. The focus group meets
as needed, convening twice in 2013.
18 Nationale Studenten Enquête
7 Results
When the field research phase is complete, the results are ready to be processed into the final
product. The raw data are cleaned and pre-edited: incomplete questionnaires are removed, checks
are performed and weighting factors are calculated and applied.
7.1 Publicly available benchmark file
This SPSS file holds all answers given by all students; however, the data is stripped of background
information such as age, gender, course year and so on. These results are synchronised with results
of past years, to allow for trend analyses. Currently the file contains the results from 2010 to 2013,
amounting to 900,000 records. Institutions can use it to compare their results to those of their peers.
Figure 10 – 2013 benchmark file
7.1.1 Licensing to external parties
The public benchmark file is also licensed to parties outside participating institutions, who may use
it to develop their own products. Four examples:
1. Elsevier magazine (www.elsevier.nl) annually publishes a research special entitles
“University course programmes: Best of the Best”, a comparison of higher education
programmes based in part on NSE data and professors’ assessments.
2. CHOI (www.choi.nl) produces the annual Keuzegids (Course Programme Guide). Aimed at
pupils as well as students, this is a critical comparison of all course programmes available,
sorted by field.
3. Studie in Cijfers developed a “quick facts” leaflet format for easy reference to objective
figures and information on a course programme. Examples of these leaflets and more
information can be found at www.studiekeuze123.nl.
4. Studiekeuze123 (www.studiekeuze123.nl) not only organises the NSE itself, but also
leverages the resulting data. Studiekeuze123 supports prospective students in choosing a
course programme by providing them with independent and reliable information on higher
education study programmes and institutions without being influenced by commercial
interests.
The public benchmark file is also licensed to organisations working in the field of education, who
use NSE data mainly for policy development. The Dutch Ministry of Education, Culture and
Science uses NSE data to set performance goals for institutes of higher education.
19 Nationale Studenten Enquête
7.2 Institution file
This SPSS file comprises the results for one institution only. It is an extract from the public
benchmark file, enriched with background data and the institution’s own additional questions and
labels.
7.3 Fact sheet A fact sheet lists an institution’s NSE results on a single page. The year’s scores are separated by
theme and question and listed alongside the previous year’s scores to highlight any fluctuations.
Studiekeuze123 also sends these fact sheets to participating institutions’ Boards of Directors.
Figure 11 – Fact sheet for Avans, 2013 (detail view)
7.4 Key file The practical use of NSE results is multiplied when the data can be linked to the institution’s
internal administration system. The key file makes this possible. Results can also be linked up with
the institution’s own research data and NSE results can even be analysed according to source group:
for instance, underperforming students who went to the same secondary school, or students coming
from another school and lacking mathematical skills. In order to safeguard privacy, Studiekeuze123
subjects the use of the key file to stricter security regulations.
7.5 NSE Online
NSE results are made available in SPSS format, the use of which can take some training. This is
why Studiekeuze123 also offers a simpler reporting tool: NSE Online, which is available at
www.nse.nl/online.
20 Nationale Studenten Enquête
Figure 12 – NSE Online
7.6 Research justification The research institute responsible for the NSE’s operational realisation publishes a yearly research
justification detailing how the survey is carried out. This in turn is assessed by the Panel of Experts
at Studiekeuze123.
The research justification is available at http://www.studiekeuzeinformatie.nl/nse/resultaten-
nse/evaluatie-en-onderzoeksverantwoording. The 2013 version is set to be published in October
2013.
21 Nationale Studenten Enquête
8 Case study: Avans University of Applied Sciences
This chapter offers a look at the value of student satisfaction figures to the improvement of
education quality. As a case study, the approach of Avans University of Applied Sciences is
presented. This section was in part inspired by NVAO’s description of Avans’ institutional quality
system.
8.1 Vision
Avans’ vision centres around collaboration, viewing education as something to be built together.
This helps the institution ensure their vision is recognised and supported. Avans organises course
programmes focused on the small scale and seeks to approach students in a personal manner. In
spite of its size (26,000 students are enrolled), the institution has succeeded in making students feel
they are individually recognised. In addition to the focus on small-scale design of its course
programmes, Avans values personal interaction and collective decision making. The institution
strives to incorporate these values in all departments of the University: not only in its formal
structures and protocols, but also in its culture and atmosphere, harmonising with the individual
departments’ discourse on their views on teaching at Avans.
8.2 Quality management and management model The quality management system in place at Avans, illustrated below, was developed based
on Deming’s PDCA cycle (see also http://en.wikipedia.org/wiki/PDCA). This is a management
method consisting of four steps: PLAN, DO, CHECK and ACT. Avans utilises this method on three
levels of its organisation:
1. University level (divided into academies, teaching locations, etc.)
2. Academy level (divided into course programmes)
3. Course programme level (divided into full‐time/part‐time/work‐study and teaching
locations)
The individual cycles are interconnected and information is fed into them from formal as well as
informal circuits. Types of information input to these cycles include recurring evaluations (e.g.
periodic education evaluations; the NSE; the HBO Monitor graduate survey; employee satisfaction
surveys; the NVAO Institutional Audit), as well as recruitment and transfer figures. This is the path
student satisfaction data follow into the organisational levels of Avans. This process is explained in
more detail below.
22 Nationale Studenten Enquête
Figure 13 – PDCA cycle
Quality care and quality management can be approached from a variety of angles. Avans is invested
in going beyond just the management of quality and creating a culture of quality. In his article,3
Dries Berings paraphrases Peters and Waterman: “Mastering the soft side of managing (Shared
Values, Skills, Style, Staff) is just as important as mastering the hard side (Strategy, Structure,
Systems).” This can be taken to mean that the formal systems need to lead into the informal
systems: that is, into the culture of the organisation. Organisational culture, as Berings quotes
Houtveen (1996), can be defined as “the often unaware and difficult to verbalise set of common
values regarding the work in its broadest sense, as shared by the members of the organisation”
(translated on behalf of authors). The focus on a culture of quality is an integral part of the Avans
management model, which was based on Robert Simons’ four Levers of Control.4 This model
revolves around being in control without hindering creativity or employees’ own sense of initiative.
The management model addresses both the formal and the informal aspects of management.
8.3 NSE input into Avans processes
8.3.1 Examples of NSE input and monitoring into formal Avans processes
– Avans is implementing a data warehouse: an integrated information system to unite planning and
monitoring data in a “dashboard” overview. NSE results will be available for viewing, as well as
other key performance indicators of critical business processes. One such performance indicator is
the fact that 75% of students award their general satisfaction a score of 4 or 5 – one of the
performance goals set for Avans by the Ministry of Education.
– At the start of each academic year, academies present their business plan containing the goals they
aim to realise in the coming year. The basis of these goals is formed in parts by NSE results. These
plans are also discussed with the Board of Governors, as are focal points arising from the NSE
results. In the course of the year, academies share their progress via quarterly management reports.
8.3.2 Examples of NSE input and monitoring into informal Avans processes
– The scores Avans is awarded, on an institutional level as well as per academy or course
programme, are discussed throughout the institution. Positive results are celebrated and broadcast
23 Nationale Studenten Enquête
via posters, prints on coffee mugs, cakes, articles in the school paper, and so forth. In brief, the NSE
is very much a topic of discussion. Attention is also paid to communicating positive results to
prospective students who have yet to choose a course programme.
– Internal rankings result in healthy competition between course programmes and academies. In
this, Avans attaches great importance to full disclosure of NSE results. This policy of complete
transparency is central to Avans culture and the institution considers the “naming and shaming”
aspect vital to the high scores Avans has attained in the past four years.
8.4 Valuable information
Formal processes and systems only play an indirect role in the realisation of quality. They are a
means to an end: that of actual changes in behaviour. In this context, the way information is made
available is important: Data should be presented in an accessible format, from which meaning can
be deduced easily from the perspective of those involved.
It should be noted that many improvements should ultimately be realised on the lowest level. After
all, students are working on their course programme, not on the academy or the university.
Information on higher levels in the hierarchy are more likely to lead to provisional or management-
related measures. Conversely, real improvement is made in the primary process – among lecturers,
students, in learning activities and so forth. The consequence of this view is that Avans results need
to be primarily communicated to lecturers, committees (such as programme committees and the
examination board) and directly to the students themselves. The NSE results allow for this type of
communication; for instance, the lecturers can get feedback on how their teaching efforts on a
specific course programme are assessed.
Especially where the softer aspects of management are concerned, NSE results can be effectively
leveraged owing to the support the NSE data generates. Stakeholders within Avans trust the
information given, meaning they are more inclined to put it to use.
8.4.1 Ways of sharing information
– When NSE results become available, each academy, as well as the Board of Governors, receive an
overview of the results that are most important to their department. These “colour boards” are
colour-coded for improved readability: progress and decline are immediately visible. These
overviews are printed on A3/Ledger-sized paper, laminated and given to the management and the
Board of Governors. Even though the results are available in digital form, this highly readable
presentation of results proves considerably more effective. The colour boards are pinned to the
walls of nearly every management office.
24 Nationale Studenten Enquête
Question
Total for
Avans Academy 1 Academy 2 Academy 3
Your study programme in
general 3.89 3.90 3.67 ……
The content of the study
programme 3.85 3.93 3.68 ……
The general skills acquired in
your study programme 3.77 3.78 3.57 ……
Applied research within your
study programme 3.67 3.64 3.38 ……
The preparation for a
professional career 3.56 3.54 3.34 …..
The lecturers in your study
programme 3.71 3.63 3.28 …..
The information provided by
your study programme 3.48 3.53 3.18 …..
The study facilities offered by
your study programme 3.62 3.80 3.20 …..
Testing and assessment 3.52 3.56 3.26 …..
Programme schedules 3.45 3.22 3.38 …..
Study load 3.49 3.50 3.33 …..
Academic
guidance/counselling 3.53 3.57 3.24 …..
Other facilities and study
environment 3.62 3.74 3.34 …..
The general atmosphere in your
study programme 4.16 4.10 4.06 …..
The degree to which you are
involved in improving your
study programme
3.64 3.62 3.40 …..
Would you recommend your
programme to friends, family
or colleagues?
4.17 3.99 3.97 …..
Table 1 – 2012 colour board (detail view)
– Directors, academy employees and students all receive an academy report. This report presents
the scores specific course programmes were awarded.
– NSE results from 2010 onwards are available via a Qlikview application. This convenient
application offers the possibility to freely explore the data and to generate ready-made reports
answering frequently asked questions. As part of the institution-wide Management Information
Programme, access to NSE results will be offered via Business Objects 4.0 from 2014 on.
– For simple questions, a central help desk is available to generate basic reports.
– Following requests from the Board of Governors as well as academy directors, a comprehensive
analysis of the NSE results can be performed as needed.
– NSE Specials are published about salient issues, in which relevant information from the NSE is
analysed and provisional conclusions are drawn. These Specials can be enriched with infographics
and supported with other sources of information. For a sample Special on scheduling issues, see
25 Nationale Studenten Enquête
below.
Figure 14 – Scheduling Infographic
26 Nationale Studenten Enquête
8.5 Sample project
The scores awarded on the subject of programme scheduling resulted in a degree of internal debate
on their accurate interpretation. NSE results showed a score of 3.45 on Scheduling, one of the
lowest scores across the themes. Although low, this score is adequate: on the scale used, a score of 3
counts as “neutral” and any higher scores signify satisfaction.
Further research within Avans revealed room for improvement in the internal scheduling process.
Students turned out to be bothered mainly by the late publication of programme schedules and the
frequent changes to the schedule.
The Board of Governors referred to this information in a request to both the support services and the
academies to improve the situation. To accomplish this, the following goals were set and linked to
NSE results:
The quality of programme schedules is rated as “good” by students, meaning a minimum
average NSE score of 3.65 (2011: 3.45) and no academy is rated below 3.3.
The programme schedules are published in time, meaning a minimum average NSE score of
3.25 (2011: 3.05) and no academy is rated below 3.0.
Notifications of changes to the current schedule are clearly communicated, meaning a
minimum average NSE score of 3.22 (2011: 3.02) and no academy is rated below 3.0.
A project has been launched to achieve this. The goals of this project are as follows:
The timely publication of the programme schedules
Better schedule quality
A more flexible approach to scheduling
A less error-prone approach to scheduling
Fewer changes to the schedule
Better communication of schedule changes
27 Nationale Studenten Enquête
9 Conclusions
This paper introduced the foundations of the Dutch National Student Survey and ways in which the
results may be leveraged within an institution.
Examples were given of the application of NSE data and of survey questions, realisation of the
research, communications and licensing of the results.
A case study was presented in the form of Hogeschool Avans’ application of NSE results: both in
general and illustrated by the programme scheduling project.
It is hoped that this paper presents the reader with ample information to assess the benefits of NSE
for institutions of higher education.
The authors and the NSE staff will be glad to answer any questions via [email protected].
28 Nationale Studenten Enquête
10 Appendices
10.1 Invitation text
The invitation and all the reminder e-mails are available in Dutch, English and German. This is the
template for the English text. Words/sentences between [] and <> are metaterms that are replaced
with the actual content.
NSE 2013 (29535)
Uitnodigingsmail
Versie 3
Subject: Give your opinion about your studies at [naam instelling].
Sender: National Student Survey 2013
English
Dear student,
Who is able to rate your study programme and [hogeschool/universiteit] better than you?
This is why we’re inviting you to participate in the National Student Survey 2013, the leading
annual survey into the quality of programmes in higher education.
The National Student Survey: why?
Your feedback will help prospective students to make the right choice when it comes to a
study programme. The results of the survey will be published on the independent
comparison site Studiekeuze123.nl.
Your feedback will help [hogeschool/universiteit] to improve the quality of their
education.
Take the survey
[Your e-mail address has been provided by your institution, which will benefit from your
participation.] [Of in plaats van de vorige zin de instellingstekst.]
The more students who participate, the better the results will be.
It’s easy to take the survey: just click on the link below and you can start straight away.
[persoonlijke link naar de vragenlijst]
If this link does not work, then copy the link into your browser’s address bar.
Completing the survey will take no more than 10 to 20 minutes. You can fill in part of the survey
now, and return to complete it later if you like. Just click on the link again, and you will
automatically pick up where you left off.
Win a great prize!
We’ll be giving away fantastic prizes to some lucky participants, including:
5 iPhone 5s (value: €800).
5 iPad minis (value €400).
240 Bol.com vouchers (value €25).
29 Nationale Studenten Enquête
Studiekeuze123 Foundation The National Student Survey is the responsibility of the Studiekeuze123 Foundation. The
foundation is an initiative of the student organizations (ISO, LSVb) and the educational umbrella
organizations (HBO council, VSNU and NRTO). The research firm Intomart GfK is conducting the
survey on behalf of Studiekeuze123. The results of the survey will be available from mid-June at
Studiekeuze123.nl.
If you are having problems logging in or if you have any questions about the survey, please send an
email to [email protected] or see www.nse.nl/faq for more information.
Thank you very much for your participation!
Kind regards,
Tom Dousma <naam>
Studiekeuze123 <onderwijsinstelling>
<bij oplevering koppelbestand>
Please note:
<naam instelling> intends to use the results of the survey for further statistical analysis. These
analyses will be used to enhance the quality of education and improve the institution as a whole.
The institution needs the background information on the students who participated in the survey in
order to complete these analyses. The relevant data will only be used by the personnel responsible
for Institutional Research at <naam instelling>, and will never be provided to third parties. The data
will be treated with the utmost confidentiality. The survey results will be anonymized and cannot be
traced back to individual persons. The reports will also be completely anonymous.
Click <hier> to unsubscribe to the National Student Survey 2013.
10.2 Introducing Intomart GfK Intomart GfK describes itself as follows:
GfK is one of the world’s largest research companies with a primary focus on realizing ‘growth
from knowledge’ for its clients. GfK Intomart, as part of the GfK concern, focusses on three main
areas: Media, Marketing and Public Services.
Within Public Services, GfK Intomart works for a broad range of non-profit clients, also addressing
a broad range of research subjects. Data collection is normally (still) an important aspect of our
services, but the next step, fact-based consultancy, nowadays becomes more and more important
since our clients express the need to use our expertise in interpreting and implementing the outcome
of the research we conduct.
Website: http://www.intomartgfk.nl/
10.3 Response development
The figure below shows the daily development of the response to the 2013 NSE. These data
represent both usable and unusable responses and are recorded before any cleaning or checks are
performed. If only usable responses are considered, totals are about 1% lower.
30 Nationale Studenten Enquête
Figure 15 – Daily development of response
The peak in student responses on 21 January coincides with the sending of invitations to the
students’ personal e-mail addresses.
10.4 Participating institutions NSE 2013
No. Institution
1 ACTA
2 Amsterdam University College
3 Amsterdamse Hogeschool voor de Kunsten
4 ArtEZ hogeschool voor de kunsten
5 Avans Hogeschool
6 CAH Vilentum Dronten / Almere
7 Christelijke Hogeschool Ede
8 Christelijke Hogeschool Windesheim
9 De Haagse Hogeschool
10 De Kempel
11 Design Academy Eindhoven
12 Driestar educatief
13 Erasmus Universiteit Rotterdam
14 Fontys Hogescholen
15 Gereformeerde Hogeschool
16 Gereformeerde Hogeschool II
17 Hanzehogeschool Groningen
18 HAS Den Bosch
19 Hogeschool Edith Stein/ Onderwijscentrum Twente
31 Nationale Studenten Enquête
20 Hogeschool Helicon (nu Hogeschool Leiden)
21 Hogeschool INHOLLAND
22 Hogeschool IPABO Amsterdam Alkmaar
23 Hogeschool Leiden
24 Hogeschool Notenboom
25 Hogeschool Rotterdam
26 Hogeschool Schoevers
27 Hogeschool Tio
28 Hogeschool Utrecht
29 Hogeschool v Pedagogisch en Sociaal-Agogisch Onderwijs Groningen
30 Hogeschool van Amsterdam
31 Hogeschool van Arnhem en Nijmegen
32 Hogeschool van Hall Larenstein
33 Hogeschool voor de Kunsten Utrecht
34 Hogeschool Wittenborg
35 Hotelschool The Hague
36 HZ University of Applied Sciences
37 Iselinge Hogeschool
38 IVA Driebergen
39 Juridische Hogeschool
40 Katholieke Pabo Zwolle
41 Markus Verbeek Opleidingen Stichting Hoger Onderwijs
42 Marnix Academie
43 Marnix Academie/Interactum
44 NHL Hogeschool
45 NHTV
46 NHTV/Wageningen University
47 Nyenrode Business Universiteit
48 Open Universiteit Nederland
49 Penta Nova
50 Politieacademie
51 Protestantse Theologische Universiteit
52 Radboud Universiteit Nijmegen
53 Rijksuniversiteit Groningen
54 Saxion Hogeschool
55 Saxion Next
56 Stenden Hogeschool
57 Stichting Hoger Onderwijs NOVI
58 Stichting Hogeschool van Amsterdam
59 Stichting NTI Hogeschool
60 Stoas Wageningen | Vilentum Hogeschool
61 Technische Universiteit Delft
62 Technische Universiteit Eindhoven
63 The New School for Information Services
64 Theologische Universiteit Kampen
65 Theologische Universiteit v.d. Christelijke Gereformeerde Kerken
32 Nationale Studenten Enquête
66 Tio University
67 TMO - Hogeschool voor Modemanagement
68 transnationale Universiteit Limburg
69 Universiteit Leiden
70 Universiteit Maastricht
71 Universiteit Twente
72 Universiteit Utrecht
73 Universiteit van Amsterdam
74 Universiteit van Tilburg
75 Universiteit voor Humanistiek
76 Vrije Universiteit Amsterdam
77 Wageningen University
78 Windesheim Flevoland
79 Zuyd Hogeschool
Table 2 – 2013 NSE participants
1 Most screenshots are only available in the Dutch language. 2 Many thanks to Frank Steenkamp (www.choi.nl) for this example. 3 Berings, D. (2010). Kwaliteitscultuur in het hoger onderwijs: de bijdrage van organisatiecultuur aan de ontwikkeling van kwaliteitszorg, Tijdschrift voor Hoger Onderwijs & Management, 4-10, p. 51-57. 4 Simons, R. (1995). Control in an age of empowerment. Harvard Business Review, March-April 1995, p. 80-88.