histproject.nohistproject.no/sites/histproject.no/files/edumecca... · web viewfrom the student...

EduMECCA

Educational Models that Encourage Creative transfer of Competence and Acquaintance in Lifelong Learning

LLP - ICT project 2008-10

Evaluation results

Circulation: Public Partners: Smartcom (Sweden), MHtE (Hungary), VuZ (Slovakia), IzV (Slovenia),

HiST Contract Research (Norway), University of Huddersfield (UK) Authors: Miro Uran (editor) and others Doc. Ref. N°:

Version: finalStage: Draft Date: 22/11/2010

1

Table of content:

1. Executive Summary................................................................................................................... 3

2. Harmonized project questions............................................................................................... 5

3. Results of Internal evaluation from Project partners:.................................................12PROJECT EXECUTION.............................................................................................................................. 12TIMELINE.................................................................................................................................................... 12TASKS OF THE VARIOUS WPs.............................................................................................................. 12ESTABLISH AND AGREE WORKING METHODS AND TECHNIQUES..........................................12PROJECT MANAGEMENT....................................................................................................................... 12Improvement and updating................................................................................................................. 12Training workshops............................................................................................................................... 12Generation of pilot- exams................................................................................................................... 13Dissemination and Valorisation Plan...............................................................................................13Quality Management.............................................................................................................................. 13

4. Method used during testing in Norway and Sweden....................................................144.1 Questionnaires.................................................................................................................................. 144.2 Student interviews........................................................................................................................... 144.3 Overview of completed surveys.................................................................................................. 15

5. Internal evaluation results in Norway..............................................................................175.1 First evaluation: use of SRS in on civil engineering class in Norway..............................175.3 Third evaluation: use of SRS in several preparatory classes for engineering in Norway........................................................................................................................................................ 315.4 Fourth evaluation: implementing SRS on a regular basis at HiST...................................35

6. External evaluation results from Norway and Sweden...............................................446.1 External testing of the SRS at Ringve High School, Norway...............................................446.3 External testing of SRS at CFL (Centrum for Flexible Learning) in Sweden..................556.4. External testing of SRS in education of welding engineers in Sweden..........................56

7. Conclusion for testing in Norway and Sweden...............................................................65

8. Experiences obtained when using SRS and ABT in IWE and IW courses in Slovakia............................................................................................................................................ 67

9. FINAL EVALUATION REPORT IN SLOVAKIA (Tig welding):........................................75

10. Experiences obtained when using SRS and ABT in IWE and IW courses in Slovenia............................................................................................................................................ 83

11. FINAL EVALUATION REPORT FOR SLOVENIA (Tig welding)...................................92

12. Final Evaluation Report from Hungary........................................................................100

13. External evaluation from Slovenia................................................................................103

14. Summary and conclusion................................................................................................. 106

15. References............................................................................................................................. 106

2

1. Executive SummaryBackgroundThe Mecca pilot project (2005-07) [1] developed the Activity Based Training (ABT) methodology to be used for vocational education and training in industrial production environments. In ABT the students always produce a product during a course. Theoretical and practical training are mixed continuously. Quality assurance of the production is obtained by exchanging the products during the production cycle. Video is used as a facilitator for initiating discussion based up on cases that are illustrated by use of cases that illustrate HOW to DO, as well as HOW NOT to DO.

Figure 1. The Activity Based Training methodology.

At the end of a course, the students have produced a product based up on specifications provided through an order. During the course they will need to learn more theory based up on the specifications in the order, and the practical problem solving process where they must decide in which sequence they are going to produce the components and afterwards assemble them. Quality assurance is integrated into the education model, since the students must check the ingoing and outgoing quality of their products during the production process.

For higher education, however, it will be challenging to copy this method as the courses usually are shorter, since it is not so many hours teaching per week, and since they involve much more theory. Thus, a possible usage of ABT in higher education requires adaptation and introduction of new ICT based methods that for instance utilize mobile technology and mobile learning.

The iPhone arrived on the market in the summer 2007. The idea of developing a communication system that utilized iPhone to collect responses from all students emerged during the winter 2007-08. During the first 6 months of the project it became clear for the Consortium that indeed a web-based solution most probably were going to be the most useful solution for students and their teachers. The reason was that any device that may read a

3

HTML page, might utilize such a solution. Thus, any PC, MAC or modern mobile device may most probably utilize a web-based solution! Based up on this assumption, the EduMecca project developed the open, online Student Response System for modern mobile devices.

This evaluation report describes the evaluation results obtained by using SRS in Norway and Sweden during the early design phase, the testing phase and to some extent the first implementation phase. The user groups proposed in the EduMecca proposal were welders and welder engineers. The SRS has been tested in courses for welders in Norway and Sweden, and for welder engineers in Sweden. We haven’t managed to test the SRS in a continuing engineering course (the International Welding Structure Designer course) in Norway due to delays in the start-up of a relevant course. The IWSD course in Norway should originally start in 2010, but was delayed to 2011 in order to recruit more students. Thus, in order to tests the concept and the usage in higher education, several engineering courses were used instead. This was also necessary in order to optimize the SRS-user interfaces. The next sections contain the details about the evaluations that have been done in Norway and Sweden.

A typical SRS mobile device session has the following structure:1. Handheld units (iPods) are distributed before class begins2. Teacher starts the SRS for mobile devices when he is ready to teach, and the students

enter the session code just before the lecturing starts. The sesson codes allocates the class to one lecture room, whereby several neighbooring classrooms may to use the same WI-FI network in paralell.

3. Teacher present new material from the curriculum4. Students are presented a conseptual quiz and asked to discuss with each other for 2-3

minutes5. The teacher starts a voting session by using a a web interface on the digital blackboard6. Student casts individual votes using the handheld units.7. The vote is closed and results are presented on the blackboard (immediately or when

the teacher decides)8. The teacher comments on the various alternatives and highlight the correct one,

explaining thoroughly why it’s the correct one and why the others are incorrect.9. Go back to point 3 and repeat.

A lesson consists of 2 lectures, each lasting about 45 minutes. During each 45 minutes period the students were usually presented for at maximum of 2 conceptual questions. In order to start a polling session (which usually lasts 30 seconds), a “ticking clock” was used to shift the students attention away from discussion and over to the voting session in progress.

EvaluationEvaluation of the project will take place during the project life time and according to an evaluation plan.

A standardized set of evaluation documents have been developed in order to ensure that the evaluation follows the same pattern and that it becomes unified in form and presentation.

A set of evaluation forms, questionnaires, and results of them are presented this paper.

4

2. Harmonized project questionsCHECKLIST for internal partner Evaluation

Version: 1.0 PROJECT Date: 2009.05.15

Item Topic Status Date Signature1 Check that all the partners understand the aim of

WP1

2 Check that all the partners understand the aim of WP2






8 Confirm that the partners understand and are fully conversant with tools and technology and thetraining requirements of the endusers

9 Confirm and check the deadlines with the partners and adjust if necessary

10 Check that there is a degree of flexibility in the time allowed to produce the work

11 Confirm with the partners that the tasks set are appropriate for them and make changes to the individual workloads if necessary

12 Confirm that all partners are undertaking the work required by this WP and that it conforms to theneeds of the WP

13 Confirm that the evaluation strategy is adequate for the WG objectives

14 Conform that the internal and external evaluation groups are correctly structured

15 Check that the main outcome, the concept definition of the project fits to the aims of the WP´s

16 Check that outcome and results are well documented

17 Confirm accuracy of informationand sources as well as possible

18 Install a feedback process to improvethe quality of outcome andresults

19 Check that interim and final reports are correctly structured

5

B.: Evaluation for end users of SRS; Part A.1. Gender

- Male

- Female

2. Age

3. My first impression of the student response system (SRS) is;

- Excellent

- Very good

- Satisfactory

- Not very good

- Bad

4. Its fun to be at lectures where the SRS is used?

- I totally agree

- I agree

- Neutral

- I slightly disagree

- Sorry I fell asleep…

5. What voting device did you use?

- I Pod provided by teacher

- Private i Pod

- Private cell phone

- Pc

- Other device

6. It was easy to use the voting device

- I totally agree

- I agree

- Neutral


- I totally disagree

7. How was the introduction of the SRS, before it was used in the classroom?

- Excellent

- Very good

- Satisfactory

- Slightly insufficient

- Insufficient

6

8. How would you grade the questions used with the SRS?

- Way to difficult

- Difficult

- Satisfactory

- Easy

- Way to easy

9. To what extent do you think the system, as it has been used, integrate as a natural

part of the training session?

- Excellent

- Very good

- Satisfactory

- Not very good

- Bad

10. Does the teacher seem comfortable using the system?

- Very comfortable

- Slightly comfortable

- Comfortable

- Slightly uncomfortable

- Very uncomfortable

11. To what extent do you feel that the SRS engage and activate you during training?

- Very large

- Large

- Neutral

- Small

- Poor

12. The SRS collects anonymous responses, how important is that for your participation

during training time?

- very large

- large

- neutral

- small

- poor

7

14. To what extent do you think the SRS can aid your learning of the course

curriculum?

- Very large

- Large

- Neutral

- Small

- Poor

15. How would you grade the importance of the teacher spending time on explaining

whether the alternatives are right or wrong, and why?

- Extremely important

- Important

- Neutral

- Not that important

- Waste of time

16. The use of SRS compromises the ordinary training time?

- Very large

- Large

- Neutral

- Small

- Poor

17. SRS should be used in all classes/lectures

- I totally agree

- I agree

- Neutral


- I totally disagree

18. Last but not least; do you have any tips/concrete advice for further use and

development of the SRS?

Evaluation form of the courses in Hungary:

No. 5 = EXCELLENT, 4 = VERY GOOD, 3 = GOOD, 2 = POOR 1= VERY POOR

1 2 3 4 5

1 My personal evaluation of the practical organization of the course ?

8

2 My personal evaluation of the course documentation is ?

3 My personal evaluation of the quality of the video material is?

4 My personal evaluation of using video material with English language is?

5 My personal evaluation of the length of the video clips are ?

6 My personal evaluation of the video examples relevance to my work experiences is?

7 My personal evaluation of the relevance of the course to my work is?

8 My personal evaluation of the learning outcome form the video examples with subsequent discussions are?

9 My personal evaluation of the effectiveness of the training methodology is?

10 My personal evaluation of the video material as a tool that improved the instructional process is?

11 My personal evaluation of the video material as a “tool” that help improving my understanding of welding topics, for instance welding defects, is ?

12 My personal evaluation of the training method as a “tool” that improved my motivation is?

13 My personal evaluation of the timetable in the course is?

14 My personal evaluation of the pedagogical effort of the course leaders is?

15 My personal evaluation of in the future having access to more video clips demonstrating advanced welding technique material is?

16 My personal evaluation of in the future having access to more video clips related to examples from real working life, is?

9

How do you evaluate the product-oriented education against the traditional education?

In the next course, please explain why you want to use the new or the traditional learning environment ?

What is your suggestions for improving the efficiency of the new learning environment ?

You have watched a number of video clips during the course. Please write down any comments about them.

Which part of the training are most relevant for your daily work ?

1-2 YEARS 3-4 YEARS 5-6 YEARS 7 OR MORE YEARS

YEARS OF EXPERIENCE WITH INTERNET?

YEARS OF EXPERIENCE WITH COMPUTERS?

10

1-2 COURSES

3-4 COURSES

5-6 COURSES

7 OR MORE COURSES

HOW MANY COURSES HAVE YOU TAKEN THAT USED COMPUTER-BASED INSTRUCTION ?

11

3. Results of Internal evaluation from Project partners:Internal Project checklist is in Apendix A.

In frame of internal evaluation Partners from SL, HU, SK and NO did checked and confirm that all can agree with positive answers on following questions :

PROJECT EXECUTIONII- 1.1Confirm that the aims of each WPs are appropriate for the Edumecca Project and achievable.II- 1.2Check that interim and final reports are correctly structured.

TIMELINEII- 2.1Check if the deadlines of the various deliverables had been respected and if in case of some delays all the necessary adjustment had been adopted in the workplan.II-2.2In the case of some delays in the deliverable there was a sufficient degree of flexibility in the time allowed to produce the expected work?

TASKS OF THE VARIOUS WPsII-3.1Confirm with the partners that the tasks set are appropriate for them and make changes to the individual workloads if necessary.II- 3.2Confirm that all partners are undertaking the work required by each WP and that it conforms to the needs of the WP.

ESTABLISH AND AGREE WORKING METHODS AND TECHNIQUES II- 4.1Confirm that the target group of Edumecca is understood by all partners.II- 4.2Confirm that the evaluation strategy is adequate for the WG objectives.II- 4.3Confirm that the internal and external evaluation groups are correctly structured.II- 4-4Confirm that evaluation check lists are adequately structured and detailed.

PROJECT MANAGEMENTIP-1.1Was the collaboration among Partners of the Consortium ensured?IP- 1.2Were all the meetings held planned?IP-1.3Was the review of the meetings done?

Improvement and updating IP- 2.1Was the database improved and updated according to the planned subdivision of the tasks?

Training workshops IP- 4.1 Were the workshops fruitful?IP- 3.1Were the supportive documents to give guidance to the use and implementation of the Examination database developed?

12

IP- 4.2 Did the workshops provide a good skill for the execution of the statistical analysis of the exams result?

Generation of pilot- examsIP-5.1 Was the ATB an effective implementation at national levels?

Dissemination and Valorisation PlanIP-6.1 Were the project results disseminated on a European basis in suitable ways, towards stakeholders in the welding community and the manufacturing industry?

Quality ManagementIP- 7.1 Did the evaluation strategy developed took into account the actual needs of the end-user?

IP- 7.2 Did it occur a continuous quality control and quality assurance of results throughout the duration of the project ?

13

4. Method used during testing in Norway and SwedenSeveral trials of the SRS have been completed in Norway and Sweden. Four trials have been performed internal at Sør-Trøndelag University College (HiST), in Trondheim. In addition to the internal testing at HiST, the SRS have also been tested by external users both in Norway and Sweden. In connection with these, numerous surveys, in form of questionnaire and interviews, have been conducted.

The English word survey means overview. A survey is a standardized questioning of a sample of persons on any topic (Ringdal 2001). There are two ways to administer surveys. One is a written questionnaire where informants read the questions and indicate their response on a form. The other way is to use an interview format (Cozby 2003). In other words: survey research employs questionnaires and interviews to ask people to provide information about themselves and their experiences (Cozby 2003).

4.1 QuestionnairesThere are a total of six questionnaires conducted in Norway and Sweden. Questionnaires in one of the most structured survey techniques (Ringdal 2001). With questionnaires, the questions are presented in written format and the informants write their answers. There are several positive features of using questionnaires. They are cost efficient and allow the informants to be completely anonymous as long as no identifying information is asked. However questionnaires require that the informants are able to read and understand the questions, and there may also be a problem with motivation; many people can find it boring to send time on reading and answering questions. Questionnaires can be administered in person to groups or individuals, through mail, on the internet and with other technologies (Cozby 2003).

Current questionnaires were conducted using both paper and internet. Three of the questionnaires were conducted through paper, the questionnaire was distributed to students on paper and students filled it out manually. The remaining questionnaires were conducted online, through Google docs, using a function in the response system called “redirect”. To manage this we created a Google doc account. A Google doc account is an online account where you can create documents, presentations and spreadsheets online. You can use this account to store many types of documents, exchange files and create a survey that is available online and can be analyzed immediately after a response has been given. You then copy the address of the survey and forward the iPods to the address with the redirect option in SRS-interface. Select REDIRECT from the menu, paste the web address in the URL field and press send. The iPods in that particular session will then get the link and students can enter the form and give their response.

4.2 Student interviewsIn addition to the questionnaires, student interviews were also conducted. The interviews were performed as focus group interviews. The main reason for the choice of interview method is that focus interviews are recognized as an independent method of revealing the informants ‘own perspectives about various topics (Johannesen, Tufte & Kristoffersen 2004).

A focus group interview constitutes a form a group interview where the conservation and discussion process is essential. Unlike more conventional forms of interviews, where those conducting the interviews take the role of interviewers, a focus group interviewer takes on the

14

role of a discussion moderator, that is, a moderator who organizes discussion within the groups. One of the main advantages of focus group interviews is that, if properly managed, it can be extremely dynamic (Bergh 2007). Interaction between informants can stimulate discussion in which informants respond to each other`s comments. Some informants may say something that the others have not thought of, or the others just disagree which prompts informants to discuss and go into more depth on that topic. One such group dynamic if often described as a synergistic group effect, an effect that allows informants to build on what others have said or to enter into a collective “brainstorming” process. The idea is that the whole contribution to the process is more than the sum of its parts. Through such a form of group discussion, a larger number of ideas, topics and themes can be entered into thus revealing knowledge and insights that can be difficult to achieve through individual conversations.

The interviews were analyzed using type of analysis called grounded theory, a method for the analysis of qualitative data that has wide acceptance in social science (Johannesen, Tufte & Kristoffersen 2004). Grounded theory is an appropriate direction for the analysis of topics such as personal experience, opinions, feelings and attitudes (Charmaz 2001). The aim of the current evaluation was to bring out the experiences, views and opinions expressed by our students in relation to the use of SRS in physics’ classes. Grounded theory was considered to be an appropriate too for achieving that goal. This method provides specific procedures for the analysis of data, where data is coded in the steps:

Line- by- line coding Focused coding Categorization

4.3 Overview of completed surveys4.3.1 Internal evaluation:

Fall 2009: use of SRS in one construction engineering class- Selection: engineering students at HiST - Method: questionnaire

o number of informants: 59

Fall 2009: use of SRS in one preparatory class for engineering - Selection: preparatory for engineering students at HiST - Method:

o questionnaire number of informants: 57

o focus group interview: number of groups: two number of informants: six analytical approach: grounded theory

Spring 2010: use of SRS in three preparatory class for engineering - Selection: preparatory for engineering students at HiST

o the survey was initially conducted in four classes, due to errors in the questionnaire one of the classes was excluded from the evaluation

- Method: questionnaire o number of informants: 92

15

Spring 2010: use of SRS in two preparatory class for engineering - Selection: preparatory for engineering students at HiST - Method: questionnaire


4.3.2 External evaluation: 2010: Experience obtained with use of SRS at Ringve High School in Trondheim, - Selection: students at Ringve High School- Method: questionnaire


2010: Experiences obtained with use of SRS in education of welders in Norway - Method: interview with the responsible teacher for training within the company Vitec.

2010: Experiences obtained with use of SRS in education of welding engineers in Sweden

- Selection: students from weld on Sweden- Method: questionnaire


Experience obtained with use of SRS at CFL (Centrum for flexible learning) in Sweden

- Written description of experiences of use of the SRS from one the responsible teacher for training within the institution.

16

5. Internal evaluation results in Norway

5.1 First evaluation: use of SRS in on civil engineering class in NorwayThe first testing of SRS for mobile devices was done over a period of 5 weeks in the autumn of 2009. Students’ feedbacks on the system were collected from a questionnaire given at the end of the test period. A selection of the results obtained from 59 students (45 male and 14 female) is displayed in table 1. These results show a clear positive picture on using the SRS during class. Students agree that the lectures were the SRS has been used, has worked very well. The majority of the students think it is fun to be at lectures where the system is used, and think that the system should be used in all courses, not just physics. The SRS encourage them to be active during the lecture, they value the group discussion they participate in before the actual voting, and they feel that using the SRS helps them to learn the curriculum of the course. The teacher’s explanation after the voting session is especially highlighted as an important contributor to their increased understanding.

The following results were obtained in a civil engineering class, in the subject of physics, with 59 students (45 boys and 14 girls) at Sør-Trøndelag University College, HiST. This group was the first one to test the system during 5 weeks in September and October 2009. Blue (dark) columns represent the boys, while the white column represents the girls. In Fig. 1-14, the best alternative have been given a score 5 (left column in the figures) while the worst alternative (right column in the figures) has been given a score 1 in the calculation of the average value and the standard deviation. Thus, in this way the median will have value 3. The evaluation was carried out at the end of the 5 week testing period by distributing and collecting written questionnaires to the class.

No. Survey question/statement Very good

Good Ok Fairly good

Poor

1. My first impression of the SRS is 77

294

61

21

11

2. I think the manner in which the lectures using SRS have worked is

136

245

71

01

11

Very large

Large Neutral

Small Poor

3. To what extent does the SRS encourage and activate the students?

206

206

31

10

11

4. How would you assess the value of the group discussion before voting?

216

165

32

30

21

5. To what extent does the SRS aid your learning of the course curriculum?

73

285

54

40

12

6. To what extent does the teacher’s explanation at the end of the vote aid your learning?

3010

113

21

00

10

Strongly agree

Agree Neutral

Disagree Strongly disagree

17

7. It’s fun to be at lectures where SRS is used

77

264

92

20

11

8. Through the use of SRS, I can give the teacher a response to his/her teaching approach, and thereby influence the learning process

63

223

116

51

01

9. SRS should be used in all classes/lectures

158

194

60

31

21

10. The SRS gives me an opportunity to actively take part during the lecture.

52

248

132

30

02

11. Using SRS during a lecture makes me an active as opposed to passive, student.

30

156

224

42

12

The services are based up on XML-based standards and web authoring facilities for the contents available on web pages, by providing XML-based universal notation and interface including visualization of scientific and engineering drawings and graphs. The search facilities retrieve the postulates of the instructor through a service-oriented architecture that integrates semantic web into the system for retrieval of information from the knowledge base system. The decision process solution system is open and flexible in order to achieve maximum interoperability, and avoid using Apps through the iTunes Store.

Figure 1: The average value is 3.92 and the standard deviation is 0.93. For the boys the average value is 3.87 and the standard deviation is 0.81. For the girls the average is 4.07 and the standard deviation is 1.22.

18

Figure 2: The average value is 4.05 and the standard deviation is 0.91. For the boys the average value is 4.07 and the standard deviation is 0.80. For the girls the average is 4.00 and the standard deviation is 1.20. .


19



20



21

5.2 Second evaluation: use of SRS in one preparatory class for engineering in NorwayThe second testing of the SRS was also done in the autumn of 2009, in a preparatory class for engineering in the subject of physics and lasted over a period of 4 weeks. The evaluation was carried out at the end of the 4 week testing period, and consisted of two focus group interviews and a questionnaire. Six students participated in the interviews, with three students in each group. The interviews were analysed using the analytical approach called grounded theory. The questionnaire was conducted online, through Google docs, using the “redirect” function in the response system. A total of 57 students participated (39 boys and 17 girls).

5.2.1 Results from student interviews: The analysis resulted in three categories:

A. Feedback on own learningB. Increased engagementC. Group discussion

These categories are made up of and represent the students' main experiences in relation to the use of student response system in class.

A. Feedback on own learningImmediate feedback on their learningUse of SRS provides students with valuable feedback on their learning and progression. To answer the quiz questions and receive immediate feedback was a way for students to test themselves on the fly. They got a immediate feedback on whether they had understood what the teacher had tried to convey, whereby they immediately tested their knowledge in practice. Two of the students had the following to say about the feedback that SRS gave them;


22

Per:"You get a feedback on how well you have understood the topic. For if you have selected the correct alternative to a quiz, you get feedback that you’ve understood this subject. You get feedback that you are able to use the formulas and laws – yes, the material that the teacher has presented."

Ole:"Yes, you get a feedback on whether you have understood it. Yes, it’s all about your own learning. You get to see if you've learned something."

For students is feedback an important part of their learning. Feedback tells them how they are doing in their own learning process. The feedback activities these students get are usually related to various types of tests and assignments. It is rarely provided any kind of feedback activity during the actual teaching time. The only opportunities the students have to receive feedback are by raising their hand and either ask or answer questions from the teacher. Whether students will receive feedback during the lessons depends, in other words, on them self, and whether they take the initiative and actually reply to or ask questions. From the student side, it too often ends up with very little feedback, because they find it very uncomfortable to raise their hand and talk loud over dozens other students. When asked whether the teacher included some feedback activities during their lessons, one of the students said;

"No, the teachers may ask, "do you understand?” and then they just look sheepishly at us and move on. None of us that dare to raise our hand and respond. In that sense, it’s our own responsibility, but no, I certainly don’t. Feedback activities are normally not included, which is a bit of a shame."

Students desire something they call constructive feedback in their academic life. This is feedback that in addition to providing them with a pointing sticks if they're on the right track or not, explain why something is right or possibly wrong. From the student side, this is feedback that gives them something concrete to work with; they get feedback on what they are possibly struggling with and what they need to focus at. Without such feedback students feel that they only have themselves to deal with, making it difficult for them to know where they stand in relation to their own learning and progression. One of the students said it quite clearly;

"Without constructive feedback, how can we really improve?"

Teacher's explanation afterwards: the key to students' understandingIn relation to students' desire for more constructive feedback, SRS came in as a long awaited breath of teaching. Firstly, the system gave them an immediate feedback on their polling, in that they got to see if they had voted right or wrong, as well as the teacher went through each option after the vote and explained thoroughly why they were correct or incorrect. For students, the teacher's explanation was perceived as a constructive feedback, and was further highlighted as critically important in relation to their experience of learning. In short, it was here that the learning came into play. Through the teacher's explanation the students get an understanding of why the alternatives were correct or incorrect. One thing is to cast a vote that turns out to be right or wrong; another thing is to really understand why it is right or wrong. If they achieve such an understanding; they feel they have really learned some of the quiz questions. Especially the wrong options are highlighted as important to get a thorough explanation around. For students, this is all about giving those who have answered incorrectly a chance to understand why they answered incorrectly. One of the student groups explained it this way;

23

Emma:"Those of us who got the answer wrong have to be given to chance to understand that we were wrong. Some part of the class usually got it wrong, and then it must be explained in such a way that we can understand where we went wrong. Because we obviously don’t know if an option is wrong – otherwise we wouldn’t have voted for it! "

Lise:"Yes, I feel it gives me a chance to understand what the subject is really about."

The second group had the following to say;Per:"There's a reason why people have answered incorrectly, it's because they have misunderstood something, and then they have to be explained why the answer was wrong."

Ole:"Yeah I think the explanation from the teacher is very important. I think it is necessary that he explains why he uses certain laws, or other parts of the curriculum, and that he shows us why it is right or wrong. "

Jens:"Spending some time to explain or discuss the different options and how to the right answer in this way is; well, I feel that the quiz becomes a bit useless if you don’t do that - if you don’t spend enough time on it and do it thoroughly. The quiz then becomes – maybe not useless, but the quiz has a much greater effect on learning if you get an explanation why the answers are right or wrong. "

For improvement – how to get a better feedback on the “actual understanding”?From the student side, there is little doubt that the use of SRS can provide them with valuable feedback on their learning, particularly if the teacher gives them a thorough explanation after the vote. At the same time, however, they leave no doubt that the SRS may have a much greater learning potential than was used in their teaching.Traditional teaching where planned for the test group of students. They usually have two or three lessons after each other. During trials of the SRS they usually got the first quiz question a little off in the first teaching hours (often after about 20-25 minutes), the next question came either later in the same teaching hour or a bit out in the next one. The times they had three hours in a row, they had no quiz questions in the last hour. The teacher started the lesson by introducing a new part of the curriculum. Afterwards the students got a quiz questions based on what they had recently been presented. According to students, this was a straightforward way to implement SRS in teaching, as they due to the teacher's explanation after the vote felt that they learned something from it. In relation to test their understanding, however, this was no optimal solution. The quiz questions were introduced too early. Whether you test comprehension or not, is according to the students depending on the time the quiz question is being asked. It is also depends on the available time they have for learning before the question is asked. In other words, if a teacher wants to use SRS to test students' comprehension, students must first be given time to work with the curriculum and acquire the academic requirements they need to answer the quiz question. I they get the quiz questions too early in the teaching hour it is not certain that students have got these prerequisites. Thus, instead of testing their understanding through the use of SRS, the teacher gives them an understanding through a thorough explanation after the voting.

Group 1:Per: “I would like to get a quiz at the end of the day too, in order to check if we’ve really understood it. After we’ve worked with the exercises for a period of time, and had time to process the material.”

Jens: “Yes, I agree.”

24

Per: “That would give a very good indication as to whether you’ve understood something or not. That would be a proper test!”

Ole: “Then we would have worked with it for a bit, and then we’ll get to see if we’ve understood it.”

Group 2:Emma: “I somehow ... need time to understand it, in a way. Sometimes I think that the quiz questions seemed to come too early for me, in a way ... There were times when I just made a guess. I had somehow not received the scientific basis for properly discussing it. I felt that it was a bit unnecessary.”

Ingrid: “Yes, we’d almost have to lie ahead, if we are to do it that way. The questions tend to be from the new subject area that we’ve just been through. So really, it might be best if he took us through the curriculum first, and included questions at the end of the class.”

Lise: “Yes, to see that people had actually … understood it.”

The students point out that in many ways it is up to the teacher how he/she wants to use the SRS. The teacher may use it to give the student an understanding, by giving them a quiz question followed by a thorough explanation, or he/she may just test their understanding. According to the students, we had in no way selected to use the SRS in an incorrect way. They just pointed out that to really test their understanding, they must first possess an understanding of the curriculum thought, and it is rarely in place after only 20 minutes. It comes usually after the material has decreased slightly, which usually happens when they have worked a bit with it through exercises. If we will test their understanding it is at this point they should get quiz questions.

B. Increased engagementFinally there's something else! Use of SRS is a great way for the students to engage in the teaching process. By answering the questions they participate actively in the production of educational content, and their involvement increases. In contrast to ordinary classes, where students feel they have more than enough to listen and take notes, use of SRS introduces a break where they get time to think about, resonate around the educational material and process, and simultaneously test themselves. From the student side, it is impossible to maintain the concentration in one or several hours. It is simply too much information to absorb at once, whereby it is easy to lose the concentration and the motivation drops. Using the SRS, however, stop the regular teaching process in the class and the students may recover during a short time frame where something else happens! Students welcome a distraction introduced by SRS, instead of just sitting and receiving information. They become activated and motivation increases. One group of students discusses the increasing involvement in relation to the use of SRS in the following manner;

Per:"You participate, yes, you are active in that you work with quiz questions and talk to other students. I was more engaged when using the system. But it has an effect on motivation as well. I think it's a bit exciting, a bit of fun, and it made classes more fun."

Ole:

25

"It was like a small activity in the middle of the lecture, which restored my motivation when I started to doze off. I think, the lecture, well, there’s too much information at once, you cannot keep up. So it’s refreshing that you get to think for yourself, even if it’s just for a short period, and get to answer questions.”Jens:"Yes, I felt that the class got a motivational boost, and became more active. It is definitely one thing that helps to maintain interest during classes! The tests are seen as a bonus, "soon a quiz will come and then I may test myself to see if I’ve understood it." I see nothing but positive aspects with SRS."

Per:"Me too. I’m normally not very active, so it was fun to join in and participate"

Jens:"Yes it was very, very positive, a real bonus."

Ole:"Yeah, well, you get a break from the usual lecture."

Anonymity: the magic key Although SRS offers students a much sought-after break from the ordinary teaching process in classes, it is according to the students the way the system allows them to participate that is the main reason behind the increase in their commitment. SRS offers students something that ordinary education is missing: a chance to participate anonymously. In contrast to ordinary classes, where students' participation usually involves raising their hands with oral responds, they may use the SRS to answer questions without that answers are traced back to them. What they respond, and if they answer correctly, it is only they themselves that know. The students explain that anonymity is a crucial role in relation to their participation and usage of SRS. In short, anonymous responses made it sure that they participate. One of the students describes the role of anonymity in the following manner;

"It ensures that everyone participates! Everyone may provide his or her vote and their “voice” will be heard (pause), that will never happen in a normal class. It [the anonymity] was the key factor which convinced me to attend, no doubt. "

The most common option, and often the only one, where students may have the opportunity to participate actively in a lesson is by raising your hand to either answer or ask a question. From too many students point of view, this is not a particularly attractive opportunity. Far too many find it uncomfortable showing off by raising their hand and talk. They are afraid to make fools of themselves, either by asking a stupid question or answer incorrectly. They have all experienced to wonder of something but not asked any question, or to avoid answering questions from the teacher, especially if they are not sure about the correct answer. The fear of exposing themselves to the class prevents them simply from active participation in the class. Students define their own role in an ordinary teaching as a spectator, and not a participant! The usage of the SRS reduces the threshold for active participation significantly. By answering the quiz by using SRS, their anonymous participation was placed in safe limits. Everyone could answer without having any fear of dumb out towards fellow students. Use of SRS was a new way for students to solve tasks, which resulted in the response from the entire class, versus the usual few.

Ultimately: learning at allFrom the student side, engagement and learning are flip sides of the same coin. One does not exist without the other! Commitment is a prerequisite for obtaining good learning. In addition it provides a better experience of learning. Being involved is described in this context as being mentally present. The student’s concentration and attention are sharpened, and they find it

26

easier to absorb information. The use of the SRS helped to initiate such processes among the students. By getting the opportunity to think for themselves, discuss with the person sitting next by, answer questions and receive responses, the students experienced increased engagement. They were activated and felt that the concentration, which often disappears during traditional teaching hours, were awakened. Four of the students had this to say about both the commitment and learning, and SRS's role in relation to commitment;

"As far as learning is concerned, unless I’m somehow engaged, I simply drop out and start thinking about other things. So for my learning, I need to be engaged during lessons to be able to absorb information. "

"You learn a lot better when you are engaged. You are more “on the ball", because you get to conclude with different things – pulling different cords in a way – you think more actively about things, you work better with the material, and you learn better! "

"I think it was good, because you are a bit like, “aha, now we’re going to run a quiz”, and you get a bit excited like that. You never as much attention as when you get the quiz up on the board. So I think, for me there was more involvement. You were more into it! "

"Let me put it this way: at least I woke up"

C. Talk with the person sitting next by Group Discussion: a good way to learnBefore each vote using the SRS, the students were encouraged to consult the persons next by discuss the quiz question and its alternative in a few minutes. An encourage that no doubt was taken seriously by students. From their side, to collaborate with other students is an effective way to learn. Hearing others' perspectives, opinions and viewpoints are highlighted as important to get deeper into curriculum and achieve a better understanding. The discussions gave them, in other words a professional benefit. Students described the discussion as follows;

Group 1:Per;"I think I learned something from them, absolutely ..."

Ole"It's always nice to get the opinion from the person sitting next to you"

Jens"Yes, when you see the questions, you form an opinion that goes one way, and then along comes the person next to you with a different opinion. Thus, you get input from somebody who may think in a completely different way, and you just realize, "I never thought about that”. Yes, you get a chance to discuss what the correct option is."

Group 2:Lise"It's very nice to be given the opportunity to speak with someone, especially since we’re covering subject areas that are new to us. It’s good to hear what others think, and together try to achieve a common understanding."

Emma"Yeah I think it worked really well. We tried to reach an agreement on the correct answer. So, if we disagreed there would be a very good discussion. You knew that both sides couldn’t possibly be right, so you’d turn the material a bit upside down and discuss it. Very good. "

Targeted discussion, they discuss not only to discussThe discussion with the person sitting next bye was perceived as a valuable element in relation to the use of SRS. One reason for this is that the discussion had a clear goal, it should end in a vote that would give students feedback on their learning. They discussed, with other words, not only to discuss, they discussed to be better understand and to cast their vote. The

27

goal was to find the correct answer that would further give them a positive feedback, and the discussion raised the chances of achieving that goal. The feedback students receive when using the SRS was thus a "carrot" that motivated them to participate actively in the discussion. According to the students this made the discussions focused and efficient. They only had a couple of minutes to try to discuss and try to find the correct answer, whereby they must use the time efficiently. One student group had the following comments about this;

Emma:"The point of the discussions we had, was to figure out an exact answer. Otherwise, when we are discussing, I think the discussion very quickly loses focus, or at least becomes a rather “free-roaming” discussion."

Lise:"Yeah true, I think voting is very important! I don’t think we would have bothered to discuss with the person sitting next to us if it would have been for nothing; if I didn’t cast a vote afterwards"

Emma:"You motivation increases."

Ingrid:"You put more into the discussion, to find the right answer."

Lise:"Yeah, that sums it up nicely!"

5.2.2 Results from questionnaire:The following results were obtained from the questionnaire: similar to results from the first trials, and in line with the interview results, the students were very positive to the use SRS in class. Most of the results from the first trial were confirmed. The student’s perceive the system as an integral part of their physics classes, and they did not feel that the use of SRS came at the expense of ordinary teaching. According to the students the system should be integrated as a regular part of their teaching, at all lectures, in all course. The students perceive the response system as a positive contribution to ordinary teaching. They experience an increased engagement, they get an opportunity to participate anonymously, which they consider as very important in relation to their participation, and they participate in valuable discussions with their fellow students. Several of the students feel that they learn better, and like the previous evaluation results, they especially highlight the teacher's explanation after the voting as an essential factor for their learning.

5.2.3 Graphs from the questionnaire:

1. My first impression of the student response system (SRS) is:

Very good 23 46 %

Good 23 46 %

Satisfactory 4 8 %

Bad 1 2 %

28

2. To what extent do you think the system, as it has been used, integrate as a natural part of the training session?

3. To what extent do you feel that the SRS engage and activate you during lecture?

4. How would you grade the importance of the group discussion you engage in before the actual voting?

Very large 7 15 %

Large 23 50 %

Neutral 13 28 %

Small 3 7 %

Poor 0 0 %

Very large 7 15 %

Large 23 48 %

Neutral 16 33 %

Small 0 0 %

Poor 2 4 %

Very important 11 22 %

Important 23 47 %

Neutral 8 16 %

Not very important 6 12 %

Not important at all 1 2 %

29

5. To what extent do you think the SRS can aid your learning of the course curriculum?

6. After the voting, how important is it for your learning that the teacher explains which alternatives are right and wrong, and why?

7. The SRS collects anonymous responses, how important is that for your participation during the lecture?

Very large 3 7%

Large 19 43%

Neutral 19 43%

Small 2 5%

Poor 1 2%


Important 19 43 %

Neutral 19 43 %


Waste of time 1 2 %


Important 15 33 %

Neutral 9 20 %



30

8. The use of SRS does not compromise the ordinary training time?

9. It’s fun to be at lectures where the SRS is used:

10. SRS should be used in all classes/lectures?

5.3 Third evaluation: use of SRS in several preparatory classes for engineering in NorwayThe third set of trials of the SRS at HiST began January 2010 and lasted for seven weeks. Trials were conducted in four classes, with about 50 students per class in the subject of physics.

I totally agree 12 27 %

I agree 24 55 %

Neutral 6 14 %

I slightly disagree 2 5 %

I totally disagree 0 0 %

I totally agree 7 16%

I agree 29 66%

Neutral 7 16%

I slightly disagree 1 2%

I totally disagree 0 0%


I agree 19 43%

Neutral 6 14%



31

After trials were finished, a total of eight interviews and a questionnaire were completed. The interviews were, as with previous evaluations at HiST, conducted as focus group interviews. The survey questionnaire was conducted in all four classes. In one of the classes, an error in the questionnaire was discovered. Consequently, only the evaluation results for the three remaining classes are available. 92 students participated (76 boys and 16 girls). The questionnaire was conducted online, through Google docs, using the “redirect” function in the response system.

5.3.1 Results from questionnaire:The evaluation shows, as in previous evaluations, that the students are positive about the use of SRS. They have a good impression of the system and view SRS is a fun contribution to regular classes. They participate more actively in classes by answering questions and discussing in small groups. Several students report an increased level of interest and a better understanding of the curriculum. Once again the students highlighted the teacher's explanation at the end of the voting session as essential to the learning process. Additionally, the teacher’s involvement with and level of interest in the system was seen as particularly important in relation to how the SRS was perceived by the students. Below is a quote from one of the interviews with the students: "Yes, there’s no doubt that the teacher’s involvement is transmitted to the students, in every context." In contrast to previous evaluations, this study shows mixed experiences among the students related to anonymity and its significance to student participation. Students also have some diverging opinions about whether the use of SRS comes at the expense of ordinary teaching.

5.3.2 Graphs from the questionnaire

1. Male/female:

2. My first impression of the student response system (SRS) is:

Male 76 83%

Female 16 17%

Very good 15 20%

Good 37 49%

Satisfactory 14 18%

Slightly bad 6 8%

Very bad 4 5%

32

3. Its fun to be at lectures where the SRS is used:




I agree 37 49%

Neutral 14 18%



Very large 7 9 %

Large 29 38 %

Neutral 27 36 %

Small 9 12 %

Poor 4 5 %

Very large 16 21 %

Large 29 38 %

Neutral 21 28 %

Small 7 9 %

Poor 3 4 %

33



8. After the voting, how important is it for your learning that the teacher explains which alternatives are right and wrong, and why?


Important 19 25 %

Neutral 24 32 %



Very large 2 3%

Large 38 50%

Neutral 19 25%

Small 13 17%

Poor 4 5%


Important 29 38 %

Neutral 9 12 %


Waste of time 0 0 %

34

9. How would you grade the importance of the group discussion you engage in before the actual voting?

10. How important is the teachers’ commitment for your impressions and experiences of the SRS?

11. The use of SRS compromise the ordinary training time?

5.4 Fourth evaluation: implementing SRS on a regular basis at HiSTAfter several short trials of the SRS, we decided to implement and test the SRS as a regular part of teaching in preparatory classes for engineering courses at HiST. The aim was to test the SRS over a longer period of time, in several subjects, in several classes. As a result the SRS was implemented as a regular part of teaching in two out of four preparatory classes. SRS was taken into use in two preparatory classes, while the two remaining classes serve as so-called reference groups. The system was implemented in the subjects of physics, mathematics and social sciences. The trial will initially last for two semesters; the fall term of


Important 40 53 %

Neutral 15 20 %




Important 35 46 %

Neutral 10 13 %




I agree 11 14 %

Neutral 24 32 %



35

2010 and spring term of 2011. A total of eight teachers are involved: four physics teachers, two math teachers and two teachers from social science.

Unlike previous trials, in which the SRS has been tested for a limited period of time, the target this time was implement the SRS as a regular part of students' teaching over a longer period of time. In earlier trials the SRS has been used in teaching a few times a week, in one subject, over a short period of a few weeks. The evaluations from these trials show us that students see the use of SRS as a positive contribution to ordinary teaching. In the current trial, the SRS will be used in about 20 lessons per week over a period of eight months, which means that students will use the system several times each day. An interesting question in this context is whether the students will respond as positively to the use of SRS as previous students have. In other words, will the use of SRS be perceived as a positive contribution in teaching when it is implemented as a regular part of teaching over a longer period of time? Or will we face a saturation point, in which the students get fed up and their engagement weakened.

A practice that differs from previous trials is that students now got the opportunity to borrow an iPod through the entire trial period. Previously, the teacher brought the iPods to each lesson, distributed them to the students and collected them back again. Now the students were able to borrow an iPod which could be used in the sessions with SRS. Beyond that, they can use the iPod as they wish. All students signed a contract where they agreed to deliver the iPod back after completing the trials. If the agreement is not kept, i.e. the iPod is not returned at the end of the term, students may be held financially responsible.

After the first semester with SRS as a regular part of teaching in three subjects, an evaluation was carried out through the use of questionnaires. A total of 76 students participated. The questionnaire was conducted online, through Google Docs, using the “redirect” function built in to the SRS. Given that SRS got tested in three different subjects, physics, mathematics and social science, the evaluation was twofold. That means that the questionnaire not only measured students' overall impressions and opinions about the use of SRS, but also the students’ experiences related to the three different subjects it was used in.

5.4.1 Results from questionnaire:The results of the evaluation clearly show that increased use of SRS has not made the students less positive to the usage of SRS in class. In fact, 77 % of the students think that the SRS should be used further in their education, and not only in the preparatory classes. In other words, the students are, as in previous evaluations, overwhelmingly positive. They feel that the use of SRS gives them valuable feedback on their understanding of the course curriculum. They get an appreciated opportunity to discuss with other students, and experience increased involvement and participation in classes with SRS. The results shows mixed experiences among the students when it comes to the anonymity aspect and its significance to their participation.

As mentioned above, the students got an opportunity to borrow an iPod during this trial, an opportunity most of them chose to use. However, the evaluation shows that not everyone needed to borrow an iPod. Several of the students actually have their own devices they can use during voting, such as iPhone, iPod, other smart phones or computers. In fact, only 32% of the students did not have a separate unit that could function as a voting device.

With regards to the division between the three courses in the evaluation, physics, mathematics and social science, the results show us that the SRS is well integrated in all three subjects, especially in mathematics and physics. The students think the system is used in appropriate amounts, apart from in social science where some of the students think the system could be

36

used more. Most of the students perceive the SRS as a positive contribution to their learning, and think that SRS can aid their learning of course curriculum. As in previous evaluations, mathematics stands out in this respect. All in all, the students seem very satisfied with the use of SRS on a regular basis. When all three subjects are compared, the SRS is considered most positive in relation to usage and learning effect in mathematics and physics.


1. Do you own a device that you can vote with, in addition to the iPod that you have borrowed from HiST?

People may select more than one checkbox, so percentages may add up to more than 100%.

2. My impression of the SRS is:

3. How would you grade the quiz questions used with the SRS?

Yes, I have a iPhone 13 23 %

Yes, I have a iPod 3 5 %

Yes, I have a Smartphone 15 26 %

Yes, I have a laptop (PC/Mac). 20 35 %

Yes, other 0 0 %

No, 18 32 %

Very good 27 47%

Good 22 39%

Satisfactory 8 14%

Slightly bad 0 0%

Very bad 0 0%

37


5. Use of SRS gives me feedback on my own understanding of the course curriculum:

Very easy 0 0%

Easy 3 5%

Satisfactory 42 74%

Slightly difficult 12 21%

Too difficult 0 0%

Very large 7 12 %

Large 35 61 %

Neutral 13 23 %

Small 2 4 %

Poor 0 0 %


I agree 34 60%

Neutral 7 12%



38

6. How would you grade the importance of engaging in a group discussion before the actual voting?


8. Have you ever experienced not to get online with your voting device?


Important 35 61 %

Neutral 7 12 %




Important 24 47 %

Neutral 16 28 %



Yes, very often 1 2 %

Yes, some times 8 13 %

Yes, but rarely 35 61 %

No, never 13 23 %

39

9. The use of SRS compromise the ordinary training time?

10. Do you think the SRS should be used further in your education?

11. To what extent do you think the system, as it has been used, integrate as a natural part of the physics teaching?

12. To what extent do you think the system, as it has been used, integrate as a natural part of the mathematics teaching?


I agree 0 0 %

Neutral 18 24 %



Yes 44 77 %

Yes, but in small amounts 8 14 %

Yes, but in bigger amounts

than in current usage 1 2 %

No 1 2 %

I don’t know 3 5%

1 - Poor 0 0%

2 5 9%

3 7 12%

4 31 54%

5 – Very large 14 25%

40

13. To what extent do you think the system, as it has been used, integrate as a natural part of the social science teaching?

14. I think the SRS is used to much/little in the physics teaching:

15. I think the SRS is used to much/little in the mathematics teaching:

1 - Poor 1 2%

2 0 0%

3 6 11%

4 24 42%


1 - Poor 1 2%

2 8 14%

3 16 29%

4 20 36%


1 – Too little 1 2%

2 13 24%

3 38 69%

4 3 5%

5 – Too much 0 0%


2 6 11%

3 47 82%

4 3 5%

5 – Too much 0 0%

41

16. I think the SRS is used to much/little in the social science teaching:

17. To what extent do you think the SRS can aid your learning of the course curriculum in physics?

18. To what extent do you think the SRS can aid your learning of the course curriculum in mathematics?


2 27 48%

3 23 41%

4 1 2%

5 – Too much 1 2%

1 – Poor 2 4%

2 3 5%

3 20 35%

4 27 47%


1 – Poor 3 5%

2 2 4%

3 14 25%

4 31 54%

5 – Very large 7 12%42

19. To what extent do you think the SRS can aid your learning of the course curriculum in social science?

1 – Poor 7 13%

2 5 9%

3 18 32%

4 24 43%


43

6. External evaluation results from Norway and Sweden

6.1 External testing of the SRS at Ringve High School, NorwayFour college teacher students tried out the system in the spring of 2010, as part of a research project. They were deployed at Ringve secondary school in Trondheim, where they would teach 3rd-year students. SRS was used as a regular part of their math classes for three weeks. Trials were completed with an evaluation, in the form of questionnaires and group interviews. Before the trials started, the college teacher students came to HiST where they received technological and methodological training in the use of SRS. They also borrowed a PC pre-installed with the SRS-Ci, a wireless router, 30 iPods and chargers for the iPods. 6.1.1 Methodology A typical SRS session was conducted as follows: the students came to classes and picked up their iPod. The teacher started teaching as normal, using traditional teaching methods. During the course of the class, two multiple-choice sessions with using the SRS were conducted. The assignments were designed by the teacher, based on what students should have learned, or sometimes to test whether students were following the lecture. The questions were read out loud by the teacher, and at the same time shown in writing (using, for example, PowerPoint, overhead projector or a normal whiteboard). The students were given a few minutes to discuss the assignment among themselves in small groups, before casting a vote. After the vote, the results were discussed in class. Both students and teachers participated in the discussion. All response options were discussed, both those that were correct and those that were incorrect.

6.1.2 The teacher students` experiences in relation to the use of SRSBased on our experiences, we believe that the SRS can contribute as a positive tool in the assessment of learning. An important point in this context is that students become increasingly more active both the learning process, as well as the assessment of the teaching being given. Particularly in large classes, we believe that the SRS can act as a good tool to provide continuous feedback. Communication with the students was an important aspect of using SRS in our classes. By communicating with the students to a greater extent than by traditional teaching, we believe we made an established a better report with them, which in turn helped to improve the learning environment of the class. Further on in our practise period we saw the importance of activating and motivating students during classes. By introducing the SRS in classes we noticed that students became more active and motivated, which also contributes to a more including and forgiving classroom culture. As future teachers, we therefore wish to stimulate the students to be orally active and, and to encourage students to participate in reflective discussions. Our experience from the practice period is that many teachers find it scary and difficult to try new things in education. We experienced the same thing when we started the research project. It was actually frightening to be introducing a brand new technological system in classes, but after a short trial period we learnt that we mastered it without major problems. The more comfortable we were on the system itself, the better our use of SRS in classes became. Our experience suggests that when a teacher is willing to take a leap of faith (technology-wise), it

44

can improve the way we teach. We hope that in the future, we don’t allow ourselves to be locked into a fixed pattern of teaching, but rather can challenge ourselves with new and exciting teaching methods. 6.1.3 Student evaluation: results from interview and questionnaireStudents experienced the SRS as a positive element in mathematics teaching. The training was more fun, it increased involvement and the students expressed that they had a more active role in classes. Student quotes: "It was a positive experience to use the SRS. It turns the math into something positive. " "Using the SRS is fun, and when something is fun, it is easier to learn" "Exciting new way to learn!"

"It's anonymous, so no one is afraid to answer incorrectly."

SRS was also seen as positive in relation to students' reflections on their own learning and learning process. In other words, SRS caused them to reflect more of their own learning. By answering questions during class, they got a feedback about their own learning process, and they could use this information to plan which parts of the curriculum they had to work harder with. Students emphasized the importance of getting a concrete response, which allows them to prepare and plan further learning. Student quotes: Sturla: "SRS tells me where I stand in relation to the curriculum."

Robert: "You got like a concrete response as to what was right and what was wrong."

Sturla: "And then you find out what you need to practice harder." After each vote, the students were encouraged to participate in a larger class discussion, where the various options and responses were discussed. These discussions were perceived as very useful in relation to their own learning by the student. The discussions gave them the opportunity to discuss with several fellow students and teacher. They received clarification about the different options while at the same time processing the subject material in a more active way. Student quotes:

"I learn more using the SRS is such a way that the answer options are reviewed and discusses in class after the vote. "When you reviewed and discussed the various options - what was wrong, and why – we felt like we were absorbing knowledge in very much the same way that a sponge is absorbing water.”

The SRS was perceived as an integral part of their mathematics classes. They did not feel that the use of SRS came at the expense of ordinary teaching. On the contrary: several of the students think that the system should be used as a regular part of their teaching in all subjects. Furthermore, the iPods were perceived as simple-to-use voting devices; the quiz questions were rated as satisfactory, and the teachers seemed comfortable with using SRS. As with previous evaluations, the teacher's explanation after the vote is once again highlighted as essential. So when a teacher initiates a larger class discussion after the vote, it’s important that

45

also the teacher contributes to the discussion. From the student side, it is important that the teacher involved and give his testimony about what is right and wrong, and why.


1. Male/female:

0

5

10

15

20

25

30

Male Female

2. My first impression of the SRS is:

0

5

10

15

20

25

Excellent Very good Satisfactory Not very good Bad

46

3. It is fun to be at lectures where SRS is used:

0

5

10

15

20

25

I totallyagree

I agree Neutral I slightlydisagree

Sorry I fellasleep…

47

4. It was easy to use the voting device?

0

5

10

15

20

25

30

35

40

I totallyagree


I totallydisagree

5. How was the introduction of the SRS before it was used in the classroom?

0

2

4

6

8

10

12

14

16

18

20

Excellent Very good Satisfactory Slightlyinsufficient

Insufficient

48


0

5

10

15

20

25

30

Way todifficult

Difficult Satisfactory Easy Way to easy


0

5

10

15

20

25


49


0

2

4

6

8

10

12

14

16

18

Verycomfortable

Slightlycomfortable

Comfortable Slightlyuncomfortable

Veryuncomfortable


0

2

4

6

8

10

12

14

16

Very large Large Neutral Small Poor

50

10. The SRS collects anonymous responses, how important is that for your participation during training time?

0

2

4

6

8

10

12

14



0

5

10

15

20

25


51

12. How would you grade the importance of the teachers spending time on explaining whether the alternatives are right and wrong, and why?

0

2

4

6

8

10

12

14

16

18

20

Extremelyimportant

Important Neutral Not thatimportant

Waste of time

13. The use of SRS compromises the ordinary training time:

0

2

4

6

8

10

12

14

16

18


14. The SRS should be used in all classes:

0

2

4

6

8

10

12

14

16

18

20

I totally agree I agree Neutral I slightly disagree I totally disagree

52

6.2 External testing of SRS in education of welders in NorwayCurrent evaluation was conducted through an interview with the responsible for training within the company Vitec, Mr K. Berg. This company provides training within material testing and welding all over Norway. They have ca. 50 instructors and testers.

Vitec in Norway has tested the system in several types of training activities, and their teachers in various types of activities and courses, have used the system. This includes:

Courses for apprentices and vocational students at various levels, and in classes of various sizes. The small groups have been from 10 to 12 students, while the big groups has included up to 30 students

Verification of knowledge that has been given to all kind of external users that visits Vitec. The system has been used to control if information has been understood. Such use includes both external teachers visiting Vitec, as well as students following professional courses provided by Vitec. By providing some control questions, Vitec may verify that teachers and/or students actually know!

Check if the students have understood the information they get from Vitec. This includes both written information, as well as information provided by the teacher in the classroom.

The company wishes to start testing and using the system in distance learning. It may be used in many ways inside the company. By using the system the company creates engagement within the student group. Their experience shows that use of the system creates a lot of engagement in the class. However, their teachers need to train more in designing and developing questions. The system is very easy to use if this training has been done.

The apprentices have got one iPod each. They get a question and vote after a period with thinking. If most of the students have selected the correct answer, the teacher sum up and confirm what is correct, why the other are incorrect. If not a large part of the group has selected the correct alternative, the teacher stimulates them to take a discussion and then the group revote. This is approximately a peer review instructional process.

In the large classes have Vitec organized the students in small groups, where each group has got one iPod. The group must discuss in order to decide which alternative is correct, before they vote.

Mr Berg at Vitec experiences that the SRS help students “that usually don’t want to be exposed by answering theoretical questions, or participate in theoretical discussions”, to increase their engagement. They experience that the SRS encourage and stimulate use of discussions inside the class. Their students are very positive, and report the usage of anonymous responses as very important in vocational education and training courses.

Mr Bergh at Vitec suggest using the SRS as a tool in the International Welding education that is soon going to start in Norway, and in particular use it a lot to stimulate discussions in those areas where the students traditionally has week skills. Mr Berg claims “it become easier to teach for the instructor due to the frequent feedback provided by the students”. In addition “it becomes more fun to teach“ for the teachers. Use of SRS stimulates the student group to “start discussing professional issues”. Furthermore, it is also necessary to adapt the teaching room in order to enhance the use of SRS. The room that Vitec has today are too small, as they have been designed for traditional ways of providing teaching. It must be enough space such that it is possible to discuss in groups.

53

Vitec suggest starting implementing SRS into training provided by the Aker Group (large international industry company that has their main office in Norway) in many fields. Vitec has demonstrated their use of the system to the leaders that are responsible for training inside this company. SRS is according to Mr Bergh a convenient tool for “controlling and verifying if the students actually have obtained the knowledge and information they have got inside the company”. The current version is OK for use in vocational education and training. It may be small delays in updating of the buttons due to the capacity in the wireless network.

Vitec has experienced that it is not enough to create questions on the fly, as these questions usually become too easy to answer. It is necessary to prepare most of the questions in advance of an training session. However, it is easy to use the SRS for some on-the-fly related activities, e.g. if a discussion is about to start inside the student group. It is according to the experienced obtained inside Vitec, smart to “use open questions that may stimulate active participation, and a free and open discussion inside the student group”. The questions must not be formulated in such a way that it ends up with a “forced discussion inside the student group”.

Figure 1: Student’s use of SRS in instruction (one loop) and peer-instruction (two loops). The question is posed on the blackboard. After the first or second loop the teacher explains why the correct alternative is correct, and why the incorrect alternatives are incorrect. It is according to experienced obtained by Vitec not necessary to spend a long time on crating questions. What is more important is “to make up a group of 2-3 teachers that may discuss and exchange experiences towards professional issues”. The teachers them self must be responsible for obtaining their own experiences such that they may raise their own expertise, whereby they may use it more frequently. The SRS system is from a technical point of view, easy to use.

Vitec will start using the final SRS version in 2011. This version has been designed for use of peer instruction processes. In such a process, figure 1

The teacher present a question on the blackboard and go through the alternatives

54

The teacher let the students think for a period (ca. 1-2 minutes) before each of them vote

Then the students discuss in group, before each of them vote once more The results are displayed on the blackboard

In this way the instructor may measure the impact of learning instantaneously.

In 2011 Vitec suggest using the system to “check if the information the students has got, actually is understood”. The key question for a industrial production company that produces high tech product is indeed “How do we know that the students actually know?”

6.3 External testing of SRS at CFL (Centrum for Flexible Learning) in Sweden The evaluation from CFL consists of a written description of the experiences in relation to the SRS from the responsible teacher for training within the institution:

To use the SRS in connection with the simulator tool gives advantages in the learning process. If we start with the form of learning the students are using computers and the Internet for the tool and they just add the SRS web site. It means that no extra time is needed to access the SRS, while the ability to user SRS is ever present. Also the possibility to use pre-planned questions mixed with spontaneous asked question gives a flexibility to shift focus from the technology to pedagogy. In our experience we used spontaneous asked question more and more during the teaching and the purpose was to check if all the students was ready to go to the new level.

6.3.1 General reflection from the teachers: - It is simple to give feedback because of the approach - not ashamed of given the

wrong answer.- Easer to work with motivation and reflection, the students can reflect on their own

learning.- Simple - the technology works - Easier when the computers are already running

6.3.2 To think about before using the SRS:- Projector in the classroom - There is access to the internet via computer or telephone - It is an advantage if the computers are already part of the teaching

6.3.3 Teacher preparation:- A database of good questions, about 100 questions that gradually can be improved - Provides a basis for teacher self-reflection, what I say and why for planning the next

lesson - In forehand, learn to manage technology

6.3.4 What is happening during the teaching?- Sharpens learning, motivation and reflection - Becomes an integrated part of teaching. - Provides a feeling that "I learned it." - Becomes clearer - could - could not – explanation

55

- Direct feedback - formative assessment - Commitment - motivation - the sensors are on even outside the classroom.

6.3.5 Long term effects:- Teaching will be more reflective – teachers will be looking at their own teaching. - The students will get higher grades due to reflective teaching - More confident students – they can repeatedly see what they learnt

6.4. External testing of SRS in education of welding engineers in SwedenWeld on Sweden has used the SRS as a regular part of their training over a period of 8 month in the spring of 2010 and fall of 2010. The system has been implemented in the course International Welded Structures Designer (IWSD). This course is modular, consisting of a total of seven modules, and runs over ca 250 hours. The Student response system (SRS) is used to set multiple choice quizzes for each topic in the various modules. The modules are taught in different parts of Sweden, with the main part of the course taking place in Växjö. There are three teachers involved in using the SRS: Kenneth Håkansson, Karl Erik Olsson and Zuzeir Barsoum. The number of students who have followed the course so far is between 6 and 8. 6.4.1 Training in the use of SRS Before the course started, two HiST representatives went to Vaxsjö to give the course organizer Ali Bahrami technical and methodological training in the use the response system. Mr Bahrami would then train the teachers who would use the SRS for the planned courses. For this purpose, Weld on Sweden was provided by HiST with a PC with preinstalled response system; a router; 12 iPods, and chargers.

6.4.2 Learning experiences with the use of SRS All in all, the teachers are left with a good impression of the response system. For these teachers, the SRS is primarily a tool to measure students' knowledge. By using the system, they examined whether the students keep up with the teaching. The teachers present the material as normal and every now and then they run a vote to check whether students follow. All students participate by answering the various questions, which according to the teachers, results in a strong commitment among the students. In addition to being a tool for measuring knowledge, the response system is emphasized as a tool to create more class discussions. The classes are run as follows: they have about three quiz questions per module, the questions being prepared by the teacher. The students work individually with the quiz questions for a couple of minutes, and then they vote. Afterwards they discuss the results together as a group. In other words, the SRS is used a tool to create and stimulate group discussions. According to the teachers, the questions get very different response distributions, and therefore create good class discussions, with both students and teachers participating. Students give their opinions, reason with each other, and try to reach a common understanding and agreement. For the teachers, the students' involvement in the aftermath of the vote is without a doubt one of the main effects of the use of SRS. Because the students now participate in discussions and thus work more actively with the curriculum, they have, according to the teachers, a unique opportunity to learn more.

56

6.4.3 Student evaluation: The student evaluation shows that the students have a positive impression of the response system. They think it's fun to follow courses in which the SRS is used, and most of them report increased interest and a better experience of learning when the SRS is used. Students are very clear on that the teacher’s explanation after the vote is very important, and many believe that the SRS should be used in all courses. Students have some mixed experiences when it comes to the importance of anonymity SRS aspect of the SRS. While some students think it is important, others see it as having little or no significance. Students also have different impressions of whether the system was seen as an integral part of teaching and whether the use of the SRS came at the expense of ordinary teaching. The evaluation was conducted in August 2010. Seven students participated in the evaluation (six males and one female).

6.4.4 Graphs from questionnaire:1. Gender:

0

1

2

3

4

5

6

Male Female

2. My first impression of the student response system is:

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5


57

3. It is fun to at lectures where SRS is used:

0

1

2

3

4

5

6

I totally agree I agree Neutral I slightlydisagree


4. What voting device did you use?

0

1

2

3

4

5

6

7

I Podprovided by

teacher

Private i Pod Private cellphone

Pc Other device

58

5. It was easy to use the voting device:

0

0,5

1

1,5

2

2,5

3

3,5

4


I totallydisagree

6. How was the introduction of the SRS before it was used in the classroom?

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5

Excellent Very good Satisfactory Slightlyinsufficient

Insufficient

59


0

1

2

3

4

5

6

Way to difficult Difficult Satisfactory Easy Way to easy

8. To what extent do you think the system, at is has been used, integrates as a natural part of the training session?

0

0,5

1

1,5

2

2,5

3

3,5

4


60


0

0,5

1

1,5

2

2,5

3

Verycomfortable

Slightlycomfortable

Comfortable Slightlyuncomfortable

Veryuncomfortable


0

1

2

3

4

5

6

7


61

11. The response system collects anonymous responses, how important is that for your participation during training time?

0

0,2

0,4

0,6

0,8

1

1,2

1,4

1,6

1,8

2



0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5


62

13. How would you grade the importance of the teacher spending time on explaining whether the alternatives are right or wrong, and why?

0

0,5

1

1,5

2

2,5

3

3,5

4

Extremelyimportant

Important Neutral Not that important Waste of time

14. The use of SRS compromises ordinary training time:

0

0,2

0,4

0,6

0,8

1

1,2

1,4

1,6

1,8

2


63

15. SRS should be used in all classes:

0

0,5

1

1,5

2

2,5

3

3,5

4

I totally agree I agree Neutral I slightly disagree I totally disagree

64

7. Conclusion for testing in Norway and SwedenThe results obtained in undergraduate engineering programs in Norway and Sweden during testing the prototype solution for a new type of Student Response Services (SRS) for next generation mobile handheld devices with pressure sensitive screens, e.g. like iPod Touch, iPhone and mobile phones, is reported. The teacher collects and visualizes the responses from class at a the digital blackboard or the PC screen, by utilizing state of the art SRS decision process solutions consisiting of a controll interface and mobile devices which the students may use for polling. Our results point out that the students appreciate attributes like feedback on learning. Furthermore, the students commonly appreciate increased involvement and more peer learning through group discussions. SRS rather than raising their hands let individual responses stay confidential. The open SRS has been tested and used in classes with from 7 to 208 students in Norway and Sweden. The experiences and feedback we have obtained is to a large extent independent of the size of the group.

The teacher lead SRS knowledge cycle contains 4 basic elements: The teacher provides a question, task or problem to activate the class and assess

student achievement through progress monitoring The students use their SRS to give anonymous feedback from the whole class to the

teacher The teacher and the class get a knowledge map with the results displayed

instantaneously at the Smartboard or projector surface The teacher must, based up on the results, decide which type of feedback can be

provided on the fly to the class. This may include a peer instruction process where group of students discuss and a re-voting is carried out.

The students provide positive feedback with respect to increased engagement and motivation, which is accordance with results reported in the literature. Many students feel it become fun to attend the lectures. They also point out that the SRS has become an integrated part of the teaching practises, since it is intuitive, easy and fast to operate by the teacher. A majority of them feel it leads to improved understanding of the curriculum, though we haven’t any indication if this is the case during the final examination process. Further research is required in order to detect if it is any differences in using SRS with respect to gender.

The new services may extend and replace existing response systems where universities and Vocational Educational Training institutions must buy dedicated and expensive hardware tools, so called "clickers" or electronic voting systems, in order to provide feedback from students during training sessions. The SRS services, which were finalized in December 2010, use the existing wireless network and may run on widely available mobile, wireless multi touch pressure sensitive hand held devices. It is also possible to use it in parallel on PC/laptop. They are constructed for easy integration and use on digital blackboards, as well as to the story telling provided by the teacher. The students use mobile devices like iPod Touch, iPhone or their mobile phone, to interact anonymously with the teacher through online questionnaires and voting sessions.

The services are based up on XML-based standards and web authoring facilities for the contents available on web pages, by providing XML-based universal notation and interface including visualization of scientific and engineering drawings and graphs. The search facilities retrieve the postulates of the instructor through a service-oriented architecture that integrates semantic web into the system for retrieval of information from the knowledge base

65

system. The SRS web-based decision process solution system is open and flexible in order to achieve maximum interoperability.

66

8. Experiences obtained when using SRS and ABT in IWE and IW courses in Slovakia.

Conclusions from Slovak evaluation questionnaires:

In Slovakia we have got good response on both ABT tests by engineers and by TIG welders as well. We will continue it on every welding course and on most TIG courses. New material produced for it was very useful and good adopted by the students. By the teachers we have difficulties especially by preparing uestions, at the end of the course was response very positive.

By SRS system, we have problems with older instructors and teachers. Also preparation of proper questions for use was not easy. Students had no problems with use and have fun of using SRS. We can conclude that we will use it mainly EWE and EWT courses from now.

The use of SRS system on welder level is basic problem due to low level of education of attendees, short courses and fact that attendees do not consider additional methodological attempts positively.

Part A. IWE course (English / Slovak)

1. Gender / účastníci

- Male / Muži 30- Female / Ženy 0

2. Age: average 32/priemer 32

3. My first impression of the student response system (SRS) is / Moje prvé dojmy o systéme SRS sú:

- Excellent (Výborný)

- Very good (Veľmi dobrý)

- Satisfactory(Dostatočný)

- Not very good(slabý)

67

4. Its fun to be at lectures where the SRS is used / Je to zábava byť na prednáške , kde sa používal SRS?

- I totally agree(Úplne súhlasím)

- I agree(Súhlasím)

- Neutral(Srednje)

- I slightly disagree(Čiastočne nesúhlasím)

- Sorry I fell asleep…(Ľutujem nesúhlasím)

5. What voting device did you use? / Aké zariadenie ste použili?

- I Pod provided by teacher(I Pod od prednášateľa)

- Private i Pod(vlastný I Pod)

- Private cell phone(Vlastný telefón)

- Pc(Počítač)

- Other device(Iné zariadenie)

6. It was easy to use the voting device / Bolo ľahké použiť hlasovacie zariadenie?


- I agree(Súhlasím)

- Neutral(V zásade áno)


I totally agree

I agree Neutral I slightly disagree

Sorry I fell asleep…

0

2

4

6

8

10

12

14

16

Je to zábava byť na prednáške , kde sa používal SRS?

68

7. How was the introduction of the SRS, before it was used in the classroom? / Ako bol SRS vysvetlený pred použitím?

- Excellent(Výborne)

- Very good (Veľmi dobre)

- Satisfactory(dostatočne)

- Slightly insufficient(čiastočne nedostatoče)

- Insufficient(nedostatočne)

8. How would you grade the questions used with the SRS? / Ako hodnotíte otázky použité SRS?

- Way to difficult (Veľmi ťažké)

- Difficult(ťažké)

- Satisfactory(dostatočné)

- Easy(ľahké)

- Way to easy(veľmi ľahké)

9. To what extent do you think the system, as it has been used, integrate as a natural part of the training session? / Ako bol systém, ktorý sa použil, zapracovaný do prednášky?

- Excellent (výborne)

- Very good(veľmi dobre)


- Not very good

69

10. Does the teacher seem comfortable using the system? / Ako sa vám zdal prednášateľ pri použití systému?

- Very comfortable (veľmi zručne)

- Slightly comfortable(celkom zručne)

- Comfortable(zručne)

- Slightly uncomfortable(s problémami zručnosti)

- Very uncomfortable(úplne nezručne)

11. To what extent do you feel that the SRS engage and activate you during training? / Do akej miery sa cítite byť pomocou SRS zapojený a aktívny počas prednášky?

- Very large (Veľmi veľa)

- Large(veľmi)

- Neutral(stredne)

- Small(málo)

- Poor(veľmi málo)

12. The SRS collects anonymous responses, how important is that for your participation during training time? / SRS zbiera informácie anonymne, je to pre vás pri prednáške dôležité?

- very large(veľmi vysoko)

- large(veľmi)

- neutral(stredne)

70

14. To what extent do you think the SRS can aid your learning of the course curriculum? / Do akého rozsahu si myslíte, že SRS zlepší osnovu vzdelávacieho kurzu?

- Very large(veľmi veľký)

- Large(veľký)

- Neutral(stredný)

- Small(malý)

- Poor(veľmi malý)

15. How would you grade the importance of the teacher spending time on explaining whether the alternatives are right or wrong, and why? Ako posúdite význam času, ktorý prednášateľ spotrebuje na vysvetlenie alternatív, ktoré sú správne a ktoré nesprávne?

- Extremely important(Veľmi dôležité)

- Important(Dôležité)

- Neutral(Neutrálne)

- Not that important(Nie veľmi dôležité)

- Waste of time(Strata času)

16. The use of SRS compromises the ordinary training time? /Skracuje SRS celkový vzdelávací čas?

- Very large(veľmi veľa)

- Large(veľa)

- Neutral(stredne)

71

17. SRS should be used in all classes/lectures! / Mal by byť SRS použitý na všetkých prednáškach?


- I agree(súhlasím)

- Neutral(Neutrálne)


- I totally disagree(Úplne nesúhlasím)

ABT Evaluation on IWE course

1. What is your opinion of the level of requirements of the course? / Aký je váš názor na požiadavky kurzu?

- Much too hard (Veľmi ťažký)

- Hard(ťažký)

- Not hard(nie je ťažký)

- Easy(ľahký)

- Very easy(veľmi ľahký)

2. To what extent are you satisfied with the knowledge acquired? / Ako ste spokojný so získanými vedomosťami?

- Very satisfied(Veľmi spokojný)

- Satisfied(Spokojný)

- Fairly satisfied(Takmer spokojný)

- Not satisfied

72

3. What do you think about the possibilities of utilization of this knowledge in your field of activity? /Čo si myslíte o možnostiach využitia poznatkov vo vašej oblasti pôsobenia?

- Very Usefull(veľmi užitočný)

- Usefull(užitočné)

- Neutral(neutrálne)

- Fairly usefull(skoro užitočné)

- Not usefull(neužitočné)

4. Evaluate, please the lecture notes and training aids. / Prosím posúďte poznámky prednášky a prínos vzdelávania.

- Very good(veľmi dobré)

- Good(dobré)

- Neutral(stredné)

- Fairly good(skoro dobré)

- Not good(zlé)

5. Evaluate, please the equipment of the schoolroom and the workshop. / Posúďte zariadenia učebne a kurz.


- Good (dobré)

- Neutral(stredné)


- Not good(zlé)

73

6. To what extent are you satisfied with the proportions of the theoretical and practical parts of the course? / Do akej miery ste spokojný s rozsahom teoretickej a praktickej časti kurzu?

- Very satisfied(veľmi spokojný)

- Satisfied(spokojný)

- Neutral(stredný)

- Fairly satisfied(čiastočne spokojný)

- Not satisfied(nespokojný)

7. Your evaluation of the course altogether. / Aké je celkové hodnotenie kurzu?

- Very good(veľmi dobrý

- Good(dobrý)

- Neutral(stredný)

8. Are you satisfied with the level of additional services? / Ako ste spokojný s úrovňou ďalších služieb?

- Very satisfied (Veľmi spokojný)

- Satisfied(Spokojný)

- Neutral(Neutrálny)

- Fairly satisfied(Čiastočne spokojný)

- Not satisfied(Nespokojný)

74

9. FINAL EVALUATION REPORT IN SLOVAKIA (Tig welding):

Part A. (English / Slovak)

1. Gender / spol

- Male / Muži 20 persons- Female / Ženy 0 persons

2. Age / vek: average/priemer 20

3. My first impression of the student response system (SRS) is / Môj prvý dojem zo systému SRS:

- Excellent (výborný)

- Very good (veľmi dobrý)

- Satisfactory(dobrý)

- Not very good(nie veľmi dobrý)

- Bad(zlý)

4. Its fun to be at lectures where the SRS is used / Je to zábava, keď sa používal SRS systém?




- I slightly disagree(čiastočne nesúhlasím)

- Sorry I fell asleep…(prosím zaspal som)

75

5. What voting device did you use? /Ktoré hlasovacie zariadenie ste použili?

- I Pod provided by teacher(I Pod od prednášateľa)

- Private i Pod(vlastný I Pod)

- Private cell phone(súkromný telefón)

- Pc(počítač)

- Other device(iné zariadenie)

6. It was easy to use the voting device / So zariadením sa hlasovalo ľahko?

- I totally agree(Áno, úplne súhlasím)



- I slightly disagree(čiastočne nesúhlasím)

- I totally disagree(úplne nesúhlasím)

7. How was the introduction of the SRS, before it was used in the classroom? / Aký bol úvod o SRS, predtým než ste ho použili v triede?

- Excellent (Výborný)

- Very good(veľmi dobrý)

- Satisfactory(dostatočný)

- Slightly insufficient

76

(čiastočne nedostatočný)

8. How would you grade the questions used with the SRS? / Ako hodnotíte stupeň otázok použitých pri SRS?

- Way to difficult(Veľmi náročné)

- Difficult(ťažké)

- Satisfactory(dostatočné)

- Easy(ľahké)

- Way to easy(veľmi ľahké)

9. To what extent do you think the system, as it has been used, integrate as a natural part of the training session? /Do akého rozsahu si myslíte, že použitý systém bol integrovaný pri vzdelávaní?

- Excellent(výborne)



- Not very good(nie veľmi dobre)

- Bad(zle)

10. Does the teacher seem comfortable using the system? / Zvládol prednášateľ použitie systému?

- Very comfortable (výborne)

- Slightly comfortable(veľmi dobre)

- Comfortable(dobre)

77

- Slightly uncomfortable11. To what extent do you feel that the SRS engage and activate you during training? Do akej miery vás SRS pri vzdelávaní motivoval a zaktivizoval?


- Large(veľmi)


- Small(málo)

- Poor(veľmi málo)

12. The SRS collects anonymous responses, how important is that for your participation during training time? / SRS zbiera anonymné odpovede, ako to ovplyvnilo vašu účasť počas vzdelávania?

- very large(veľmi veľa)

- large(veľmi)

- neutral(stredne)

- small(málo)

- poor(veľmi málo)

14. To what extent do you think the SRS can aid your learning of the course curriculum? / Do akej miery môže SRS pomôcť obsahu vzdelávacieho kurzu?


- Large(veľmi)

- Neutral(stredne)

- Small(málo)

78

15. How would you grade the importance of the teacher spending time on explaining whether the alternatives are right or wrong, and why? /Ako by ste vyhodnotili význam času prednášateľa, ktorý strávil pri vysvetľovaní či sú alternatívy správne alebo nesprávne a prečo?

- Extremely important (veľmi dôležité)

- Important(dôležité)

- Neutral(stredne dôležité)

- Not that important(nie celkom dôležité)

- Waste of time(strata času)

16. The use of SRS compromises the ordinary training time? Skracuje použitie SRS čas vzdelávania ?


- Large(veľmi)

- Neutral(stredne)

- Small(málo)

17. SRS should be used in all classes/lectures! / Mal by byť SRS používaný pri všetkých prednáškach?

- I totally agree(Áno úplne súhlasím)




79

ABT Evaluation

9. What is your opinion of the level of requirements of the course? / Aký je váš názor na úroveň požiadaviek kurzu?

- Much too hard (veľmi ťažký)

- Hard(ťažký)

- Not hard(stredný)

- Easy(ľahký)

- Very easy(veľmi ľahký)

10.To what extent are you satisfied with the knowledge acquired? / Do akého rozsahu ste spokojný s požiadavkami na vedomosti?



- Fairly satisfied(čiastočne spokojný)


- Waste of time(Strata času)

11.What do you think about the possibilities of utilization of this knowledge in your field of activity? / Čo si myslíte o možnosti využitia vedomostí vo vašej oblasti pôsobenia?

- Very Usefull(veľmi užitočné)

- Usefull(užitočné)

- Neutral(stredne užitočné)

- Fairly usefull(čiastočne neužitočné)

- Not usefull

80

12. Evaluate, please the lecture notes and training aids. / Prosím posúďte poznámky prednášky a prínos vzdelávania.

- Very good(veľmi dobrý)

- Good(dobrý)

- Neutral(stredne dobrý)

- Fairly good(skoro dobrý)

- Not good(zlý)

13. Evaluate, please the equipment of the schoolroom and the workshop. / Prosím ohodnoďte zariadenie školiaceho pracoviska a workshopu.


- Good(dobré)

- Neutral(stredne dobré)


- Not good(zlé)

14.To what extent are you satisfied with the proportions of the theoretical and practical parts of the course? / V akom roszahu ste spokojný s rozsahom teoretickej a praktickej časti kurzu?



- Neutral(neutrálny)

- Fairly satisfied(čiastočne nespokojný)


81

15. Your evaluation of the course altogether. / Ako hodnotíte kurz celkove?


- Good(dobre)

- Neutral(stredne dobre)

- Fairly good(skoro dobre)

- Not good(zle)

16.Are you satisfied with the level of additional services? / Ste spokojný s úrovňou doplnkových služieb?



- Neutral(stredne spokojný)

- Fairly satisfied(čiastočne nespokojný)

- Not satisfied(nie som spokojný)

82

10. Experiences obtained when using SRS and ABT in IWE and IW courses in Slovenia.

Conclusions from Slovenian evaluation questionnaires:

In Slovenia we have got very fine response on both ABT tests by engineers and by TIG welders as well. We will continue it on every EWE course and by most TIG courses. New material produced for it was very welcome and good adopted by the students. By the teachers we have difficulties especially by preparing the material, at the end of the course was response excellent.

By SRS system, we have thought that we will have great problems with older professors from the universities, which was not the case. On teach the teachers course there was 16 attendees from 24 invited. Greatest problem was by preparing proper questions for use of it so staff from Welding institute has to help it. Students on engineer level had not any problem with use and have lot of fun with it. We can conclude that we will use it on all EWE courses from now.

With use of SRS system on welder level we have problems with use because of: low level of education of attendees, short courses and fact that 80% of attendees are not Slovenian and too big language problems ti give fast responses.

Part A. IWE course (English / Slovenian)

1. Gender / spol

- Male / Moški 17- Female / Ženski 2

2. Age: average 35

3. My first impression of the student response system (SRS) is / Moji prvi vtisi SRS sistema so:

- Excellent (Odlični)

- Very good (Zelo dobri)

- Satisfactory(Zadovoljivi)

- Not very good(Ne preveč dobri)

- Bad(Slabi)

My first impression of the student response system (SRS) is:

01234567

Excellent Very good Satisfactory Fairly good Poor

83

4. Its fun to be at lectures where the SRS is used / Ali vam je bilo zabavno prisostvovati na predavanjih, kjer se je uporabljal SRS sistem?

- I totally agree(Se popolnoma strinjam)

- I agree(Se strinjam)

- Neutral(Srednje)

- I slightly disagree(Se rahlo ne srtinjam)

5. What voting device did you use? / Katero napravo za glasovanje ste uporabljali?

- I Pod provided by teacher(I Pod s predavanj)

- Private i Pod(osebni I Pod)

- Private cell phone(Osebni telefon)

- Pc(Računalnik)

- Other device(Druge naprave)

6. It was easy to use the voting device / Napravo za glasovanje je bilo lahko uporabljati:


- I agree(Sesrtinjam)

- Neutral(Srednje)

- I slightly disagree(Se rahlo ne strinjam)

- I totally disagree(Se popolnoma ne strinjam)

Its fun to be at lectures where the SRS is used

012345678

I totallyagree

I agree neutral I slightlydisagree



012345678

I totallyagree



It was easy to use the voting device?

0

2

4

6

8

10

I totallyagree


I totalydisagree

84

7. How was the introduction of the SRS, before it was used in the classroom? / Kakšna je bila uvodna predstavitev SRS sistema, preden ste jo dejansko uporabili?

- Excellent(Odlična)

- Very good(zelo dobra)

- Satisfactory(zadovoljiva)

- Slightly insufficient(Rahlo nezadostna)

8. How would you grade the questions used with the SRS? / Kako bi ocenili vprašanje postavljena pri SRS?

- Way to difficult(Pretežka)

- Difficult(Težka)

- Satisfactory(Zadovoljiva)

- Easy(Lahka)

- Way to easy(Prelahka)

9. To what extent do you think the system, as it has been used, integrate as a natural part of the training session? / V kolikšni meri menite da se lahko uporabljen sistem vključi kot del usposabljanja?

- Excellent(Odlično)

- Very good(Zelo dobro)

- Satisfactory(Zadovoljivo)

- Not very good(Ne preveč dobro)

- Bad(Slabo)

How was the introduction of the SRS, before it was used in the classroom?

012345678

Excellent v ery good Satisf actiory Slightlyinsuf f icient

Insuf f icient

How would you grade the questions used with the SRS?

0

2

4

6

8

10

Way todif f icult

Dif f icult Satisf actory Easy Way to easy

To what extent do you think the system, as it has been used,

integrate as a natural part of the training session?

0

2

4

6

8

Excellent Very good Satisfactory Not verygood

Bad

85

10. Does the teacher seem comfortable using the system? / Ali je bil po vaši oceni predavatelj vešč med uporabo sistema?

- Very comfortable(Zelo)

- Slightly comfortable(Rahlo)

- Comfortable(Vešč)

- Slightly uncomfortable(Rahlo nevešč)

- Very uncomfortable(Zelo nevešč)

11. To what extent do you feel that the SRS engage and activate you during training? / V kolikšni meri vas je po vašem mnenju SRS system motiviral med predavanji?

- Very large(Zelo veliki)

- Large(Veliki)

- Neutral(Srednje)

- Small(malo)

- Poor(Zelo malo)

12. The SRS collects anonymous responses, how important is that for your participation during training time? / V kolikšni meri je za vas pomembno to, da je sodelovanje po sistemu SRS anonimno?

- very large(Zelo veliki)

- large(Veliki)

- neutral(Srednje)

- small(majhni)

Does the teacher seem comfortable using the system?

01

234

567

89

Verycomf ortable

Slightlycomf ortable

comf ortable slightlyuncomf ortable

Veryuncomf ortable

To what extend do you feel that the SRS engage and activate you during training?

0123456789


The SRS collects anonymous responses, how important is that for your participation


012345678


86

- poor(Zelo majhni)

14. To what extent do you think the SRS can aid your learning of the course curriculum? / V kolikšni meri vas po vašem mnenju SRS system usmerja med usposabljanjem?


- Large(Veliki)

- Neutral(Srednji)

- Small15. How would you grade the importance of the teacher spending time on explaining whether the alternatives are right or wrong, and why? / Kako bi ocenili pomembnost časa, ki ga porabi predavatelj za razlago nepravilnosti in zakaj?

- Extremely important(Zelo pomembno)

- Important(pomembno)

- Neutral(Srednje)

- Not that important(Ni tako pomembno)

- Waste of time(Zguba časa)

16. The use of SRS compromises the ordinary training time? /

- Very large(Zelo)

- Large(Veliko)

- Neutral(Srednje)

- Small(Malo)

- Poor(Zelo malo)

To what extent do you think the SRS can aid your learning of the course curriculum?

0

1

2

3

4

5

6


How would you grade the importance of the teacher spending time on

explaining whether the alternatives are right or wrong, and why?

0

1

2

3

4

5

6

Extremelyimportant


Waste of time

The use of SRS compromises the ordinary training time?

01234567


87

17. SRS should be used in all classes/lectures! / Uporaba SRS sistema se bi morala uporabljati pri vseh predavanjih!



- Neutral(Srednje)

- I slightly disagree(Rahlo se ne strinjam)

SRS should be used in all classes/lectures

0

1

2

3

4

5

6


I totallydisagree

88

ABT Evaluation on IWE course

17.What is your opinion of the level of requirements of the course? / Kakšno je vaše mnenje o zahtevnosti izobraževanja?

- Much too hard (Pretežko)

- Hard(Težko)

- Not hard(Ni težko)

- Easy(Lahko)

- Very easy(Zelo lahko)

18.To what extent are you satisfied with the knowledge acquired? / V kolikšni meri ste zadovoljni s pridobljenim znanjem?

- Very satisfied(Zelo zadovoljen)

- Satisfied(Zadovoljen)

- Fairly satisfied(Komaj zadovoljen)

- Not satisfied(Nisem zadovoljen)


19.What do you think about the possibilities of utilization of this knowledge in your field of activity? / Kaj menite o možnosti uporabe tega znanja v svojem področju delovanja?

- Very Usefull(Zelo uporabno)

- Usefull(Uporabno)

- Neutral(Srednje)

- Fairly usefull(komaj uporabno)

- Not usefull(Ni uporabno)

What is your opinion of the level of requirements of the course?

0

2

4

6

8

Much toohard

Hard Not hard Easy Very easy

To what extent are you satisfied with the knowledge acquired?

0

2

4

6

8

Verysatisf ied

Satisfied Fairlysatisf ied

Notsatisf ied

Waste oftime

What do you think about the possibilities of utilization of this knowledge in your field of

activity?

0

2

4

6

Very usefull Usefull Neutral Fairlyusefull

Not usefull

89

20. Evaluate, please the lecture notes and training aids. / Prosimo ocenite skripto in učne pripomočke.


- Good(Dobro)

- Neutral(Srednje)

- Fairly good(komaj dobro)

- Not good(Ni dobro)

21. Evaluate, please the equipment of the schoolroom and the workshop. / Prosimo ocenite opremo, učilnico in delavnico.


- Good(Dobro)

- Neutral(Srednje)



22.To what extent are you satisfied with the proportions of the theoretical and practical parts of the course? / V kolikšni meri ste zadovoljni z razmerjem teoretičnega in praktična dela tečaja?



- Neutral(Srednje)



Evaluate, please the lecture notes and training aids.

0123456789

Very good Good Neutral Fairly good Not god

Evaluate, please the equipment of the schoolroom and the workshop.

01234567


To what extent are you satisfied with the proportions of the theoretical and practical parts

of the course?

01234567

Verysatisf ied

Satisf ied Neutral Fairlysatisf ied

Not satisf ied

90

23. Your evaluation of the course altogether. / Kakšna je vaša kompletna ocena tečaja?


- Good(Dobro)

- Neutral(Srednje)



24.Are you satisfied with the level of additional services? / Ali ste zadovoljni z obsegom dodatnih storitev?



- Neutral(Srednje)



Your evaluation of the course altogether.

0

2

4

6

8

10


Are you satisfied with the level of additional services?

0

2

4

6

8

10

Verysatisf ied

Satisfied Neutral Fairlysatisf ied

Notsatisf ied

91

11. FINAL EVALUATION REPORT FOR SLOVENIA (Tig welding)

Date: 15.11.2010

Part A. (English / Slovenian)

1. Gender / spol

- Male / Moški 10 persons- Female / Ženski 0 persons

2. Age / Starost: average 34

3. My first impression of the student response system (SRS) is / Moji prvi vtisi SRS sistema so:

- Excellent (Odlični)

- Very good (Zelo dobri)

- Satisfactory(Zadovoljivi)

- Not very good(Ne preveč dobri)

- Bad(Slabi)

4. Its fun to be at lectures where the SRS is used / Ali vam je bilo zabavno prisostvovati na predavanjih, kjer se je uporabljal SRS sistem?



- Neutral(Srednje)

- I slightly disagree(Se rahlo ne srtinjam)

My first impression of the student response system (SRS) is:

0

1

2

3

4

Excellent Very good Satisfactory Fairly good Poor


0

1

2

3

4

I totallyagree



92

- Sorry I fell asleep…(Oprostite, spi se mi…)

5. What voting device did you use? / Katero napravo za glasovanje ste uporabljali?

- I Pod provided by teacher(I Pod s predavanj)

- Private i Pod(osebni I Pod)

- Private cell phone(Osebni telefon)

- Pc(Računalnik)

- Other device6. It was easy to use the voting device / Napravo za glasovanje je bilo lahko uporabljati:


- I agree(Se srtinjam)

- Neutral(Srednje)

- I slightly disagree(Se rahlo ne strinjam)

- I totally disagree(Se ne strinjam)

7. How was the introduction of the SRS, before it was used in the classroom? / Kakšna je bila uvodna predstavitev SRS sistema, preden ste jo dejansko uporabili?

- Excellent(Odlična)

- Very good(zelo dobra)

- Satisfactory(zadovoljiva)

What voting device did you use?

0

2

4

6

8

10

I podprov ided by

teacher

Priv ate I pod Priv ate cellphone

PC Other dev ice


0

1

2

3

4

I totallyagree


I totalydisagree


0

1

2

3

4

I totallyagree


I totalydisagree93

- Slightly insufficient(Rahlo nezadostna)

- Insufficient(nezadostna)

8. How would you grade the questions used with the SRS? / Kako bi ocenili vprašanje postavljena pri SRS?

- Way to difficult(Pretežka)

- Difficult(Težka)

- Satisfactory(Zadovoljiva)

- Easy(Lahka)

9. To what extent do you think the system, as it has been used, integrate as a natural part of the training session? / V kolikšni meri menite da se lahko uporabljen sistem vključi kot del usposabljanja?

- Excellent(Odlično)


- Satisfactory(Zadovoljivo)

- Not very good(Ne preveč dobro)

- Bad(Slabo)

10. Does the teacher seem comfortable using the system? / Ali je bil po vaši oceni predavatelj vešč med uporabo sistema?

- Very comfortable(Zelo)

- Slightly comfortable(Rahlo)

- Comfortable(Vešč)

- Slightly uncomfortable

How would you grade the questions used with the SRS?

00,5

11,5

22,5

33,5

Way todif f icult

Dif f icult Satisf actory Easy Way to easy

To what extent do you think the system, as it has been used,

integrate as a natural part of the training session?

0

1

2

3

Excellent Very good Satisfactory Not verygood

Bad

Does the teacher seem comfortable using the system?

0

1

2

3

4

5

Verycomf ortable

Slightlycomf ortable

comf ortable slightlyuncomf ortable

Veryuncomf ortable94

(Rahlo nevešč)

- Very uncomfortable(Zelo nevešč)

11. To what extent do you feel that the SRS engage and activate you during training? / V kolikšni meri vas je po vašem mnenju SRS system motiviral med predavanji?


- Large(Veliki)

- Neutral(Srednje)

- Small(malo)

12. The SRS collects anonymous responses, how important is that for your participation during training time? / V kolikšni meri je za vas pomembno to, da je sodelovanje po sistemu SRS anonimno?

- very large(Zelo veliki)

- large(Veliki)

- neutral(Srednje)

- small(majhni)

- poor(Zelo majhni)

14. To what extent do you think the SRS can aid your learning of the course curriculum? / V kolikšni meri vas po vašem mnenju SRS system usmerja med usposabljanjem?


- Large(Veliki)

- Neutral(Srednji)

- Small

To what extend do you feel that the SRS engage and activate you during training?

0

1

2

3

4


The SRS collects anonymous responses, how important is that for your participation


0

1

2

3

4

5


To what extent do you think the SRS can aid your learning of the course curriculum?

0

1

2

3

4


95

(Majhni)

- Poor(Zelo majhni)

15. How would you grade the importance of the teacher spending time on explaining whether the alternatives are right or wrong, and why? / Kako bi ocenili pomembnost časa, ki ga porabi predavatelj za razlago nepravilnosti in zakaj?

- Extremely important(Zelo pomembno)

- Important(pomembno)

- Neutral(Srednje)

- Not that important(Ni tako pomembno)

16. The use of SRS compromises the ordinary training time? /

- Very large(Zelo)

- Large(Veliko)

- Neutral(Srednje)

- Small(Malo)

- Poor(Zelo malo)

17. SRS should be used in all classes/lectures! / Uporaba SRS sistema se bi morala uporabljati pri vseh predavanjih!



- Neutral(Srednje)

- I slightly disagree(Rahlo se ne strinjam)

How would you grade the importance of the teacher spending time on

explaining whether the alternatives are right or wrong, and why?

0

1

2

3

4

Extremelyimportant


Waste of time

The use of SRS compromises the ordinary training time?

0

1

2

3

4

5


SRS should be used in all classes/lectures

0

1

2

3

4

5

6


I totallydisagree

96

- I totally disagree(Se ne strinjam)

ABT Evaluation

25.What is your opinion of the level of requirements of the course? / Kakšno je vaše mnenje o zahtevnosti izobraževanja?

- Much too hard (Pretežko)

- Hard(Težko)

- Not hard(Ni težko)

- Easy(Lahko)

- Very easy

26.To what extent are you satisfied with the knowledge acquired? / V kolikšni meri ste zadovoljni s pridobljenim znanjem?






27.What do you think about the possibilities of utilization of this knowledge in your field of activity? / Kaj menite o možnosti uporabe tega znanja v svojem področju delovanja?

- Very Usefull(Zelo uporabno)

- Usefull(Uporabno)

- Neutral(Srednje)

- Fairly usefull(komaj uporabno)

- Not usefull

What is your opinion of the level of requirements of the course?

0

1

2

3

Much toohard

Hard Not hard Easy Very easy

To what extent are you satisfied with the knowledge acquired?

0

1

2

3

4

Verysatisf ied

Satisfied Fairlysatisf ied

Notsatisf ied

Waste oftime

What do you think about the possibilities of utilization of this knowledge in your field of

activity?

0

1

2

3

Very usefull Usefull Neutral Fairlyusefull

Not usefull

97

(Ni uporabno)

28. Evaluate, please the lecture notes and training aids. / Prosimo ocenite skripto in učne pripomočke.


- Good(Dobro)

- Neutral(Srednje)



29. Evaluate, please the equipment of the schoolroom and the workshop. / Prosimo ocenite opremo, učilnico in delavnico.


- Good(Dobro)

- Neutral(Srednje)



30.To what extent are you satisfied with the proportions of the theoretical and practical parts of the course? / V kolikšni meri ste zadovoljni z razmerjem teoretičnega in praktična dela tečaja?



- Neutral(Srednje)


- Not satisfied

Evaluate, please the lecture notes and training aids.

0

1

2

3

4

5


Evaluate, please the equipment of the schoolroom and the workshop.

0

1

2

3

4

5


To what extent are you satisfied with the proportions of the theoretical and practical parts

of the course?

0

1

2

3

4

Verysatisf ied

Satisf ied Neutral Fairlysatisf ied

Not satisf ied

98

(Nisem zadovoljen)

31. Your evaluation of the course altogether. / Kakšna je vaša kompletna ocena tečaja?


- Good(Dobro)

- Neutral(Srednje)


32.Are you satisfied with the level of additional services? / Ali ste zadovoljni z obsegom dodatnih storitev?



- Neutral(Srednje)



Your evaluation of the course altogether.

0

1

2

3

4

5


Are you satisfied with the level of additional services?

0

1

2

3

4

Verysatisf ied

Satisfied Neutral Fairlysatisf ied

Not satisf ied

99

12. Final Evaluation Report from Hungary

Written: 20 October 2010-10-22 Subject: Results of the EduMecca project activity

1. An important aim of the EduMeccca project is to acquaint the project aims and possibilities as widely as possible and to measure the results. Since the project is creating an absolutely new education technological method it is important to have it accepted by a wide range of educators and users. Within the framework of the project the MHTE has organised a number of information meetings and invited some educators who have experience with the usage of digital boards and have applied similar systems in their fields. We organised information meeetings at several places of the country to involve a wide circle of experts int he program. This is how we made testing sin Visonta, Szombathely, Dunaújváros, Székesfehérvár and Budapest and we made demonstration in Pécs. In addition to trainings we gave lectures at several conferences and at international exhibitions of welding technology. We made tesings ten times with more than 200 participant’s altogether. On one occasion made testing sin the course of usage parallel with the Slovakian partner. As a result it could be concluded that the program can be used reliably at the same time.

During the training and information meetings we introduced the participants the structure of the system and we inquired concerning their ideas on possible usage. During the training we demonstrated the usage of the iPod and the connection between the server and the voting system to the participants. We asked them to express their opinion ont o what extent they think that the spread of digital systems at examinations and int he creation of statistic charts makes the work of educators and students easier. The surveys were carried out witht the help of questionaires elaborated together with our partners and interviews. The questions are attached in the enclosure.

The summary of answers given are the following:

The educators found the information available sufficient The statistics work with too simple data but the conclusions adequately help

education Replicants and statistics concerning individuals have to be seperated The closed question groups can be used for certain tasks buti n the forthcoming

phase of developement this can be made task-specific Graphic representation of the results makes their handling more spectacular

and more explicit Graphic representation is useful for demonstrating results The iPod provides many more possibilities for usage than what the project can

use at present so the SRS system has a number of developement potentials The internet database is appropriate therefore the system can be used

everywhere Connection and system maintanance should not be too complicated since

teachers are not IT experts, easy-to-use, userfriendly surfaces are necessary. Elderly users do not prefer small screens as they are hard to see At examination device stability is important, the device cannot go wrong or flat

100

IPods are bound to registration which on one hand is good concerning security, ont he other hand it makes maintenance more complicated

It is good that the SRS program can be connected to the existing and workind computer systems and functioning programs

Summary:

As a conclusion it can be established concerning the application of the iPod system that educators are interested in the potential usage of the equipment, however, its technical background has to be taught thoroughly. They believe that the target of the project is a very important milestone in the creation of the actual usage of the finalised SRS system.

2. During the project several materials to be published have been written. The A/3 and A/4 format posters, which have been created on the basis of the sample created by the Norwegian partner, are ready. They have been photocopied and sent to educational representatives of nearby companies and to some educational institutions. WE have written the guidebook to the first testing version of thr SRS system which we have given to the institutions taking part in the survey. It can be found at the end of the report.

The evaluation of the given answersThe evaluation has two main aims:

Providing continuous feedback on the effectiveness of the learning process; giving basis for teaching-material improvenet and for the settling of the level of

education.

The documentation of the evaluation and the storage of results are done (either digitally or on sheets of paper) according to the local rules. Results can be saved from the system for further discussion and evaluation.

Self- evaluation by the SRSThe e-student evaluates his/her own work and improvement by feedbacks received from the SRS (online evaluation, meeting deadlines, teacher’s evaluation, later on exams etc.).Self-evaluation and self-reflection has a great significance

in maintaining motivation in the definition and correction of an effective learning strategy, in the creation of appropriate learning methods, in gaining self-esteem in the assignment of further fields of improvement, in the shaping of one’s independent initiative skill

Automatic online evaluationThe system is able to evaluate a part of compulsory and practising tasks inbedded in the ABT learning material which makes prompt feedback possible.

Evaluation of the teacher

The evaluation of the teacher is partly connected to the subject: the grading and the evaluation of tasks to be answered and revision tests; and partly learning-supportive: by the examination of subject task items and the basis of the regularity and improvement of the work.

101

Field of application concerning the SRSAccording to Hungarian experience the system can be usefully applied in courses and at any field of higher educational (collage or university) programmes. The MHTE has successfully tested the system under significantly different circumstances.Primarily, the SRS has been introduced in welding training and has also been tested in other educational programmes. It has been applied in management and in language teaching as well as in secondary school programmes. On every occasion experience has pointed forward, difficulties could be solved within a short period of time and a positive, profesionally reliable programme has been created within the framework of lessons.We suggest further improvement and completion of the system on the basis of the demand of specific exam fields. It is proved by statistical evaluation and the representation of diagram results.

102

13. External evaluation from Slovenia

1. PROJECT TITLE

New Educational Models that Encourage Creative transfer of Competence and Acquaintance in LLLifelong Learning Program Sub-programs, Multilateral Projects, Networks, Accompanying measures.Edu MECCA

2. EVALUATOR

Evaluation was provided by Slovenian Welding Society. It is 60 years old non profit professional association on welding and allied processes. The Association bind together education, research and production in field of welding.

3. SUBJECT OF EVALUATION

Subject of evaluation is carrying out external evaluation of the two welding specialist and engineering courses in Slovenia.This includes translation and circulating questionnaires to students and instructors.

4. TERM OF EVALUATION

December 2009 to December 2010

5. OVERWIEV of PLANED ACITIVITIES of PARTNER P5

IzV will contribute towards the adaptation of MECCA skill development methodologies for the Slovenian market. They will translate educational material. Their instructors will participate in the train-the-trainer courses.

They will also have a particular focus towards the skills development for European market through the European Welding Federation umbrella.

They will organize at least one instructor training course in Slovenia

IzV will adapt the MECCA training content and courses for the Slovenian market in collaboration with Smartcom and HiST. VuZ will utilize a new state of the art technical and pedagogical solution during their training session.

IzV will collaborate with a number of the educational centers for metal welding in Slovenia, and offer training to welders by utilizing the new learning environment in combination with ABT, video streaming technology, and “time to think” learning activities.

IzV will carry out the pilot testing in two lifelong sectors in Slovenia: welder specialist training and welding engineer education.

They will also support HiST with questionnaires for the “time-to-think” learning objects.

The task will involve the actual delivery of EduMECCA courses in real-market conditions to external learner groups of welders in Slovenia. The course delivery will be a two-tire implementation

103

This will partly consist of seminars highlighting the structure and methodology obtained through the MECCA project and focusing on the relevant findings for this market

After the first training seminars for the target group teachers and instructors, then the practical courses from this group will be offered and implemented as a second activity for the ultimate target group welder specialists. The starting point will be a pilot course for testing the translated educational material and the ABT methodology. Second a planned educational schedule will be defined together with the teachers and instructors from the first schedule. This schedule will be developed at a national level but with additional coordination on a regional level.

Based on these schedules the actual courses for the welder specialists and welder engineers, they will be offered at national level for the remaining project period to the industry.

With regards to the validation that will take place in Slovenia, the following institutions are potential users: Montavar - Processing equipment, Himomontaza - Mounting on side, TVT Nova - Express trains, Armature - Ball vaves, Meteorit – Supporting structures, Primat - Banknote dispensers, Palfinger, - Vehicle lifts, Arcont – Containers, Metalna SRM, - Cryogenic technique, Tovarna vozil - Special buses, Monter - Supporting structures, Nafta - Petroleum industry, Varstroj - Welding equipment, KAL - Vehicle lifts, Farmatech -Agricultural machines, INOX, Cernelavc - Equipment for food industry

IzV will be responsible of carrying out external evaluation of the two welding specialist and engineering courses in Slovenia. This includes translation and circulating questionnaires to students and instructors.

IzV participates in the development of the Quality Handbook. IzV will disseminate in the Slovenian market, through several publications of project

related-info on their web site, and through the organization of one national seminar. It will further pursue dissemination activities about the project results at Seminars and exhibition where IzV will present them selves.

They will also act as a liaison towards the IIW (International Institute of Welding) system, which covers interest points for national bodies like IzV that certify welders and welder engineers all over the world. As a member in this world wide umbrella organization, IzV will disseminates the project results through the general assembly and through their participation in the relevant work groups for education and training.

6. OUTCOMES

Evaluation is focused on welding specialist and engineering courses in Slovenia. Outcomes are:

Courses curriculum Pedagogical methodology containing Active Based Training (ABT) approach. Education Material Pilot production Courses realization Trainers and mentors instructions Multiple choice questionnaires Evaluation of expected results.

7. SPECIALIST COURSE FOR TIG WELDING

104

Course was carried out in Welding Training Centre in Ljubljana in October 2010.Course was conducted by 3 experienced instructors. Facilities for theoretical and practical training were available. Laboratories for destructive and non destructive testing were used. There are course details:

Course was carried out between 11th October and 22nd of October Course lectures consist of 72 hours of theoretical and practical training Present were 10 participants Presence lists are available 160 pages printed course material was prepared Technological instructions for pilot production were completed Participants fabricated stainless steel vessel. Questionnaires to students and instructors were analyzed.

8. ENGINEERING COURSE

Course was carried out in Authorized Training Body of Slovenian Welding Institute in Ljubljana in spring 2010. Facilities for theoretical training and laboratory work were available. .There are course details:

Course was carried out between 18th of February and 19th of March. Course lectures consist of 92 hours of theoretical training and laboratory work. Individual seminar continued 60 hours. Course visited 20 participants from 13 companies and 1 from research institute Presence lists are available Course was conducted by 8 lecturers and 5 mentors. 250 pages printed course material was prepared Technological instructions for pilot project were completed Questions for students response system (SRS) were prepared Participants prepared technical documentation for production of housing for

transformer. It was used for electric locomotives. The documentation consists of 300 pages printed and drawn material. Questionnaires to students and instructors were analyzed.

9. EDUCATION PROCESS

New knowledge transfer methodology was applied. Active Based Training was used. Students fabricated pilot products and prepared technical documentation for housing of locomotive transformer. New approach is also student response system (SRS).

10. EVALUATION

On base of elaborated project material and observation of project activity in Slovenia we conclude that two courses were conducted in accordance with project proposal instructions.New education methodology was applied. Project results will make possible training centers and VET to introduce new metrology in regular education process.

Slovenian Welding SocietyG. Janez Rihar D. Sc.

Ljubljana, 30th of DNovember 2010

105

14. Summary and conclusion

Internal, external evaluation and evaluation from end users was completed as planned. QA system surveillance was executed as planed. Project timeline was all the time ahead the planed. No complains was reported.

Internal audit by the partners did find that work with experienced project leader as John Birger Stav is key of fluid and succesfull project.

Users of the project outputs were very satisfied with improovment of the IWE and IW courses in all the partner countries (detailed reports are in report text) and results of the project.

Technical results are above planned results.

External evaluators did not find non conformities.

15. References

1. The Mecca project, online at http://histproject.no/node/57 2. Berg, B. L. (2007). Chapter 5: Focus group Interviewing. I: Berg, Bruce L. (2007): Qualitative Research Methods for the Social Sciences, Pearson Education Inc. (s. 144-170).3. Charmaz, K. (2001). Grounded Theory. In: Smith, J. A., Harre, R, Langenhove, L. (eds). Rethinking methods in psychology. London: Sage Publications. (s.27-49). 4. Cozby, P. C. (2003). Methods in Behavioral Research, 8th ed. McGraw-Hill companies 2003. 5. Johannessen, A., Tufte, P. A., og Kristoffersen, L. (2004). Introduksjon til samfunnsvitenskapelige metoder. Abstrakt forlag as 2004.6. Ringdal, K. (2001). Unity and diversity. Social Science Methodology and quantitative research. Fagbokforlaget Vigmostad & Bjørke AS 2001.

106

http://histproject.no/node/57

12. Appendix A

107

histproject.nohistproject.no/sites/histproject.no/files/edumecca... · web viewfrom the student...

Documents