coesp year one report final · the coesp has twitter, facebook and instagram accounts. the most...

The College of Biological Science Office of Educational Scholarship and Practice

at the University of Guelph: Year One Report

June 13, 2017

Director: John Dawson

Prepared for:

Jonathan Newman, Dean, CBS

Brian Husband, Associate Dean Academic, CBS

1

TABLEOFCONTENTS

TABLEOFCONTENTS 1

EXECUTIVESUMMARY 2

ADMINISTRATION 3

CO-OPSTUDENTINVOLVEMENT 3

STRATEGICPLANNING 4

VISION: 4MISSION: 4VALUES: 4COMMENTS: 4NEEDSANDCHALLENGESINBIOLOGYHIGHEREDUCATIONTODAY 5

PROMOTIONOFTHECOESP 6

WEBSITE: 6SOCIALMEDIA: 6PHYSICALMEDIA: 6CONFERENCEANDMEETINGS: 6MEETINGTHENEEDSINCBS 7

SEMINARSANDWORKSHOPS 7RECOGNITIONANDPROMOTION 7INCLUSIVENESS 8CBSTEACHINGDISCUSSIONGROUPS 8FORMINGTHEBIOEDRESEARCHHUB 8FUNDINGSUPPORT 91) COESPCONFERENCETRAVELAWARDS 102) COESPCOURSEIMPROVEMENTGRANTS 103) BIOEDRESEARCHGRANTS 10CONCLUSION 10

OTHERCHALLENGESFACINGEDUCATION 11

ENGAGINGFACULTYINOURTEACHINGMISSION 11LESSONSLEARNED 12

FINANCIALREPORT 13

YEAR2OFTHECOESP 14

IDEASFORYEAR2: 141) CONFERENCEDEBRIEFSESSIONS 142) PRESENTATIONBYDESIGN 153) BOOKCLUB 154) STEMTEACHINGCOURSES 15

CONCLUDINGREMARKS 16

2

EXECUTIVESUMMARY The College of Biological Science Office of Educational Scholarship and Practice (COESP) was initiated in March 2016 with the appointment of its first faculty Director, Professor John Dawson. The COESP is the only education office in Canada that focusses on biology higher education. Its formation happened at a very opportune time, with the realignment and restructuring of higher education at Guelph and in Ontario, demonstrating leadership in the College of Biological Science to address teaching and learning issues. The COESP consists of a Director and a steering committee of members from all segments of CBS. In its first year, the COESP developed its mission, vision and values, holding its value of including all segments of the College as its main guiding value. To achieve its vision, the COESP is being actively promoted on the University of Guelph campus and to the wider biology higher education community beyond Guelph. In its first year, the COESP researched the needs and challenges facing biology higher education today. To address those needs and challenges, the COESP has been tremendously active in organizing a wide variety of events and discussion groups that include all segments of the College. To fulfill part of its mission, the BioEd Research Hub was formed to build a community of faculty active in discipline-based education research. In Winter 2017, the BioEd Research Hub successfully obtained Learning Enhancement Fund support to research best practices for using student evaluation of teaching feedback as a means of improving teaching practice in CBS. To encourage and promote innovation, the COESP launched four awards or grants in its first year:

• The BioEd Graduate Student Research Award • Conference Travel Awards • Course Improvement Grants • BioEd Research Grants

Engagement of faculty in the efforts of the COESP was a challenge in year one. Several factors contribute to varying levels of engagement, including other priorities that are more strongly rewarded through Tenure and Promotion processes. How teaching is recognized in CBS was the theme of a major workshop in Fall 2016; the conclusions of that workshop formed the research program of the BioEd Research Hub and will inform future discussions on how best to evaluate effective teaching. Year Two of the COESP will see a shift from creating our foundation and activities, to growing the community and making an impact. Most of our grants and awards will be administered for their first cycle and we will continue to promote our activities both inside and outside Guelph. The COESP Director will also be on a Study/research Leave for the Winter 2018 semester, providing an opportunity to others to participate in the coordination of COESP activities; thereby building the sustainability of the COESP. We believe Year Two will be the best year yet for the COESP, working from our foundation with the experience and wisdom we have collected in Year One.

John Dawson, Director

3

Administration The COESP was formed in March, 2016 with the appointment of Dr. John Dawson was the first faculty Director. The inaugural steering committee consisted of: Faculty Representatives: Dr. William Bettger (HHNS) Dr. Shoshanah Jacobs (IB) Dr. Steffen Graether (MCB) Staff Representative: Charlene Winchcombe-Forhan Student Representatives: Paisley Worthington (Undergraduate) Tina Hobbins (Graduate) OpenEd Liaison: Dr. Erin Aspenleider That committee continued throughout the first year of the COESP, with the following changes: In Fall 2016, Erin Aspenleider moved to McMaster, and Laura Sloat became the OpenEd Liaison with the steering committee. In Fall 2016, Paisley Worthington completed her undergraduate program, and Phoebe Bruce became the undergraduate representative Co-opStudentInvolvementIn Winter 2017, the COESP took on a full time Marketing Co-op student, Charles Di’Illio. Charles’ tasks were to promote and organize COESP activities in CBS using his experience with running and promoting events in his other co-op positions. Charles participated in COESP steering committee meetings for the Winter 2017 semester.

4

StrategicPlanning The COESP steering committee defined our Vision, Mission and Values upon begin formed so that we could report these elements out to the community in our COESP Kick-Off event in May 2016. The timeline required intense regular meetings where the roles, tactics, and attitudes of the COESP were discussed in detail. Ultimately, the steering committee did a great job: Vision: To be recognized as the national leader in educational scholarship and practice in the biological sciences. Mission:To Support and Promote:

• Evidence-based Teaching Practice • Learning Outcome Development & Assessment • Biology Education Research

Values:The COESP is:

• Inclusive • Accessible • Responsive • Evidence-Based

Comments:The COESP was formed at a very opportune time, with the realignment and restructuring of teaching at Guelph and in Ontario in higher education. Since the COESP was formed, we have seen strategic planning for Teaching and Learning at Guelph initiated by the Provost, strategic planning for CBS initiated by the Dean, emphasis on experiential learning / workforce integrated learning from the Province (resulting in a proposed Experiential Learning Hub being approved by the University of Guelph Senate), growth and restructuring of the Educational Developer Office in OpenEd, inclusion of knowledge translation and transfer components in Federal tri-council granting, and increased accountability for Learning Outcomes at the University of Provincial levels. We hired Tara Allohverdi, a OWSP student, to help search for other education offices with a similar emphasis on biology higher education. The goal of this research was to determine whether 1) such discipline-based higher education offices exist elsewhere, and 2) what effect they have had on their audience. We did not find any offices that focus on biology in Canada. Therefore, the COESP is the national leader in biology higher education. However, our vision states that we want to be recognized as such. In this regard, promotion of the COESP and its mission outside of Guelph becomes part of our work. We have focused on Evidence-based teaching practice and Biology Education Research in our first year. Our role in Learning Outcomes Development and Assessment is embedded in our Evidence-base teaching practice as the COESP would support these activities by providing best practice information and assistance. However, the COESP does not have the authority to enact and assess learning outcomes; our role here is to mainly support instructor activities.

5

The COESP steering committee is very clear in its value of inclusivity of all faculty, staff, and students in CBS. Throughout our strategic discussions, we felt that our faculty is a primary target of COESP activities, since they do the lion’s share of teaching in the College. At the same time, this group is the most challenging to connect with. We also are targeting graduate student and postdocs, as they represent the next generation of faculty and instructors. NeedsandChallengesinBiologyHigherEducationTodayThe COESP gathered opinion from instructors regarding the needs and challenges facing biology education from several sources. See attached Appendix 1. It is clear from our work that student engagement and motivation is the primary challenge facing biology education today. This theme is not unique to biology; therefore, there are opportunities to collaborate with other units and offices to meet these challenges.

6

PromotionoftheCOESPWebsite:A preliminary website was created in Summer 2016. In April 2017, the COESP site was migrated over to the ADA’s site (we retained our original uoguelph.ca/coesp URL with a redirect to the ADA-hosted site). The website was updated to include resources from our events, information and materials for COESP grant applications, and future event reminders. We will now grow the links to internal and external resources that are specific to Biology, tapping into news feeds from CBS Life Science, or external repositories of active learning examples such as case-based learning modules, in-class activities, or online biology learning modules. SocialMedia:The COESP has Twitter, Facebook and Instagram accounts. The most utilized media outlet is Twitter, which is most commonly used to report out COESP activities at conferences or at events on campus. These social media outlets help the COESP to promote itself inside and outside the University. We currently have 33 followers on Twitter, with our highest hit posting announcing award winners and publication of education research in CBS. PhysicalMedia:A COESP logo was developed very early in the time of the COESP to help promote its brand. Cards with our logo and addresses for our social media outlets were printed and are handed out at events. We have recently developed a banner that has been used at conferences to promote the COESP and COESP-related research. We also had pads of paper and pens with the COESP logo produced for our 2017 COESP Day event. Conferenceandmeetings:In its first year, the Director of the COESP attended conferences and meetings inside and outside of Guelph, representing the COESP and promoting its formation and work. In the future, this work must continue for the COESP to achieve its Vision. In addition, meetings within the College with community members must occur continuously to establish, maintain, and grow relationships so that the COESP adheres to its values in its decisions.

7

MeetingtheNeedsinCBSIn its first year, the COESP initiated several activities to carry out its mission. These activities and our challenges were reported at the Teaching and Learning Innovations conference (May 2-3, 2017) and at the COESP Day (May 4, 2017) (Appendix 2). The activities and the results or resources coming out of our activities are all accessible on the COESP website: https://www.uoguelph.ca/ada-cbs/coesp/coesp-events/previous-coesp-events SeminarsandWorkshopsDate Event Title

May 10, 2016 COESP Kick Off Keith Trigwell Plenary Lecture Introducing the COESP - John Dawson COESP Global Cafe

July 7, 2016 BOPPPS Lesson Planning - John Dawson

July 21, 2016 MultipleInstructorsinaCourse-SteffenGraether

November 9, 2016 Grad student journal club - Preparing for Jim Smith's visit

November 15, 2016 Jim Smith Guest Lecture

November 17, 2016 Educational Technology Fair - Student Response Systems *Supported by the COESP

December 12, 2016 Evaluation of Teaching Event Student Views on Teaching Evaluations The Synoptic Tenure and Promotion Policies - John Dawson Student Evaluations of Teaching - Shoshanah Jacobs Alternative Evaluations of Teaching - Kerry Ritchie

January 26, 2017 Making Grading Easier with Crowdmark - John Dawson

May 4, 2017 COESP DAY 2017 - Engaging Your Students Welcome - John Dawson Michelle French Plenary Lecture / Workshop

RecognitionandPromotion

August 25, 2016 BioEdGraduateStudentAwardSeminar- Kaitlin Roke

8

InclusivenessUndergraduates -- To include the breadth of the CBS community, the COESP initiated an undergraduate blog where they can write about their education experience in biology at the University of Guelph (https://guelphbioed.wordpress.com/). The students spoke clearly about their concerns over the future and about how to deal with stress and maintaining mental health, particularly during midterm and exam time. The themes of the first year of the undergraduate blog are currently being analyzed and will be reported back to the COESP over the summer. This analysis will inform the efforts of the College and COESP to meet the needs of our undergraduates. Graduate Students -- Graduate students have been active participants at COESP events, co-learning with faculty and staff about learning strategies and activities. Participation in COESP summer seminars in 2016 was credited for graduate students towards their University graduate teaching credential offered through OpenEd. The COESP graduate student representative also lead a journal club to prepare students and faculty for our visiting speaker seminar in November. The result was a very engaged and prepared audience for Jim Smith when he came, yielding high quality questions and discussion. The model of holding a journal club before a visiting speaker comes will be repeated. Both undergraduate and graduate students presented at the Evaluation of Teaching Event on December 12, 2016. Their insights had a major impact on participants and they contributed to the conclusions of the event. CBSTeachingDiscussionGroupsWe re-branded the PEER consultation groups into the CBS Teaching Discussion groups to reflect that activities of the groups. The PEER groups were designed to have a small group of faculty (4-5) come together to discuss teaching issues of concern in a safe and confidential environment. Each member of the group would host a meeting, facilitating their topic of discussion. The group of potential CBS teaching discussion group members met in October 2016. The group decided to have one big group of 11 participants (schedule attached – Appendix 3) to obtain a wider breadth of feedback. With a larger group, external facilitation was requested by the group. As a result, the COESP Director became a participant in the discussions. As has happened in the past, participation in the group waned over time. Three of the 11 presentations were cancelled because of conflicts. A core group of 4-5 people came consistently. Moving forward, the goals, format and effectiveness of the CBS teaching discussion groups must be evaluated. The groups as they are currently structured and intended may have run their course to ensure the sustainability and impact of the program. FormingtheBioEdResearchHubThe BioEd Research Hub is a group of faculty who are engaged in discipline based education research in biology; something we are calling BioEd. Dr. Gen Newton from the HHNS Department was appointed the first Coordinator of the BioEd Research Hub in Summer 2016 and facilitated

9

the first discussion of the group that Fall, with participation of members from all 3 Departments, including: HHNS IB MCB Gen Newton Kerry Ritchie Coral Murrant John Zettel William J Bettger

Shoshanah Jacobs Karl Cottenie Patricia A. Wright

John Dawson Steffen Graether

The group met to discuss their research interests and ways that they could support each other in BioEd research efforts. The group also agreed to assemble a research project as a team. Through the end of Fall and early Winter, the group worked on a Learning Enhancement Fund research proposal that had its genesis in the COESP event on December 12, 2016 around Evaluation of Teaching. That event recommended that 4 dimensions of teaching evaluation be adopted for effective assessment of teaching effectiveness: 1) student evaluations of teaching, 2) focus groups with students during and after courses, 3) peer evaluation of teaching (e.g. COPUS), and 4) mechanisms promoting self-reflection and improvement goal-setting for instructors. After discussing the Hub’s strategy with the acting AVPA, the Hub decided to first target how instructors can most effectively utilize student evaluations to improve their teaching practice. Future research will examine the effectiveness of the questions being asked of students, especially considering potential changes in the questions being considered by the University. Beyond student evaluations, the Hub can pilot in-class peer evaluations using tools like COPUS or can examine best practices for student focus groups as reliable and actionable feedback for instructors. The LEF proposal “An Instructor’s Guide to Using Student Evaluations of Teaching to Improve Teaching Effectiveness in the College of Biological Sciences. Phase 1: How Instructors Use Student Evaluations of Teaching” was awarded funding on May 5th, 2017 (Appendix 3). Gen Newton has since taken on a Master’s student to begin the research. The work of members of the BioEd Research Hub was highlighted at the Teaching and Learning Innovations Conference at Guelph in May and at the COESP Day 2017, when our new banner was revealed along with poster presentations. Notably, 4 of the 7 posters at the University TLI conference were from the BioEd Research Hub. The Hub plans to have a significant presence at the upcoming Western Conference on Science Education in London (in July 2017) and at other local conferences. In addition, the Hub will continue to meet to discuss our own research and ways we can collaborate and support each other. The Hub plans to submit an LEF application for Winter 2018 around assess the questions of student evaluations, as part of our larger collective research plan around the best means of measuring teaching effectiveness. FundingSupportPointing to the foundational document describing the idea of the COESP (Sept 2015), the COESP has put three sources of funding support together for CBS. Detailed information about each of these grants is found on the COESP website here:

10

https://www.uoguelph.ca/ada-cbs/coesp/coesp-grants

1) COESPConferenceTravelAwardsThis award provides support for undergraduate and graduate students, postdoctoral fellows, staff, sessional instructors and faculty from the College of Biological Science to attend a Scholarship of Teaching and Learning conference. Since we firmly believe that exposure to the encouraging teaching and learning community will inspire CBS members, priority is given to faculty and staff who have never been to teaching and learning conferences before. Up to $7,000 annually

2) COESPCourseImprovementGrantsThese grants provide support to improve and develop courses in CBS in ways that enhance student learning, course delivery and methods to assess the outcomes of the course. The intention is to encourage instructors to renew and innovate their courses, providing support for those changes. Up to $10,000 annually

3) BioEdResearchGrantsThese grants provide seed funding or support to conduct inquiry research of higher education in biology (BioEd) that has strong potential to lead to peer-reviewed publications. Areas include, but are not limited to, investigation of student learning, teaching activities and the student learning experience. Up to $10,000 annually ConclusionIn our first year, the COESP has been incredibly active in the College and in the Teaching and Learning Community of the University of Guelph and in Southwestern Ontario. Our grants were just introduced in the Winter Semester of 2017. Year 2 of the COESP will see the first iteration of the grants, so we will be able to report on the interest and outcomes of the grants next year.

11

OtherchallengesfacingeducationOur research and discussions have identified other challenges to consider moving forward:

• large class sizes – the effective integration of online materials in classes o classroom response systems o textbook publishers providing more robust homework / quiz support

• alternatives to lecturing – active learning classes • technology in the classroom

Results of the University-level Teaching and Learning Strategic Planning currently underway at Guelph identified these top 3 actions in the next 3-5 years:

1) improvements to the physical spaces for learning on campus, permitting more active learning approaches

2) greater formalized support, reward and recognition for engagement in SOTL and in classroom innovation at the level of administrative validation and tenure and promotion policies. Development of SOTL Chairs for each College of the University.

3) the formation of teaching and learning faculty at the university; teaching stream tenure-track faculty

Items #2 and #3 are related to a general perception that SOTL or innovation in teaching and learning is under-promoted, supported or rewarded with the current system. This is a major challenge. Our analysis of the Tenure & Promotion (T & P) documents from the three CBS departments suggests that there is enough flexibility to permit teaching and learning-focused faculty in CBS that would be evaluated using the same T & P policy; however, the perception (discussed below) is that the policies are not fully interpreted or embraced in practice. Primary among the concerns are the evaluation of peer-reviewed SOTL publications as Scholarship within the Research segment of the DOE. Access to graduate students to carry out education research in CBS is another issue. Current discussions around graduate students and SOTL research in CBS at Dean’s Council are a positive step. EngagingFacultyinourTeachingMissionIt has been challenging to engage faculty in COESP events and services. To provide support, we hired a Marketing Co-op student, Charles Di’Illio to help promote the COESP and its events. Despite his experience with marketing, Charles was challenged with drumming up support and participation in COESP events. He promoted the Crowdmark event January through several means, including printed fliers, email blasts, and personally visiting faculty. Ultimately, there were 3 faculty and staff members at the event who were not on the COESP steering committee. He was also challenged with the administrative structure of the University in the context of event planning; because the COESP is a new Office, Charles needed to describe the COESP and his role. He learned to get help from the Director or the Dean’s office to move things along. Several complex and inter-related factors contribute to disengagement of faculty in our teaching mission, including:

12

• how teaching activities are recognized and rewarded at the tenure and promotion level is

a major contributing factor. There is the perception that teaching is evaluated at T & P meetings almost exclusively by the average Student Evaluation of Teaching (SET) score provided on our SET reports

o This perception is further underscored by College T & P data, which only reported normative SET data for undergraduate teaching (“CBS Norms” email January 11, 2017)

• the inclusion of graduate student supervision as a teaching activity. o Currently, there are no quality measures of graduate supervision. o Only the number of graduate students overseen by faculty was reported in the CBS

Norms report. The COESP was pleased to see the report from the College Strategic planning process that included teaching as a priority and essential part of the mission of the College. At the same time, we are concerned with comments from faculty who state that research is their only priority. To address the core issue of how teaching is recognized and rewarded, the COESP held a half day workshop on Dec 12 regarding the evaluation of teaching. The content from the event included a presentation from undergraduate and graduate students about their perceptions of teaching evaluations and where they obtain their information about future courses; the information and resources from this workshop are available on the COESP website. Despite the calibre of the information and the importance of the issue, only a handful of people who were not directly involved in planning or presenting at the event chose to attend, again underscoring the challenge of engaging faculty in the teaching domain. LessonsLearnedUltimately, we learned that we cannot expect faculty to come to us; we need to go to them. Going to them means establishing relationships to learn about their needs so that we can help them “where they are.” As a result, a future plan is to go on a “listening tour” where COESP members meet with faculty and staff in small groups or one-on-one to simply talk about how things are going. Through those discussions, issues and potential ideas can be shared such that the receiving person can be connected with the resources available to them through the COESP and elsewhere.

13

FinancialReport The COESP has an annual budget coming out of the Dean’s Office of $29,000. In our first year, we did not provide any COESP grants. As of May 18th, the budget remaining in the Project for the COESP was: $19,061.50 (See Appendix 5). The breakdown of expenditures as of May 18th is: Student salary – Co-op student W17: Charles Di’Illio OWSP W17: Tara Allohverdi

$7,704.72

Work Study recovery -$364.65 SALARY SUBTOTAL $7340.07 Hospitality / entertainment Student Response System (SRS) Food Service Food Service for CBS TGD

$926.14

Space – rental of Atrium for SRS $50 Awards – Graduate student award (Aug ’16) $500 EVENTS /MEETINGS SUBTOTAL $1476.14 Registration –Director participation in conferences to promote the COESP

$260.19

Mileage – Director participation at conferences to promote COESP $151.29 Printing services / graphic design – COESP Logo and cards $393.50 Photocopying $285.81 PROMOTION SUBTOTAL $1090.79 TOTAL Expenses $9907

This number does NOT include all of the costs associated with the COESP Day run on May 4th. The total amount claimed for that event was: $1,208.26 gift, COESP Promotion materials $500 honorarium $823.70 food TOTAL: $2,531.96 Some costs associated with COESP Promotional materials are one-time items, such as the COESP Banner design and production. We estimate that we made 660 colour copies for the COESP through the ADA’s office. These copies will be accounted for these via Sherry Hall. At a cost of 40¢/page, the total cost is: $264 Based on the available information, the funds spent in Year One are:

$12,734.46 This is well below the annual budget of $29,000. In the future, we do not plan on hiring a co-op student for the COESP. Most of the funding in Year Two will go toward fulfilling our Grants. One budget item that must be factored in is the promotion of the COESP to raise awareness of the Office and its mission. We understand that our budgeting is over-extended for year 2 and 3 because of the allocation for the grants, but we believe the College will recognize the value of the COESP and support its work.

14

YEAR2oftheCOESP An issue for the COESP in year 2 is that the Director will be on a Study/research Leave from mid-December to the end of May. Therefore, the COESP must find an acting Director for that time. This situation presents an opportunity for others to experience the realities of the position and gain experience with this work, developing future leaders. Another arising issue is turn-over in the membership of the steering committee. Undergraduate students routinely cycle through; however, we also need to address succession in the faculty representative positions. It is possible that within the next year we might need to find new representatives for one or two of the steering committee members. Addressing succession requires consultation with the ADA and Department Chairs. The Physical Sciences and Engineering Educational Research group (PSEER) has been renewed with the appointment of its new Director, Dr. Dan Gillis from Computing Science. The Director of COESP met with Dr. Gillis in December 2016 to discuss the plans for re-starting PSEER. In Year Two of the COESP, we might see greater collaboration with PSEER to bring in high-profile speakers and events that relate to STEM in general. In addition, the COESP steering committee has learned that it must continue working through the summer semester to plan events for the Fall and Winter semesters. Primary among these plans will be events where recipients of COESP awards and grants present their work or provide feedback from conference they have attended. Such reporting is a new initiative coming out of the COESP Conference Travel Awards that has the potential to infuse the innovation and energy from conferences into the CBS community at Guelph. In year 2, we must set up administrative structures to provide COESP awards and grants. The main questions are how and who will administer these funds? Support and ideas from the ADA and Dean’s Office are needed. Summer 2017 Fall 2017 Winter 2018 COESP DAY 2017 Summer event: Grad student award (August) COESP Travel Awards – Conference Debriefs Feedback CBS TDG • format / effectiveness • vision Undergraduate blog analysis

Event: Improved PowerPoint, Improved learning? CBS TDG becomes… SOPs: cases, essays, opinion, monographs • ARES? Journal Club Book club? EdX course ?

Director on SLR Event (smaller) TBD Plan COESP Day 2018

IdeasforYear2:

1) ConferenceDebriefSessionsThroughout the year, we plan on holding Conference debriefing sessions with the participants of Teaching and Learning Conferences to share what they learned at the Conference(s). A priority

15

will be to document new approaches, tips and tricks, insights or upcoming issues regarding biology education that arise from the conferences. These ideas will provide the COESP and the College in general with information to guide future work. Two events are planned for the Summer 2017 semester: debriefs from the oCube UnConference and Western Conference on Science Education.

2) PresentationbyDesignA COESP Steering Committee member will attend the McMaster Symposium on Education and Cognition (July 27-28, http://edcogmcmaster.ca/) and return with notes from the Presentation by Design workshop: design strategies based on scientific principles of memory and attention to improve your presentations. Here at Guelph, we will develop a workshop to transfer the skills here in CBS that will improve student learning through better use of PowerPoint presentations.

3) BookClubTo engage in the literature around education research and grow the teaching and learning community in CBS, a regular book club can be formed. The format is a group of people who commit to reading a chapter regularly, come together over lunch, discuss the content and answer some questions. Mirroring best practice, there would be questions to consider while reading the chapter, and then questions to answer during the book club meeting. The final section of the book club would be creative thinking about how to apply what we’ve learned. At the end of the book, we would have a debrief with a short self-reflection paper about how we individually will use what we’ve learned. Books for consideration include: “Thinking Fast & Slow” by Dan Kahneman “The Undergraduate Experience” by Peter Felten, et al “Specifications Grading” by Linda Nilson “Discipline-based Education Research: understanding and improving learning in undergraduate science and engineering” published by the National Research Council “The Life-Changing Magic of Not Giving a F*ck” by Sarah Knight

4) STEMTeachingCoursesContinuing on the theme of deepening our expertise as instructors, the COESP could sponsor taking online education courses. For example, there are “EdX” online courses: (https://www.edx.org/ ) An Introduction to Evidence-Based Undergraduate STEM Teaching: Learn how to implement evidence-based teaching strategies in your university classroom as well as effective methods for assessing teaching and learning. Advancing Learning Through Evidence-Based STEM Teaching: Learn effective teaching strategies and research that supports them, and how to collect, analyze, and act upon evidence of student learning. We could apply COESP Course Improvement funding to pay for those who complete this course and wish to get the certificate. Such a certificate would be a credential for faculty and instructors, demonstrating evidence of professional development on the T & P reports. This course would be especially useful for graduate students looking to broaden their post-graduate opportunities.

16

CONCLUDINGREMARKS The COESP has been incredibly active in its first year. From creating its mission, vision and values, to holding and supporting teaching and learning events and facilitating ways for the CBS teaching and learning community to have a voice, the members of the COESP have demonstrated creativity, perseverance, commitment and incredible work ethic. We believe we have set a strong foundation and have started to build a solid reputation at Guelph and beyond as leaders in biology education. As the leader in biology higher education in Canada, the COESP is an opportunity for the College to gain external support for COESP activities as part of an integrated College fund-raising and promotion strategy. The COESP faces challenges to engage faculty in its mission. We recognize that this group has several competing priorities and that engaging faculty may require a cultural shift. To succeed, a cultural shift requires commitment and leadership from administration. The COESP stands ready to support teaching and learning innovations in CBS being promoted and championed by the Associate Dean Academic, Dean, and Provost. As we move into our second year, we shift from the excitement of creating the Office, to the work of growing the community and making an impact. The COESP will be promoting more in the second year with our grantees and with our events. We believe our second year will be our best year yet as we build on the experience and wisdom collected from our first year.

Appendix 1:

Needs and Challenges in Biology Higher Education Today

Needs and Challenges in Biology Higher Education Today Mixed-method collection of information Analysis by qualitative connections and correlations. Sources of data:

1. Job talk stickie notes 2. Teaching and learning developers in Ontario 3. COESP Steering committee members 4. TLI conference session attendees 5. oCube UnConference list of proposed discussion topics

These sources contribute to a general framework of issues: CLASSROOM ISSUES

• large class sizes, motivation and engagement of students in classes, classroom technology, and the physical space of classes

ASSESSMENT CHALLENGES

• effective feedback, setting expectations and standards, academic integrity issues, student evaluation of teaching concerns

PROGRAM ISSUES

• Integration of course materials across programs, maintaining relevant and revised courses.

PROFESSIONALISM

• Students: politeness and respect, preparing students, mentoring • Instructors: dealing with student feedback, keeping current, mentoring

TEACHING AND LEARNING ISSUES

• Having time to keep current and revise pedagogies and courses STUDENT ISSUES

• Developing metacognition, instilling high standards, diversity / preparedness, understanding and meeting student needs

SUPPORT

• Developing a teaching and learning culture, funding for research, mentoring

Job talk stickies Dec 14 Audience included: undergrads, grads, faculty, staff, admin

Asked: What is one need or challenge for biological sciences education at Guelph?

Then – set up the 4 categories on the board and forced them to put their idea in ONE category.

Category stickies

Education Research

• acceptance culture for SOTL research • funding

• student space for collaboration • promotion – to entice students to come to CBS @ Guelph

Learning outcomes development and assessment

• assessing the value of field-based experiecnes • experiential, field-based learning opportunities in large (>100 learners)

classes • experiential challenges - hands on and problem solving • experiential opportunities – show rather than tell

• engaging instructors - who are up-to-date and interesting and qualified / open-minded

• career direction and career development of grad students – total professionalism

• keep content / process relevant to job-related opportunities • career / job placement for our graduates

• large class sizes - difficulties with skills-based assessments (competencies, critical thinking / problem solving, communication)

• needs for multiple forms of assessment

• how to get beyond teaching primarily knowledge

Evidence-based teaching

• large class size • reduce student:faculty ratio in classes • class size (large) • increasing class sizes • class size • large numbers of students - diminishing research

• bringing engagement to lectures of large classes • engaging large classes of students, often in core courses - how to capture the

interest of students coming with diverse disciplines • how to engage students to enjoy learning bout metabolic pathways • students: engagement, motivating • motivate students to get more engaged in class work (lectures)

• labs, active learning in face of growing enrollment

• integration across levels of biological complexity • flow of information through the courses, i.e. consistent information

• more DE opportunities for biology students

• educate faculty and students in benefits of evidence-based teaching • building a culture that values teaching as much as research

Other

• create scientifically literate citizenry – through teaching & learning in the context of biological sciences

seeing the intersection of several themes at once – large classes, engagement – other issues around large classes (active learning)

So.. in order of numbers:

1. large classes - engagement, assessing skills, active learning in, assessment 2. engagement of students 3. experiential learning issues 4. professionalism - career planning and development 5. building a culture that values scholarly teaching 6. integration across programs / biological organization

2016-04-13 Building on the framework Following after the Dec 14 stickies and external friends, we had this framework:

Challenges in Biological Sciences education – faculty target

Culture valuing

Scholarly teaching

Large Classes Engagement of Students

SOTL Research

Integration across

programs / biological

organization

Professionalism

stickies & externals

T&L folks appear to say we need to build a culture that values teaching to begin with, not to mention that it's scholarly. We need to use evidence-based teaching and learning approaches (aligns with active-learning in CBS definition?). They say we need to make this visible – this aligns with our own assessment of the situation here

But also talk about learning practices that promote deep learning – deep, sustained, progressive, enduring. I think maybe we are saying the same things, it's just that our faculty are not familiar with active-learning techniques and so we don't through that out there, although one of the responses from the Stickies did talk about active learning. Perhaps, too, we tend to talk about experiential learning. We need to define active-learning experiences within CBS to include those that can occur in a lecture theatre and with large classes.

1. o inquiry-based,

problem-based o active-learning

T&L folks also mention diversity of student experiences, foundations and motivation. We need to understand student motivation and leverage that to enhance engagement

demonstrate the knowledge, skills and abilities that students are acquiring over the course of their programs. Kind of a loose connection; this is more the bailiwick of the Department's UCCs and then the ADA overseeing the programs. However, it would be good to ask about sharing information to foster better integration and reduce repetition or conflicting information.

Here, the T&L folks suggest striking an intentional balance between knowledge and skills and the need to ensure the acquiring of skills that will best prepare students for future success. We need to define those skills that will prepare students for success (including those that will not go into research) and move beyond the sentiment that teaching laboratory / field skills is preparing our students for a career.

Tina

Lack of knowledge surrounding what SoTL is

Lack of knowledge re: resources such as OpenEd teaching workshops, UTTP

SoTL-focused students in CBS more isolated/have to work harder to be involved in their departments

does SoTL research belong in CBS?

Is SoTL research “real” research?

CBS identity

Advisor - Student Relationships

Mental Health

Funding

Science Program @ Guelph

Shoshanah

Faculty want a tool kit of ideas and strategies.

They want a pros and cons of things.

And they don't want to have to do the work putting that stuff together. (opportunity to be useful!!!!)

to cope with the growing class size I've just gotta do a crappier job of teaching. I imagine that this new approach is going to be met with some stress by the faculty

lack of funding for SoTL research

lack of mentoring faculty into improving for next time

mentorship program

Erin

Inclusion of teaching in tenure and promotion. This point isn't so much as a question as it is a recurring theme in conversations.

Folks who are considering making changes, or who are excited about something going on in their classroom, but hesitant to do too much because it will take time from research and/or it won't be recognized in their promotion process.

Large classes (and assessment). Questions here tend to focus on how to engage students in the large class setting, but also around methods of providing formative feedback in the large class setting.

Sometimes these questions dovetail with questions around technology, but not always.

Related questions around modes of assessment 'beyond' the MCQ or 'authentic assessment' emerge less often, but are common enough

Motivating/engaging students. Questions about motivation take a lot of forms.

Sometimes the questions surround large classes, or classes with a lot of content (it's funny, I've never encountered an instructor teaching a class with *not* a lot of content!) or classes in the early morning.

More recently I've encountered this question with folks who are flipping their classes and experiencing the challenge of student preparation and engagement in the more active components of the course

Steffen

It takes time away from research

While not explicitly said, faculty seem to indicate that their teaching is fine ‘as is’.

improved teaching practices is something that should be looked into, but generally few have indicated that they would participate

Most of the research out there is not valid (because it is poorly done, the bystander effect)

2016-04-26 TLI feedback

1 2 3 4 5 6 Notes

Participants

Pamela Barkwell (Brock), Danyelle, Mary, Andrea Bucholz

Char, Annette, Sofie Lachapelle, Kristin Bianco

Justine Tishinky, Maryann Kope, Shannon Rushe, Nick Strzalkowski

Clarke Mathany, Carol Sparkes, Natalie Cuerbrecht, Tariq Akhtar

Tina, Genevieve, Megan, Jason Dodd

Kerry, Lissemore and 2 others (Chris from Humber?)

CLASSROOM ISSUES

for ESL instructors, getting students to speak English cell phones / computer technology timeblocks for classes, i.e. 50 min, 3 h, etc

engaging students watching TV / Youtube / Facebook - Laptop in the lecture - videogames

large class sizes engaging students use of technology - including student use of tech in class and distractions physical space

teaching 100's of students in one class getting students to do homework getting students to come to class getting students to show up after the midterm netflix groupwork (effective)

teaching effectively in large group classes engaging students technology - teaching; student distraction depth of inquiry-based

class size reliable technology students' focus within class purpose of lecture

Large classes

Engagement and motivation

Technology (&distraction)

Pedagogies / Time and Space

ASSESSMENT

assessment / fair questions for midterms and finals how to give positive feedback penalties for late assignments grade inflation / grade expectations / regrades

determining appropriate assessment academic misconduct

effective assessment of learning outcomes time for effective feedback feedback - end of year too late student expectations integrity online exams student evaluations of teaching

standards - you can handhold as much as you want technology - cheating

bar / expectiations - what is excellent vs good vs acceptable academic integrity face-time (students & profs)

How to provide effective feedback

What are the expectations and standards?

Academic integrity (often with technology)

SET

PROGRAM ISSUES alignment (curriculum, course) etc

understanding your course's place in the bigger curriculum connection with faculty who teach pre-requisite courses

keeping it relevant to everyday issues

lack of departmental / institutional support ($) help determining low-resource solutions -- $ and time

students see inconsistencies people who aren't here!

Alignment / Integration

Revisioning processes

How to be inclusive?

PROFESSIONALISM

Students

STUDENTS students walking out of class /arriving late dealing with aggressive students professionalism of students (or lack thereof)

get students prepared

political correctness -- safe space

professionalism & hand holding skills for workplace good mentoring (students & profs)

Politeness / Respect

Preparing students

Mentoring

PROFESSIONALISM

Instructors

access to technology; keeping up with tech keeping up on pedagogical strategies; evidence feedback - end of year too late

good mentoring (students & profs)

Mentoring

Keeping up (with technology)

mid-semester feedback

TEACHING & LEARNING

planning time time to innovate, create, change and for TAs

answering emails (time)

help determining low-resource solutions -- $ and time

evidence vs myths time constraints DOE

TIME!

STUDENT ISSUES

student-perceived entitlement self-direction, independence student apathy (falling asleep) unprepared students changing demographic of students cultural differences with students poor writing skills of students

make them realize what they know and what they don't know make students intentional about learning develop a l ove of learning making judgment calls -- prioritize what info is worth retaining make them do the best they can higher standards -- also to their ability make them good communicators learn what are your strengths -- should you be here? technology -- to do more? Do we limit its use?

YOUR STUDENTS: recognize & support at-risk students mental health

understanding student needs student learning goals student backgrounds & variance

student attitudes

students' DOE

Are students homogeneous? - diversity - do we care? what do we do?

Understanding and meeting student needs

Developing metacognitive / self-awareness skills

Instilling high standards

Diversity / preparedness

SUPPORT funding

YOU: support & advice from colleagues a strong teaching community that's accessible to you TA support which would allow you to do more creative assessment access to teaching oriented conferences to meet people interested in the same things - is there a thing like this at Waterloo?

Mentoring

developing a teaching & learning culture / community

Funding for research

OTHER

getting money's worth students as consumers helicopter parents and teachers at high school

2016-05-24 oCube UnConference Topics I pitched a session on "needs and Challenges in Biology Higher Education" and then discovered that ALL of the ideas pitched essentially spoke to the needs and challenges!

so the approach is to strip the ideas pitched at oCube and sort them into the preexisting framework.

Here is the votination site: https://www.surveymonkey.com/r/PG8M3ZX

Participants oCube Pitched Idea

CLASSROOM ISSUES

Using peer discussions & activities in class - reducing setup & take-down time

Does it matter when learning occurs? By midterms? By finals?

Virtual labs – simulations

Videos in class/outside of class how to use them efficiently

Variety of expertise and proficiencies in class & strategies to do so.

Inquiry in the lecture: removing the cookbook lecture

Where learning occurs - it isn't always in the lecture.

Social media in our courses: Snapchat, etc.; comfort level with students paying for it. What do we do with social media (engaging students?)

Active learning - strategies for large/small classes

How to use teams really really well in classes.

ASSESSMENT

Does it always have to be Blooms? (Is Blooms' behind the decline in natural history teaching?) Experiences with other frameworks

How do we not kill the bottom, address the middle, and give value for money to the upper performing students

Assessment (broad sense) - how much variety, what weighting, tips to assess large classes with technology

Specification grading - how to implement

Cheat/aid sheets in midterms?

How to write really good questions that assess higher order cognition and how the hell do you mark those questions

Two-stage (collaborative) assessments/assignments

PROGRAM ISSUES

Supportive instruction - successful implementation & reduction of scaffolding across a course

How do we balance course technology use for both instructors and students? (How many technologies are too many?) (structured use of technology - NOT social media)

How to change a course from content heavy to appreciation (histology course)?

How to introduce next generation sequencing data to students?

Course-based research experiences (for lab component)

Interdisciplinarity within biology/outside biology/outside the sciences - re-creating first year biology

Training students to be problem solvers and applying content in community engaged way

Transfer credits - how to determine equivalencies. Sharing info about our courses PROFESSIONALISM

Students

TA training: collective wisdom & practices. (Helping to change the professoriate of tomorrow.)

PROFESSIONALISM

Instructors

Mentoring early career instructors in evidence-based practises to improve undergraduate education

proper assessment of teaching (what's expected from us, how we teach, framework for how we're assessed, recommendations for how we feel we should be assessed)

TEACHING & LEARNING Biology education research in Canada - state of the work

STUDENT ISSUES Teaching and cultivating creative confidence in biology undergraduate students

SUPPORT

Keepin' it fresh: teaching the same course year after year and making it interesting for yourself when there's only limited scope for change in basic outcome.

Tools and strategies to help save time/keep organized, sane, help to find that unicorn "more work/life balance"

How to deal with student and faculty pushback against new practices

OTHER Losing the textbook: when to, alternatives, and other considerations

Big data in courses: how can we organize cross-university data collection & analyses

EdTech keeps coming up. Different types, however --

1. discipline-specific tech – data gathering and analysis – practical tech related to the discipline

-- would this include analysis of LOs? – the discipline is LOA; analogy is statistical analysis of ecological data

2. virtual education – labs, lecture capture, assessment

– social media use related to education

Appendix 2:

A Discipline-Specific Education Offices: Lessons Learned in Our First Year

Poster presented at the Teaching and Learning Innovations Conference, 2017, and the COESP Day, 2017.

A Discipline-Specific Education Office: Lessons Learned in Our First YearJustine Hobbins*1, Erin Aspenlieder4, Bill Bettger1, Phoebe Hui Bruce1, Charles Di Ilio5, Steffen Graether2, Shoshanah Jacobs3, Laura Sloat4, Charlene Winchcombe-Forhan6, Paisley Worthington2, and John Dawson*2.

1 - Human Health and Nutritional Sciences, 2 - Molecular and Cellular Biology, 3 - Integrative Biology, 4 - OpenEd, 5 - CBE, 6 - CBS

Teaching is just one of a long list of priorities

Implementation of change is a process, not an event

Operational ChallengesScheduling of events

Meeting needs of target audience

Reaching beyond the “usual suspects” – closing the gap

COESP Inauguration

• COESP Kickoff • Undergrad Student Blog Commences• BOPPPS Lesson Plan Seminar• Multiple Instructors/Course Seminar• Grad Student Led Seminar/Award

• BioEd Research Hub Commences• Journal Club & Guest lecture• Student Response System Fair Support• CBS Teaching Groups Commence• Faculty/Student Led SETs Seminar

• Crowdmark Seminar• Travel Grants Announced

• COESP Day• Course Improvement & SOTL Research Grants

Announced

Classroom Issues• Large class sizes, motivation & engagement of students in classes, classroom technology, physical space of classes

Assessment Challenges• Effective feedback, setting expectations & standards, academic integrity issues, student evaluation of teaching concerns

Program Issues• Integration of course materials across programs, maintenance of relevant & revised courses

Professionalism• Students: politeness & respect, preparing students, mentoring, career planning

• Instructors: dealing with student feedback, keeping current, mentoring

Teaching & Learning Issues• Having time to keep current and revise pedagogies & courses

Student Issues• Developing metacognition, instilling high standards, diversity/preparedness, understanding & meeting student needs

Support• Developing a teaching and learning culture, funding for research, mentoring

Visi

onM

issi

onVa

lues

About the COESP

“Report back” sessions by COESP Travel Grant holders

Re-visioning of the COESP Teaching Discussion Groups

Listening tour – fostering connections with colleagues

BioEd Research Hub applying for fundingHow faculty use student evaluations of teaching (SETs)

College of Biological Sciences Office of Educational Scholarship and Practice

www.uoguelph.ca/coesp

COESP’s Responses

To be recognized as the national leader in educational scholarship and practice in the

biological sciences.

To support and promote:

Evidence-Based Practice

Inclusive

Learning Outcome

Development & Assessment

Biology Education Research

Evidence-Based

TeachingAccessible Responsive

Summer ‘16

Fall ‘16

Winter ‘16

Summer ‘17

March ‘16

Challenges in BioEd

Appendix 3:

CBS Teaching Discussion Groups Schedule 2016/17

CBSTeachingDiscussionGroups2016/17TUESDAYS11:30-1:30ROOM2315intheScienceComplexEachpersonpresentinghoststheirsessionandisresponsibleforsendinganyneededinformationtotheparticipantsbeforethesession.Date Host1/Topic Host2/TopicNovember22 JosephYankulov/

Learningstylesandperformancestyles.Aretherepreferredperformancestyles?

BerenRobinson/Iaminterestedinthehypothesisthatpeer-reviewisausefulinstructionalstrategy

January24 LizBoulding/Challengesandbenefitsofteamteaching.

AmyNewman/Optimizingco-teachingofintegrativecoursestoalignwiththeIBinitiativetoincreaseflexibilityinthefacultyteachingloadwhilenotcompromisingthephilosophy,continuityandflowofacourse.

February28 JohnSrbely/SocraticQuestioningTechniquesintheClassroom

PatWright/Creativeassessmenttoolsthatenhancestudentengagementandlearningintheclassroom

March21 GeorgeHarauz/Difficultieswithengagingstudentsina4thyearbiochemistrycourse.

AnnetteNassuth/Engagingstudentsinelectivecourses

April18 LoriAnnVallis/1)howtoincorporate"knowledgetransfer"throughcoursesupportedexperientiallearning?2)howcanIimprovestudentengagementinmy2ndyearclass?

DickMosser/Areweteachersorlecturers?

May16 Char/Studentevaluationsonteaching

Appendix 4:

BioEd Research Hub Learning Enhancement Fund Research Proposal

“An Instructor’s Guide to Using Student Evaluations of Teaching to Improve

Teaching Effectiveness in the College of Biological Sciences. Phase 1: How Instructors Use Student Evaluations of Teaching”

ExploringtheUseofStudentEvaluationsofTeachingbyInstructorsintheCollegeofBiologicalSciences

1

LEARNINGENHANCEMENTFUND(LEF)

FundingApplicationforthe2017-18AcademicYearSubmissiondeadlineMarch17,2017

• Applicationsshouldbewritteninlanguageaccessibletonon-specialistsandincludeo proposal(5pagesmaximumincluding,budget,etc.)o teachingphilosophystatement(1pagemaximum)o c.v.(5pagesmaximum)o lettersofsupportfromcollaboratingindividualsandunits

• ApplicationsaretobesubmittedelectronicallytotheAssociateVicePresident(Academic)at:avpa@uoguelph.caalongwithonepapercopythatincludesallappropriatesignaturestotheOfficeoftheAVPA,ExecutiveSuite,4thfloor,UniversityCentre

PrincipalApplicant’sName:Dr.GenevieveNewton,ResearchCoordinator,TheCBSBioEdResearchHubCollegeorUnit:CollegeofBiologicalSciencesDepartment/School:DepartmentofHumanHealth&NutritionalSciencesNamesandAffiliationsofOtherApplicants:

1) Dr.JohnDawson,Director,CBSOfficeofEducationalScholarshipandPractice2) Dr.ShoshanahJacobs,DepartmentofIntegrativeBiology3) Dr.StefenGraether,DepartmentofIntegrativeBiology4) Dr.KarlCottenie,DepartmentofIntegrativeBiology5) Dr.CoralMurrant,DepartmentofHumanHealth&NutritionalScience6) Dr.JohnZettel,DepartmentofHumanHealth&NutritionalSciences

TitleofProposal:AnInstructor’sGuidetoUsingStudentEvaluationsofTeachingtoImproveTeachingEffectivenessintheCollegeofBiologicalSciences.Phase1:HowInstructorsUseStudentEvaluationsofTeaching.BudgetRequestedfromLEF:$12100.00Budgetfromothersources:$0


2

BriefProjectDescription(100words)TheBioEdResearchHubisanewlyformedfacultygroupintheCollegeofBiologicalScience(CBS)activein

thescholarshipofteachingandlearning.Wehavearesearchprogramtostudymeasuresofteachingeffectivenessasameansofimprovingstudentlearning,whichincludesmaximizingfeedbackfromStudentEvaluationsofTeaching(SET).Theseevaluationsareintendedtoprovidemeaningfulandinsightfulfeedbackaboutteachingeffectivenesstoinstructors,yetthereislimitedknowledgeregardinghowinstructorsuseSETforformativedevelopment.Inourcurrentapplication,theBioEdResearchHubproposesthefirstphaseofalonger-termprojecttodevelopa‘GuideforInstructors’regardingthebestuseofSETtoimprovestudentlearning.WeproposetodeterminehowSETfeedbackisperceived,utilized,andhowitinformsteachingpracticebyinstructorsacrossalldepartmentsinCBSusingamixedmethodsapproachincludingfocusgroupsandsurveys.ThefindingsfromthisstudywillinformthenextphaseofourstudyinvolvingstudentperceptionsofSETquestionsandourplantostudytheimpactofourGuideonteachingimprovement.ProjectRationale

TheBioEdResearchHubisagroupoffacultyintheCollegeofBiologicalScience(CBS)whoareactiveinthescholarshipofteachingandlearning.TheHubwasformedin2016aspartofthemandateoftheCBSOfficeofEducationalScholarshipandPractice(COESP)tosupportandpromoteresearchinhighereducationinbiology.

ThisLEFapplicationisthefirstpartofalargeresearchprogramproposedbytheBioEdResearchHubtostudyandpilotteachingassessmentmeasures.Thesemeasuresinclude:studentevaluations,peerevaluation,focusgroups,andself-reflection.GiventhattheUniversityiscurrentlydiscussingnewSETquestions,theBioEdResearchHubhaschosentostudythewaysthatfacultyuseSETtoimprovetheirteachingpracticeinthefirstphaseofourresearchproject.WhileweacknowledgethatfutureSETquestionsarestillundecided,theprinciplesofSETareconsistentacrossquestionstypes.Asoutlinedbelow,theinformationwegatheraboutbestpracticeofSETusewillbeincorporatedintoaGuideforInstructorsforthebestuseofSETtoimproveteachingeffectiveness.

ThoughtheearliestreportofusingSETinapost-secondarycoursewasin1920,theywerenotusedwidelyuntilthe1960s(Murray2005).Themotivationsforintroducingthemwerethreefold:1)Studentswantedtofeelasthoughtheyhadsomeimpactonteaching,2)Administratorswereseekingmeansofensuringaccountabilitywithrespecttoteaching,3)Earlycareerfacultysoughtameasureotherthanthenumberofpublicationsfortenureandpromotionevaluation(Murray2005).Generally,SETareadministeredduringthefinalweeksofacourseandbeforethefinalexam.Aseriesofcourse-relatedandinstructor-relatedquestionsareposedandstudentsareaskedtorespondusinga5-7pointLikertscale.Participationratesareusuallyaround45%(GuderandMalliaris2013).

SincethefirstuseofSET,therehavebeenover2,000publishedstudies(Centra2003)relatedtotheirvalidity,biases,use,andeffectivemeansofimplementation(WrightandJenkins-Guarnieri2012).ThoughmanystudiesprovideevidencethatSETarevalidindicatorsofeffectiveteaching(whenmetricsofstudentachievementareusedasindicatorsofeffectiveness;e.g.Marsh2007),otherstudieshavedemonstratedthatstudentsrateinstructorshigherincoursesinwhichtheyexpecttoreceiveorarereceivingahighergrade(e.g.Ewing2012,Matos-DiazandRagan,2010).ThoughthevalidityofSETinmeasuringinstructoreffectivenessisthereforeinquestion,SETdoesprovidevaluableinsightintostudentenjoymentof,andinterestin,theircourses(GravestockandGregor-Greenleaf2008),bothofwhichcontributetostudentpersistence.

AlthoughSETprovidefeedbacktoinstructorsabouttheperceptionofteachingeffectiveness(mostnotablyengagement)fromstudents,GravestockandGregor-Greenleaf(2008)notedthatinpractice,SETaremostcommonlyusedforsummativepurposes(suchasinfacultytenureandpromotionreviewsandpersonneldecisions),ratherthanforformativefacultydevelopment.WhiletherearelimitationstoSETthatshouldbeconsidered,SEThavethepotentialtoprovidemeaningfulandinsightfulfeedbacktoinstructors,whichcouldinformteachingandcoursedevelopment.However,facultyhaveconcernsaboutwhetherSETscoresarevalidandreliable(Kelly,2012),whichlikelycontributestotheirlackofformativeuse.Overall,thereisalackofpublishedliteratureregardinghowSETareperceivedandutilizedbyinstructors,andperhapsmoreimportantly,howutilitymightbeenhancedforinstructors.

EvaluationofteachingusingSETistheapproachcurrentlyinuseintheCollegeofBiologicalSciences(CBS),attheUniversityofGuelphandatUniversitiesacrossCanada,andistheonlymandatedmeansofprovidingfeedbackfromstudentstoinstructorsinCBS.However,thereislimitedknowledgeregardingthewaythatSETfeedbackisused


3

byinstructorsforformativedevelopmentandwhetheritinformsoverallteachingpractice(includingcoursedesignandre-design),aswellashowSETutilitymightbeimprovedtopromoteuseinaformativecontext.ThisgapneedstobeaddressedtoenhancetheeffectivenessofSETfeedback.“Explainhowtheprojectdevelopsanewacademicinitiativeorsubstantiallystrengthensanexistingone”:Thisresearchstudyispartofalong-termprojectthatwill:(i)improvethevalueandefficacySETattheUniversityofGuelph,(ii)enhancetheuseofSETfeedbackbyinstructors,and(iii)enhancestudentengagementandlearningintheclassroom.ProjectObjectivesThisstudyisthefirstphaseinalong-termresearchproject.Thelong-termoverallobjectiveofthisprojectistodevelopaGuideforInstructorsforthebestuseofSETtoimproveteachingeffectivenessattheUniversityofGuelph.EnhancedutilizationofSETfeedbackwillimprovethevalueandefficacyofthistool,andenhancestudentengagementandlearningintheclassroom.Althoughthefullscopeofthisprojectisoutsidetheone-yeartimelineofastandardLEFaward,wedescribeeachphaseoftheresearchherewiththeirspecificobjectivestoillustratehowtheoverallprojectobjectivewillbeachieved.PhaseOne(YearOne-currentLEFproposal):ExploringtheUseofStudentEvaluationsofTeachingbyFacultyintheCollegeofBiologicalSciencesSpecificObjectives–

(i) Toidentify,usingfocusgroups,how~18-24randomlyselectedinstructorvolunteersinCBSperceiveSETfeedback,howSETfeedbackhasorhasnotbeenusedtoinformteachingpractice(includingcoursedesign/re-designanddelivery),andhowtheutilityofSETfeedbackmightbeimproved.

(ii) Todevelopasurvey(theSETFeedbackSurvey,seedescriptionbelow)basedonthefocusgroupfindings.(iii) Todetermine,usingtheSETFeedbackSurvey,howallfacultyinCBSperceiveSETfeedback,howSET

feedbackhasorhasnotbeenusedtoinformteachingpractice(includingcoursedesign/re-designanddelivery),andifithasnotbeenused,howtheutilityofSETfeedbackmightbeimproved.

PhaseTwo(YearTwo):DevelopmentofaSETFeedbackGuideforInstructorsSpecificObjectives–

(i) Toidentify,usingfocusgroups,howstudentsinterpretSETquestions.(ii) TodevelopaSETFeedbackGuideforInstructors,usingtheresultsfromPhaseOneandthefocusgroups

conductedwithstudentsinPhaseTwopart(i),toenhancetheinterpretationandutilizationofSETfeedbackfromstudents.

PhaseThree(YearThree):MeasuringSETFeedbackGuideEffectivenessSpecificObjective–TodeterminewhetherinstructoruseoftheSETFeedbackGuidehasimprovedtheutilityofSETfeedbackforformativecourseandinstructordevelopment.ProjectMethods

Themethodsforallthreephasesoftheprojectwillbedescribedhere,withadetailedfocusonPhaseone.AllresearchwillbeprecededbyREBapproval.PhaseOne-ExploringtheUseofStudentEvaluationsofTeachingbyFacultyintheCollegeofBiologicalSciencesTherewillbethreepartstoPhaseone,includingfocusgroupsandsurvey(developmentandimplementation).FocusGroups:ToinvestigatehowinstructorsutilizeSETfeedback,focusgroupswithdiversefacultypopulationswillbeconductedacrossallthreedepartmentsinCBSthatteachdifferentcoursesinarangeoflevelsanddisciplines.Threefocusgroupswith6-8facultyineachwillbeconducted,withfacultyselectedatrandomandinvitedtoparticipatebyemail.Asemi-structuredinterviewguidewithquestionsdevelopedbytheresearchteamwillbeusedtoguidethefocusgroupsessions.Facultywillbeaskedtoreflectontheirexperienceswithend-of-semesterSET


4

feedback,andtodescribehowthefeedbackhas(orhasnot)beenusedtoinformtheirteachingpractice,includingcoursedesignanddelivery.Inaddition,facultywillbeaskedhowtheutilityofSETfeedbackmightbeimproved.FocusgroupswillbeaudiorecordedandtranscribedbyaGraduateResearchAssistant(GRA).ContentanalysisofthefocusgrouptranscriptswillbeanalyzedusingthematicanalysissoftwaresuchasNvivo(availablethroughaUniversityofGuelphsitelicense),andthefrequencyandvarietyofresponsestoquestionsrelatedtoSETfeedbackperception,utilizationandpotentialimprovementwillbedetermined.Tworesearchers(theGRAandDr.Newton)willindependentlyanalyzealltranscriptsandresultswillbecomparedforconsistency.

SurveyDevelopment:Basedontheanalysisoffocusgroupfeedback,aSETFeedbackSurveywillbedevelopedthatincludesquestionsregardingtheperceptionandutilizationofSETbyfaculty,aswellassuggestionsforhowtoimproveSETfeedbackutility.ThissurveywillincludeprimarilyLikert-scaleandclose-endedquestionstoallowquantificationofperceptionandutilizationoffeedback.Optionsforopentextresponsestosolicitfurtherqualitativefeedbackfromthelargersubjectpopulationwillalsobeincluded.

SurveyImplementation:AllfacultyinCBSwillbeinvitedtocompletetheSETFeedbackSurvey,whichwillbeadministeredonline.Responsestoquestionswillbeanalyzedtodeterminethefrequencyofresponsestoquestions,andopentextresponseswillbeexploredqualitativelybytheGRAandDr.Newton.ThroughtheanalysisofthefocusgroupsandSETFeedbackSurvey,thebreadthanddepthoffacultyperception,utilization,andsuggestionsforimprovementstoSETfeedbackinCBSwillbedetermined.

Asevidenceofthefeasibilityofourapproach,membersoftheBioEdResearchHubareexperiencedwithfocusgroupsasaresearchmethod(Raceyetal.,2016)andhavesuccessfullyperformedsurveysarepartoftheirteachingandlearningscholarship(Lougheedetal.,2012).WewilldisseminateourfindingslocallyattheTeachingandLearningInnovationsconference,andthroughaCOESP-sponsoredpresentation.Fartherafield,weplantopresentourfindingsattheCOUconferenceonlearningassessmentheldannuallyinToronto,oratthenextSTLHEconference(2019).Finally,wewillproduceamanuscriptforpublicationintheCanadianJournalofHigherEducation,anopen-accessjournalwithnosubmissionorprocessingfees.

PhaseTwo:DevelopmentofaSETFeedbackGuideforInstructors(notpartofthisLEFfunding)TodeveloptheSETFeedbackGuideforinstructors,wewillfirstinvestigatehowstudentsinterpretSET

questions.FocusgroupswithdiversestudentpopulationswillbeconductedacrossallthreedepartmentsinCBSaswellasarangeofcourselevelsanddisciplines.Studentswillrespondtoquestionsusingthetraditional,close-ended5-pointscale,andthenwillparticipateinfocusgroupsinwhichtheSETquestionsareusedasasemi-structuredinterviewguide.Focusgroupdatawillbequalitativelyanalyzedtodeterminethemesassociatedwithstudentperceptionofquestions,andtocharacterizethemeaningofquestionresponses.Wewillalsoincludequestionsanddiscusspointstoexpandonspecificpointsofinstruction(e.g.engagement,clarity).Inaddition,wewillanalyzethethemesderivedfromthefocusgroupanalysisusingPrincipleComponentAnalysis(PCA),todeterminewhichquestionsprovidefeedbackthatclusterstogetherandthereforedescribesimilarvariables,andtolookatpatternsandvariationinresponses.Moreover,wewillestablishrelationship(correlations)betweenSETquestionsandthosespecificpointsofinstructionprobedinthefocusgroups.Basedontheseresults,wewillcreateaSETFeedbackGuideforInstructors,whichcanbeusedtoenhancetheirunderstandingofresponsesobtainedthroughthetraditionalsurveymethod,andcanbeusedformativelyforcourseandinstructordevelopment.PhaseThree:MeasuringSETFeedbackGuideEffectiveness(notpartofthisLEFfunding)

AcontrolledstudywillbeconductedtomeasuretheeffectivenessoftheSETFeedbackGuide.Facultywillberandomizedintotwogroups;thecontrolgroupwillreceivenoadditionalfeedbackregardingSETinterpretation,whiletheinterventiongroupwillreceivetheSETFeedbackGuide.Effectivenesswillbemeasuredusingbothsubjectiveandobjectiveoutcomes,includingfacultyperceptionofusefulnessandlongitudinalchangestocoursedesign.PotentialProjectImpact


5

Thepotentialimpactoftheoverallprojectissubstantial:itaddressesakeyneedtobolsterboththeprovisionandutilizationoffeedback,whichistheverystartingpointforinstructorandcoursedevelopment.NotonlywillthedevelopedtoolenhanceSETfeedback,butanevenbroaderimpactisanticipated:enhancinganddemonstratingtheeffectivenessofSETfeedbackisexpectedtopromote‘buy-in’ofitsusebybothstudentsandfacultyalike.StudentswilltakemorecareandconsiderationincompletingSETsknowingtheywillbemoreeffectivelyused,andlikewisefacultywillbemorelikelytoutilizetheirevaluations.Moreover,whilethescopeofthisprojectislimitedtoCBS,thesefindingscanbetranslateduniversitywide.Specifictophaseoneoftheproject,theimpactofthisstudywillbetofillacurrentresearchgapbydescribinghowinstructorsperceiveandutilizeSETfeedbackinaformativecontext,andtoidentifywaystoimprovetheutilityofSETfeedbackwhichmightbeusedinfutureendeavors.ProjectOutcomesandMeasurement(OnlyforPhaseOne/CurrentLEFProposal)

(i) FocusGroupOutcomes–WewillmeasurethefrequencyandvarietyofresponsestoquestionsrelatedtoSETfeedbackperception,utilizationandpotentialimprovement.Asanoutcomeofthisanalysis,theSETFeedbackSurveywillbedeveloped.

(ii) SurveyOutcomes–Wewillmeasurethefrequencyofresponsestoclose-endedquestionsandwillqualitativelyexploreopen-endedquestionsontheSETFeedbackSurvey.

ProjectWorkPlan/Timeline(OnlyforPhaseOne/CurrentLEFProposal)Summer2017–GetREBApprovalforprojectFall2017–Conductfocusgroupswith~18-24facultyinCBSWinter2018–DevelopandadministerSETFeedbackSurveytoallfacultyinCBSSummer2018–Analyzedata,writemanuscriptdescribingstudyfindings,submitmanuscriptforpublication(targetjournal:TheCanadianJournalforHigherEducation).PresentfindingslocallyattheTLIandCOESP-sponsoredpresentation.ProjectCollaboration ThisprojectisacollaborativeendeavorbytheBioEdResearchHubwithmultipleresearchersintheCollegeofBiologicalSciences.AllfacultyacrossthethreedepartmentsofHumanHealth&NutritionalSciences,IntegrativeBiology,andMolecularandCellularBiologywillbeinvitedtocompletetheSETUtilizationSurvey,andfocusgroupswillbeheldwithfacultyfromallthreedepartments.BudgetHumanResources $7500Software FreeHospitalityandIncentives $200Conferencetravel&registration $4400DisseminationCharges(CJHEisanopen-accessjournal) FreeTOTAL $12,095HumanResources:Graduatestudentresearchassistant$7100:A1.0GraduatestudentGRAallocation.TheGRAallocationpaysfor70hoursofworkfromaGraduatestudent.These70hourscanbespreadover2semesters.Alternatively,$7500willpayafull-timeundergraduateCoopstudent(35hours/weekfor15weeks)foronesemester,including:Costtoemployer:9.25%($693.75),VacationPay:4%($300.00),Pay:$433.75/weekor$12.39/hour.Software:Nvivoqualitativeresearchanalysissoftware–availableforfreethroughGuelph’sCCSsoftwaredistributionsystem


6

HospitalityandIncentives$100Lunchforfocusgroups–3lunches,6-8participants,plustheadministrator;casuallunchthroughHospitalityServices(3pizzas@$18/each+30drinks@$1/eachplusincidentals)$100:Surveyreward-$100UofGgiftcardConferenceregistration,travelCOUconference,Torontofor2presenters(October2018):$120:mileagetoandfromtheConference(DoubletreeHilton)andparkingfee,$500:accommodationatDoubletreeHiltonfor2presenters,$850:conferenceregistrationfor2presenters.TOTAL:$1470STLHEconference(2019inWesternCanada(seeSTLHEwebsite),sincethe2018conferenceisscheduledforFebruaryandourstudywillnotbecompleteduntilsummerof2018)Foronepresenter:$500registration,$1000Airfare,$180groundtransportation,$250meals,$1000accommodations.Total:$2930ReferencesCentra,J.A.(2003).Willteachersreceivehigherstudentevaluationsbygivinghighergradesandlesscoursework?ResearchinHigherEducation44:495–518.Ewing,A.M.(2012).Estimatingtheimpactofrelativeexpectedgradeonstudentevaluationsofteachers.EconomicsofEducationReview:31:141-154.Gravestock,P.andE.Gregor-Greenleaf(2008).Studentcourseevaluations:Research,modelsandtrends.HigherEducationQualityCouncilofOntario:Toronto,ON.Kelly,M.(2012).Studentevaluationsofteachingeffectiveness:ConsiderationsforOntariouniversities.COUAcademicColleaguesDiscussionsPaper.http://cou.on.ca/wp-content/uploads/2015/07/Academic-Colleagues-Paper-Student-Evaluations-of-Teaching-Effectiveness.pdfLougheed,J.,Kirkland,J.B.,Newton,G.S.(2012)Usingbreakoutgroupsasanactivelearningtechniqueinalargeundergraduatenutritionclassroom.CanadianJournalfortheScholarshipofTeachingandLearning3:Article6.Marsh,H.W.(2007).Students’evaluationsofuniversityteaching:Amultidimensionalperspective.InThescholarshipofteachingandlearninginhighereducation:Anevidencebasedperspective,ed.R.P.PerryandJ.C.Smart,319–84.NewYork:Springer.Matos-Diaz,H.andRagan,J.(2010).Dostudentevaluationsofteachingdependonthedistributionofexpectedgrade?EducationEconomics18:317-330.Murray,H.G.(2005).Studentevaluationofteaching:hasitmadeadifference?AnnualmeetingoftheSocietyforTeachingandLearninginHigherEducation,Charlottetown,PrinceEdwardIsland.Racey,M.,Machmueller,D.,Field,D.,Kulak,V.,andNewton,G.(2016)Perceptionsanduseofsourcesofhealthknowledgebyadolescents.InternationalJournalofAdolescentHealth.Doi10.1515/ijamh-2016-0002.Wright,S.L.andJenkins-Guarnieri,M.A.(2012).Studentevaluationsofteaching:combiningthemeta-analysesanddemonstratingfurtherevidenceofeffectiveuse.Assessmentandevaluationinhighereducation37:683-699.

Appendix 5:

Financial Reporting System Summary from May 17, 2017

Unit Month End Summary by Object Last Update: 17-MAY-2017 Fiscal Period: 30-APR-2017 Program: 800081 - DEPARTMENT PROJECT #81Fund: 100 - General Operating Status: OpenDepartment: 015880 - DEAN - BIOLOGICAL SCIENCE - ASSOCIATE DEAN ACADEMIC

FRS 800081-DEPARTMENT PROJECT #81 Page 1 of 1

800081-DEPARTMENT PROJECT #81

Project Leader: DEPT ACCT Department: 988888

Program: 060 - DEPARTMENTAL PROJECTS Status: OpenExpenses Object Desc Mtd Ytd Comm Budget Amt Balance AmtPERSONNEL 61417-STUDENT LABOUR - TPT 2,400.96 7,052.38 0.00 0.00 -7,052.38

62306-BENEFIT ALLOCATION 222.09 652.34 0.00 0.00 -652.34Total PERSONNEL 2,623.05 7,704.72 0.00 0.00 -7,704.72 TRAVEL 62402-HOSPITALITY /

ENTERTAINMENT0.00 926.14 0.00 0.00 -926.14

62409-MILEAGE 0.00 151.29 0.00 0.00 -151.29Total TRAVEL 0.00 1,077.43 0.00 0.00 -1,077.43 OPERATING 62553-REGISTRATION 0.00 260.19 0.00 0.00 -260.19

62902-SCHOLARSHIP/AWARDS - GRADUATE

0.00 500.00 0.00 0.00 -500.00

63555-SERVICES-GRAPHICS, PRINTING, PHOTOGRAPHY

0.00 393.50 0.00 0.00 -393.50

Total OPERATING 0.00 1,153.69 0.00 0.00 -1,153.69 INTERNAL CHARGES

64361-PHOTOCOPYING COSTS 46.86 285.81 0.00 0.00 -285.8164452-SPACE CHARGES 0.00 50.00 0.00 0.00 -50.0064533-WORK ORDER CHARGES-1 0.00 31.50 0.00 0.00 -31.50

Total INTERNAL CHARGES 46.86 367.31 0.00 0.00 -367.31 INTERNAL RECOVERIES

65550-WORKSTUDY/URA RECOVERY -364.65 -364.65 0.00 0.00 364.65

Total INTERNAL RECOVERIES -364.65 -364.65 0.00 0.00 364.65 BUDGET ADJUSTMENTS

66003-BUDGET CARRYOVER - BUDGET ONLY

0.00 0.00 0.00 29,000.00 29,000.00

Total BUDGET ADJUSTMENTS 0.00 0.00 0.00 29,000.00 29,000.00 TOTAL Expenses 2,305.26 9,938.50 0.00 29,000.00 19,061.50TOTAL 800081-DEPARTMENT PROJECT #81 2,305.26 9,938.50 0.00 29,000.00 19,061.50

Budget Remaining = 19,061.50

coesp year one report final · the coesp has twitter, facebook and instagram accounts. the most...

Documents