evaluation of online courses
DESCRIPTION
TRANSCRIPT
Evaluation of Online Courses Using Social, Cognitive, and Teaching Presence:
Using the Community of Inquiry survey instrument for continuous improvement of collaborative, student-centered, active-learning blended or online courses
Diane OnoratoMercyhurst University, Erie, PAIndiana University of Pennsylvania, Indiana, PA
This session will use a wiki page entitled evaluatingacoi.wikispaces.com
Take time before we begin to log-on or create an account at Wikispaces.com
Directions are included in your handouts.
Session Objectives
Theoretical Framework
Define teaching, cognitive, and social presence using the Community of Inquiry (CoI) theoretical framework, and consider the dynamic inter-connectedness and the need for balance of each presence (Garrison, Anderson, & Archer, 2000).
Survey Data
Examine the CoI survey and data on students’ perceptions of the degree of each aspect of presence and expressed satisfaction. Consider additional survey items which provide enhanced student feedback.
Course Improvement
Propose elements of course design which target different aspects of presence with the goal of balancing perceptions of each type of presence and increasing satisfaction.
The Community of Inquiry (CoI) Theoretical Framework
“collaborative-constructivist… learning experience through the development of three interdependent elements – social, cognitive and teaching presence”
From CoI webpage http://communitiesofinquiry.com/model Figure adapted from (Garrison, Anderson, & Archer, 2000)
The CoI theoretical framework is intended only for collaborative course design with a focus on process.
“The reality is that the CoI theoretical framework with its collaborative constructivist perspective is essentially incompatible with traditional distance education approaches that value independence and autonomy over collaborative discourse in purposeful Communities of Inquiry” (Garrison, 2012, p. 251).
Teaching Presence (TP) is . . .
“the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes”
(Anderson, Rourke, Garrison, & Archer, 2001, p. 5). Figure adapted from (Garrison, Anderson, & Archer,
2000)
Examples of Indicators for Teaching Presence
Direct InstructionLeading discussion, summarizing ideas in play
Questioning and probing
Injecting of new knowledge
Providing models
Offering direction for technical support
Discourse Facilitation
Defining boundaries and ideas
Focusing the conversation on point or allowing exploration of topic
Motivating with attention to students’ efficacy
Clarifying ideas and shared experiences
Managing netiquette
Providing feedback and direction
Instructional Design &
OrganizationSetting curriculum, standards, targets, scaffolding
Creating and implementing course calendar
Designing methods, activities
Selecting media or tools
(Garrison, 2007; van Shie, 2008, retrieved from http://communitiesofinquiry.com/sites/communityofinquiry.com/files/concept-map.gif)
Social Presence (SP) is . . .
“the ability of participants to identify with the group or course of study, communicate purposefullyin a trusting environment, and develop personal and affective relationships progressivelyby way of projecting their individual personalities”
(Garrison, 2011, p. 34).
Figure adapted from (Garrison, Anderson, & Archer, 2000)
Examples of Indicators for Social Presence
Group Cohesion
Encouraging and including others
Using names and greetings
Using plural pronouns
Working together toward a goal; mentioning the goal in terms of each other’s participation and valued contributions
Open Communication
Continuing threads
Quoting and referring to each others’ statements when developing one’s own responses
Complimenting and agreeing
Expressing statements of personal growth or thought processes without risk
Affective Expression
Using humor, text styles, emoticons to express feelings
Expressing feelings
Expressing a sense of safety to speak without fear of unfair treatment by any member of community
Garrison, 2007; van Shie, 2008, retrieved from http://communitiesofinquiry.com/sites/communityofinquiry.com/files/concept-map.gif
Cognitive Presence (CP) is . . .
“the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse”
(Garrison, Anderson, & Archer, 2001, p. 11).
Figure adapted from (Garrison, Anderson, & Archer, 2000)
Examples of Indicators for Cognitive Presence
(Akoyol & Garrison, 2011)
Triggering Event
Puzzling about a topic
Recognizing the problem but not yet working
Exploration
Exchanging information in discussion
Recognizing divergent views
Offering suggestionsor brain-storming
Integration
Connecting ideas
Finding point of convergence
Offering tentative conclusions or solutions
Resolution
Testing or defending solutions
Applying new ideas to problem
Applying knowledge to other settings
Four Stages of the Practical Inquiry Model
Evaluate Functionality of a CoI
In his seminal work on communities of learners, Dewey warned against the dangers of allowing any one aspect of presence to overshadow another aspect. He wrote of the necessity to maintain balance between all groups of the community.
(Tolu, 2013)
“The CoI is a generic theoretical framework that must be viewed as a means to study collaborative constructivist educational transactions – be they in online, blended, or face-to-face environments.
The validation of this framework would also suggest that it can be used as a rubric to test for functioning communities of inquiry” (Garrison, 2012, p. 251, bolding not in the original text).
The degree of each aspect of presence shifts over time during the administration of a course; social presence may be stronger at first in a blended design (Akyol & Garrison, 2008, 2011b).
Cognitive
Social
Teaching
Importance of Teaching Presence
Ten years of research demonstrated that teaching presence is “a significant determinant of student satisfaction, perceived learning, and sense of community” (Garrison, Anderson, & Archer, 2010, p. 7).
“Despite a CoI’s learner-centered, socially rich experience, learners demand instructional presence and sharing of expertise from their instructors” (Shea, Hayes, Smith, Vickers, Bidjerano, & Picket, 2012).
Percentage of Dissatisfied Students: Average CoI Survey Scores for the Three Aspects
of Presence for Three Classes Undergoing TP Improvement Strategies for Three Terms
After employing strategies to improve teaching presence, the percentage of positive student perceptions of teaching presence in the course increased, and negative responses decreased.
(CoI survey retrieved from http://communitiesofinquiry.com/methodology )
2.50
7.50
12.50
17.50
22.50
20.33
13.33
15.50
12.00 11.6712.25
8.67
1.00
11.00
Improving Teaching Presence
Three Small Enrollment Course Designs Adjusted with Targeted Improvements to Teaching Presence
Over a Period of Three Terms (N = 25-35)
Perc
enta
ge V
ery/
Dis
satis
fied
Stud
ents
Teaching Presence Aver-age
Social Presence Average Cognitive Presence Aver-ageClass #1
FallClass #2 Winter
Class #3 Spring
Teaching Presence: Suggestions to Improve Design and Organization
1. Scavenger hunt: Create a synchronous or asynchronous scavenger hunt to orient students within the learning management system
2. Course calendar: Develop, distribute, and post a complete and thorough course calendar with specific directions, deadlines, and objectives/rubrics
3. Chronological template: Chronologically organize study materials and assignments; include TED talks, MOOC segments, and podcasts (consider a program design template)
4. Scaffolding strategies: Construct “soft” scaffolded assignments with lessening support
5. Clear directions: post audio, video AND text-based directions and technical support
6. Clear expectations: Visibly post minimum forum expectations and samples (take care to post samples that will not impair creative response to assignments)
7. Prompt feedback: Provide text feedback, audio feedback and combinations of audio and written feedback (Ice, Curtis, Philips, & Wells, 2007)
8. Coffee-shop talk: Create an “open” forum for off-topic conversations or help-needed question and answer
(Bonk & Zhang, 2008; Lehman & Conceição, 2010; Stavredes, 2011)
Teaching Presence: Suggestions to Improve Motivation and
Enhance Cognition
9. Options: Provide choices
10. Chunks: Present instruction and assignments in segments
11. Goals: Require students to set goals and track progress
12. Communication: Create clarification/muddy point forum
13. Application: Assign authentic case studies (later have students create their own case studies to share and to resolve)
14. Tension: Assign or take minority or unpopular view in discussions (may use surveys to determine minority view)
15. Reflection: Ask students to reflect on and connect new learning to life situations (Dennen & Bonk,
2007)
Suggestion to Enhance Teaching Presence:
Apply the Phases of Engagement Model
Phases of Engagement Model (Conrad & Donaldson, 2012)
Connect
Communicate
Collaborate Guide or assist small 3-5 person contract-based teams to complete tasks that build real world skills
Help pairs share and move to consensus on content-specific tasks including reflection
Manage large groups of students with one-on-one interaction on ice-breakers and finding commonalities
Co-Facilitate
Continue Empower learners to reflect, recognize personal growth, and apply content knowledge for their own career or daily needs
Act as fellow learner & guide while students take more responsibility in learner-initiated/led tasks
Suggested Activities to Improve Teaching Presence
(and Motivation)
Webliography Wiki: Glossary of terms, ideas, & links
Choice of Assignments: Address Same Objective
Wiki Pictionary:Describe/Define and Collaborate (Low-Risk Activity)
Template: Introduce new skill OR begin a task
Muddiest Point orVitals Check:Wiki or DB Forum
Pro/Con Wiki Debate: Defend both sides
Case Studies:Resolve Using Groups
Importance of Social Presence
Social presence is critical if learning occurs because of dialog and collaboration. Students must be present, feel present, engage with others in a safe environment for exploring and applying new knowledge (Garrison, 2011).
In years past, researchers established that it is possible to be feel and to project oneself socially in a virtual world (Gunawardina, Lowe, & Anderson, 1997; Rovai 2002).
Perceptions of social presence are linked to student satisfaction and perceived learning outcomes (Richardson & Swan, 2003; Swan, Garrison, & Richardson, 2009).
High levels of social presence were most necessary at the beginning of a course, but as the group bonds, the need for social presence lessens, and teaching and cognition are more in the foreground (Akyol & Garrison, 2008).
Percentage of Dissatisfied Students: Average CoI Survey Scores for the Three Aspects of
Presence for Three Classes Undergoing SP Improvement Strategies for Three Terms
(CoI Survey retrieved from http://communitiesofinquiry.com/methodology )
After employing SP improvement strategies, the percentage of positive student perceptions of SP increased, and negative responses decreased.
1 2 3
Series1 17.33 24.67 21.50
Series2 10.67 13.67 11.50
Series3 3.67 5.00 4.50
2.50
7.50
12.50
17.50
22.50
27.50
17.33
24.67
21.50
10.67
13.67
11.50
3.675.00 4.50
Improving Social Presence
Three Small Improvement Course Designs Adjusted with Targeted Improvements to Social Presence
Over a Period of Three Terms (N=25-35)
Perc
enta
ge V
ery/
Dis
satis
fied
Stud
ents
Class #1 FallClass #2 Win-terClass #3 Spring
Teaching Presence Aver-age
Cognitive Presence Aver-ageSocial Presence Average
Send a Welcome E-mail to Class
Social Presence: Suggestions
)
When students respond with dissatisfaction about measures of social presence, they are saying that they did not interactive effectively or feel comfortable online.
Class Roster Wiki: Self-Introduction with Photo
Post Rules: Discourse Netiquette
Greetings, and welcome to our ABCD-123 course. Join us for a scavenger hunt . . .
Try Brookfield’s Discussion Techniques with the CoI framework Face2Face or Synchronously (Armstrong & Thornton, 2012)
Importance of Cognitive Presence
Social and teaching presence create the path to engagement and reflection necessary for higher levels of cognition (Garrison, 2007; Swan, Garrison, & Richardson, 2009, bolding not in the original text).
“70% of the variance in the online students’ levels of cognitive presence, a multivariate measure of learning, can be modeled based on their reports of their instructors’ skills in fostering teaching presence and their own abilities to establish a sense of social presence” (Shea & Bidjerano, 2009).
Difficulties noted in attaining higher levels of cognition on discussion forums were attributed to types of questions posed (Garrison & Arbaugh, 2007; Garrison & Cleveland-Innes, 2005, bolding not in the original text).
Students proceed to higher levels of cognition when given a specific problem to resolve or provided with directives that demand integration and resolution, operation at top cognitive levels (Shea & Bidjerano, 2008, bolding not in the original text).
Percentage of Dissatisfied Students: Average CoI Survey Scores for the Three Aspects
of Presence for Three Classes Undergoing CP Improvement Strategies for Three Terms
Cognitive presence may be the most important aspect of a class, being as the goal of teaching is learning, but it may be the most difficult to establish partly due to its reliance on teaching and social presence
(Arbaugh 2007; Garrison, & Cleveland-Innes, 2005).
(CoI Survey retrieved from http://communitiesofinquiry.com/methodology )
1 2 3
Series1 14.67 11.00 11.50
Series2 9.00 9.00 9.50
Series3 6.67 4.33 0.00
1.00
3.00
5.00
7.00
9.00
11.00
13.00
15.00 14.67
11.0011.50
9.00 9.009.50
6.67
4.33
0.00
Improving Cognitive Presence
Three Small Enrollment Course Designs Adjusted with Targeted Improvements to Cognitive Presence
Over a Period of Three Terms (N= 25-35)
Perc
enta
ge V
ery/
Dis
satis
fied
Stud
ents
Teaching Presence Average Social Presence AverageCognitive Presence Aver-
ageClass #1 Fall
Class #2 Winter
Class #3 Spring
1. Repeated practice: Provide several opportunities for practice on a desired objective: Increase the number of wikis in the course to improve social presence and collaboration. See (West & West, 2009) 2. Multiple demonstrations and sample target papers: Improved the process for Peer Review and Case Study responses using a combination of tasks: DB (exploration) > Wiki (integration) > Journal (reflection)
3. Varied and connected discussion tasks: Structured, scaffolded, role play, debate mixed methods in discussions (Darabi, Arrastia, Nelson, Cornille, & Liang, 2011)
4. Posed divergent or minority points of view: Assigned or argued in discussion, wiki, & journal topics to challenge critical thinking or reflective responses (Dennen & Bonk, 2007)
5. Assigned reflection: Various activities for Journals & Wiki for personal and shared reflection (Conrad & Donaldson, 2012)
Suggestions to Improve Cognitive Presence
Journal or Blog Discussion Board Wiki
Design Features
Author centered reflection, author-owned, static, linear, comments at the end, static , chronological, single post
Topic centered and comment driven threads and postings, static, collaborative, linear
Document centered, collaborative creation, constantly changing edited by entire group; all interactive; nonlinear
Graphic Illustration
Possible Uses
Reading responses, notes, reflections on topics, summaries, evaluations, rough drafts of projects
Self-introductions, role play, exploration, case studies, debate, sharing of ideas & some media
Creating documents, projects, sharing media or resources, building pages, analysis, brainstorming
Suggestion to Improve Cognitive Presence
Employ Combinations of Interactive Tools
in a Series of Related Assignments
Collaboration Activity
Click icon to add pictureParticipants are grouped in manageable-sized groups, preferably 3-5. Groups will address one of the forms of presence: Teaching, Cognitive, or Social.
Each group will find its own previously created and labeled Wikispace.com page on ImprovingaCoI.
To build the wiki page, each group will describe activities which could enhance the aspect of presence that they have been assigned.
After the conference ends, your presenter will edit and combine all entries to form one wiki page from the conference which will be accessible to you and editable by you for as long as Wiki Spaces website exists.
Go to Improvingacoi.wikispaces.com
Login (make an account)
Find your group page. Click on it to open it for editing. Add ideas for activities to consider.
Troubles? Check the How to Use Wikispaces Handout in your packet of materials.
Collaboration: Teaching Presence
Contribute ideas that you have regarding activities/practices that could improve Teaching Presence to your group wiki page.
Instructional Design Strategies
Facilitative Discourse TechniquesDirect Instructional Tools
Collaboration: Social Presence
Contribute ideas you have regarding activities/practices that could improve Social Presence to your group wiki page.
Affective Expression
Open Communication
Group Cohesion
Collaboration: Cognitive Presence
Contribute ideas that you have regarding activities/practices that could improve Cognitive Presence to your group wiki page.
Triggering Exploration Integration Resolution
“The findings from this study underscore the importance of quality of teaching presence and social presence, but also the fact that rankings of cognitive gains are likely to depend on the characteristics the learner bring to the learning environment when quality of teaching or the quality of social interaction are low or inadequate”
(Shea & Bidjerano, 2012, p. 326)
New Research on CoI Survey Items:Learner Presence & Self-Regulated
Learning
Shea & Bidjerano, 2010, 2012
Teaching Presence
Social Presen
ce
Cognitive
Presence
Structuring environment
Setting goals
Managing time and tasks
Seeking help
Self-evaluating performance
Learning Presence
Knowledge of Cognition
Monitoring of Cognition
Regulation of Cognition
Current research: the construct of metacognition may be related to the aspect of cognitive presence in a CoI
See the metacognition survey items which have been recently introduced for research development in the handouts for the session.
New Research on CoI Survey Items: Metacognition as a Factor
Adapted from (Garrison & Akyol, 2013; see also Akyol & Garrison, 2011a, 2011b)
Cognitive Presence
Emotional Presence: “the outward expression of emotion, affect, and feeling by individuals and among individuals in a community of inquiry, as they relate to and interact with the learning technology, course content, students, and the instructor” (Cleveland-Innes & Campbell, 2012, p. 283).
New Research on CoI Survey Items:Emotional Presence
Emotional Presence
Emotions could act on all aspects of presence.
Novice students must learn to act in a virtual environment; instructors who are sensitive to emotional presence present interventions to allow students to adjust and learn
(Cleveland-Innes & Campbell, 2012).
References (See
handout)
References Akyol, Z., & Garrison, D. R. (2008). The development of a community of inquiry over time in an online course: Understanding the progression and integration of social, cognitive, and teaching presence. Journal of Asynchronous Learning Networks, 12(3-4), 3-22.Akyol, Z., & Garrison, D. R. (2011a). Assessing metacognition in an online community of inquiry. The Internet and Higher Education, 14(3), 183-190.Akyol, Z., & Garrison, D. R. (2011b). Understanding cognitive presence in an online and blended community of inquiry: Assessing outcomes and processes for deep approaches to learning. British Journal of Educational Technology 42(2), 233-250.Arbaugh, J. B. (2007). An empirical verification of the community of inquiry framework. Journal of Asynchronous Learning Network, 11(1), 73-85.Armstrong, A., & Thornton, N. (2012). Incorporating Brookfield's discussion techniques synchronously into asynchronous online courses. Quarterly Review of Distance Education, 13(1), 1-9. Bonk, C. J., & Zhang, K. (2008) Empowering online learning: 100+ activities for reading, reflecting, displaying, and doing. San Francisco, CA: Jossey-Bass, Wiley.Caspi, A., & Blau, I. (2008). Social presence in online discussion groups: Testing three conceptions and their relations to perceived learning. Social Psychology of Education, 11(2), 323-346.Cleveland-Innes, M. & Campbell, P. (2012). Emotional presence, learning, and the online learning envi-ronment. The International Review of Research in Open and Distance Learning, 13(4), 269-92. Community of Inquiry Survey (2011). Retrieved at http://www.communitiesofinquiry.com/ Conrad, R. M., & Donaldson, J. A. (2012). Continuing to engage the online learner: More activities and resources for creative instruction. San Francisco, CA: Jossey-Bass, Wiley.Darabi, A., Arrastia, M. C., Nelson, D. W., Cornille, T., & Liang, X. (2011). Cognitive presence in asynchronous online learning: A comparison of four discussion strategies. Journal of Computer Assisted Learning, 27(1), 216-227. Dennen, V. P., & Bonk, C. J. (2007). We’ll leave the lightp on for you: Keeping learners motivated in online courses. In L. A. Tomei (Ed.). Online and distance learning: Concepts, methodologies, tools, and applications (pp. 64-76). Hershey, PA: Idea Group Global.Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Boston, MA: D. C. Heath.Garrison, D. R. (2007). Online community of inquiry review: Social, cognitive, and teaching presence issues. Journal of Asynchronous Learning Networks, 11(1), 61-72.Garrison, D. R. (2011). E-Learning in the 21st century: A framework for research and practice (2nd ed.). London: Routledge/Taylor and Francis.Garrison, D. R. (2012). Article review: Social presence within the community of inquiry framework. International Review of Research in Open & Distance Learning, 13(1), 250-253. Garrison, D. R., & Akyol, Z. (2013). Toward the development of a metacognition construct for communities of inquiry. Internet and Higher Education 17(2), 84–89.Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2–3), 87–105.Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7–23.Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. Internet and Higher Education 13(1-2), 5–9.Garrison, D.R. & Arbaugh, J.B. (2007). Researching the community of inquiry framework: Review, Issues, and Future Directions. The Internet and Higher Education, 10(3), 157-172. Garrison, D. R. & Cleveland-Innes, M. (2005). Facilitating cognitive presence in online learning: Interaction is not enough. American Journal of Distance Education, 19(3), 133-148.
Garrison, D. R., Cleveland-Innes, M., & Fung, T. (2010). Exploring causal relationships among cognitive, social and teaching presence: student perceptions of the community of inquiry framework. The Internet and Higher Education, 13(1–2), 31–36. Gunawardena, C., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431.Ice, P., Swan, K., Diaz, S., Kupczynski, L., & Swan-Dagen, A. (2010). An analysis of students’ perceptions of the value and efficacy of instructors’ auditory and text-based feedback modalities across multiple conceptual levels. Journal of Educational Computing Research, 43(1), 113-134.Lehman, R. M., & Conceição, S. C. O. (2010). Creating a sense of presence in online teaching. San Francisco, CA: Jossey-Bass.Richardson, J. C., & Swan, K. (2003). Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7(1), 68-88. Richardson, W. (2010). Blogs, Wikis, Podcasts. Thousand Oaks, CA: Corwin/Sage.Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14(2), 51-70.Rovai, A. P. (2002). Development of an instrument to measure classroom community. The Internet and Higher Education, 5(3), 197-211.Shea, P., & Bidjerano, T. (2008). Measures of quality in online education: An investigation of the community of inquiry model and the net generation. Journal of Educational Computing Research, 39(4), 339-361.Shea, P., & Bidjerano, (2009). Community of inquiry as a theoretical framework to foster "epistemic engagement" and "cognitive presence" in online education. Computers in Education, 52(3), 543-553.Shea, P., & Bidjerano, T. (2010). Learning presence: Towards a theory of self-efficacy, self-regulation, and the development of a communities of inquiry in online and blended learning environments. Computers and Education 55(4), 1721-1731.Shea, P., & Bidjerano, T. (2012). Learning presence as a moderator in the community of inquiry model. Computers & Education 59(2), 316-326.Shea, P., Hayes, S., Smith, S. U., Vickers, J., Bidjerano, T., & Picket, A. (2012). Learning presence: Additional research on a new conceptual element within the community of inquiry (CoI) framework. Internet and Higher Education, 15(2), 89–95.Stavredes, T. (2011). Effective online teaching: Foundations and strategies for student success. San Francisco, CA: Jossey-Bass.Swan, K., Garrison, D. R., & Richardson, J. (2009). A constructivist approach to online learning: The community of inquiry framework. In C. R. Payne (Ed.). Information technology and constructivism in higher education: Progressive learning frameworks (pp. 43-57). Hershey, PA: IGI Global.Swan, K., Shea, P., Richardson, J., Ice, P., Garrison, D. R., Cleveland-Innes, M., & Arbaugh, J. B. (2008). Validating a measurement tool of presence in online communities of inquiry. E-Mentor, 2(24), 1-12. Tolu, A. T. (2013). Creating effective communities of inquiry in online courses. Procedia: Social and Behavioral Sciences, 70(1), 1049-1055.Van Shie, J. (2008). Community of inquiry visual. Community of Inquiry. Retrieved from http://communitiesofinquiry.com West, J. A. and West, M. L. (2009). Using wikis for online collaboration. San Francisco, CA: Jossey-Bass, Wiley.
Session Evaluation