exploring connectivist massively open online course (cmooc) microblogging data through a student...
DESCRIPTION
This poster will be presented at the Educause ELI Annual Meeting in Anaheim, California February 9-12, 2014. It is accompanied by a detailed handout, which is also available.TRANSCRIPT
Exploring Connectivist Massively Open Online Course (cMOOC) Microblogging Data through a Student Assessment Lens
Laura Gogia
Abstract
In 2014 six Virginia Commonwealth University (VCU) faculty designed and
implemented an eight week-long summer pilot course called Digital Engagement:
Living the Dreams, Digital Investigations and Unfettered Minds, or UNIV 200:
Inquiry and the Craft of Argument. While most of the course took place on open
blogging platforms, an associated Twitter community emerged around the hashtag
#Thoughtvectors. Intentionally initiated by course instructors, #Thoughtvectors
Twitter activity was captured in a Twitter Archiving Google Spreadsheet (TAGS) and
informally shared with all participants through the course website. This study explores
the TAGS data through the lens of student assessment. It finds that while it is possible
to create dashboards for student assessment from TAGS data, optimal use might
require adapting archiving spreadsheets to capture data specific to formalized
connected learning objectives and activities.
Study Purpose
This study was an initial step in a research agenda that aims to advance understanding
of how to best document connected learning in formal higher education settings.
Research questions included:
• What data are captured by Twitter Archiving Google Spreadsheets (TAGS)?
• How might data collected in this format be used to describe the microblogging
behaviors of learning communities, their subgroups, and individual participants?
• Is it possible to develop a dashboard for student assessment from data captured in
Twitter Archiving Google Spreadsheets? If so, what are its limitations?
Methods
A Twitter Archiving Google Spreadsheet (TAGS; Hawksey, 2013) was used to capture
#Thoughtvectors activity from the Twitter Application Programming Interface (API).
The spreadsheet was monitored and maintained from the first use of the
#Thoughtvectors hashtag until one month after the formal course (UNIV 200) had
ended.
The data captured in TAGS were transferred to Microsoft Excel where duplicates and
errors were removed. Participants were sorted into four subgroups for more detailed
analysis. A comprehensive quantitative content analysis of the captured data was
performed to extract information related to mentions, retweets, and links. Descriptive
statistics were generated through Microsoft Excel and social network analysis
performed with the addition of NodeXL (Social Media Research Foundation, 2014).
4. Others480 other individuals
tweeted or were
mentioned along with the
#Thoughtvectors hashtag
3. Open Participants
35 non-students and
over a dozen VCU
staff formally engaged
in at least some UNIV
200 learning activities.
Of these, 28
participated in
#Thoughtvectors-
related Twitter
activity..
2. Instructors#Thoughtvectors was
facilitated by six
faculty who possessed
a total of eight Twitter
accounts.
1. Students95 VCU students
enrolled in UNIV 200;
of those, 68 were
identified as
participating in course-
related Twitter
activities. activity.
TheThoughtvectors
Community
What’s Possible with TAGS Data?
12%
31%
32%
25%
Who's Tweeting?
Students Instructors Open Participants Others
24%44%
14%
73%
40%
76%56%
86%
27%
60%
Are Tweets Retweets?Yes No
42%
12% 13% 4% 12%
18%
17%27% 39% 26%
28%
33%
34% 22% 31%
12%
37%26%
35% 30%
Who’s Being Mentioned?Students Instructors Open Participants Others
30%
57% 61% 62% 56%
70%
43% 39% 38% 44%
Do Tweets Include Links?
Yes No
Developing a Hypothetical Assessment StrategyConclusions
1. It is possible to use TAGS data to develop dashboards for the assessment of
student microblogging behaviors.
The data collected from TAGS are appropriate for descriptive statistics and data
visualizations consistent with formative and summative assessments of group and
individual student performance.
2. It is better to tailor data archiving spreadsheets to the pedagogical objectives of
the microblogging activity.
In this case researchers hoped to capture the incidence of student “connection-
making” because of the pedagogical importance placed on making connections
across space, time, and disciplines in connected learning environments.
Information regarding mentions, links, and retweets was available through TAGS,
but it had to be mined from text cells in a time-consuming content analysis. Such
an analysis would not be feasible in an authentic, real-time educational context of
similar size. Ideally archiving spreadsheets could be designed to mine much of this
information automatically.
3. If assessment strategies around connection-making are to be implemented,
instructors must send clear messages to their students regarding the usefulness
of microblogging as a connection-making tool.
Individuals in the #Thoughtvectors community tweeted in very different ways.
Heterogeneity in student use might have been related to the lack of consistent and
formalized pedagogical messaging around the use of Twitter for class-related
activity. If microblogging is to be seen as a powerful learning tool that affords (1)
connection of ideas across space, time, and disciplines; (2) strategic navigation
through distributed discourse; and (3) signal amplification; and if students are to
practice using it as such, explicit pedagogical messaging around these goals should
be in place before the students’ ability to perform such tasks are assessed.
Next Steps
• Gaining a better understanding of how students and faculty use social media tools in
connected learning environments.
• Building evidence to suggest certain digital practices or patterns of practice (e.g.
mentioning, linking) promote the sort of digital engagement, digital literacies, and
connection-making valued in connected learning environments.
• Developing archiving spreadsheets specific to the purpose of documenting student
learning in connected learning environments.
• Developing meaningful, scalable, and flexible assessment systems for connected
learning that may incorporate the data from digital artifact archiving spreadsheets
similar to or based on TAGS.
Student A
#Tweets: 85
#Mentions: 66
#Links: 11
#Retweets: 64
In-Degree Centrality: 5
Out-Degree Centrality: 17
Betweeness Centrality: 3110
Student B
#Tweets: 24
#Mentions: 5
#Links: 12
#Retweets: 6
In-Degree Centrality: 5
Out-Degree Centrality: 9
Betweeness Centrality: 1108
Student C
#Tweets: 0
#Mentions: 0
#Links:0
#Retweets: 0
In-Degree Centrality: 1
Out-Degree Centrality: 0
Betweeness Centrality: 0
Characterizing Individual Student Participation through TAGS Data Analysis
Student Twitter Activity
Whole Community
Twitter Activity