analyzing educators online interactions: a framework of...
TRANSCRIPT
This article was downloaded by: [205.178.115.194]On: 02 December 2014, At: 13:23Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Learning, Media and TechnologyPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/cjem20
Analyzing educators’ onlineinteractions: a framework of onlinelearning support rolesDenise C. Nacua, Caitlin K. Martina, Nichole Pinkarda & Tené Graya
a College of Computing and Digital Media, DePaul University,Chicago, IL 60604, USAPublished online: 07 Nov 2014.
To cite this article: Denise C. Nacu, Caitlin K. Martin, Nichole Pinkard & Tené Gray (2014):Analyzing educators’ online interactions: a framework of online learning support roles, Learning,Media and Technology, DOI: 10.1080/17439884.2015.975722
To link to this article: http://dx.doi.org/10.1080/17439884.2015.975722
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Analyzing educators’ online interactions: a framework of onlinelearning support roles
Denise C. Nacu*, Caitlin K. Martin, Nichole Pinkard and Tené Gray
College of Computing and Digital Media, DePaul University, Chicago, IL 60604,USA
(Received 21 January 2014; accepted 7 October 2014)
While the potential benefits of participating in online learning communitiesare documented, so too are inequities in terms of how different populationsaccess and use them. We present the online learning support roles (OLSR)framework, an approach using both automated analytics and qualitativeinterpretation to identify and explore online teaching roles. We analyzethe OLSR using data logs of iRemix, the online component of the DigitalYouth Network, a face-to-face and online program for urban youth in under-served communities. In three middle-school classrooms, six educators usediRemix most to interact with individual students, especially as a windowinto their work. Although many roles were documented, few were playedregularly, raising questions about design and intentionality. To address par-ticipation inequities, our results suggest that the OLSR and related data canbe used to support productive practice improvement conversations amongeducators and to inform the design of online social learning networks.
Keywords: online teaching roles; online data analysis; social networkingplatforms; blended learning; data visualizations
1. Introduction
As online learning environments rapidly become part of the education land-scape, new methods to make sense of what is occurring in these environmentsare critical. Although Web-based systems can automatically collect enormousamounts of use data, there is much we as a field need to understand aboutwhat to examine and how to analyze what we see to reveal patterns of learning.In this paper, we explore how educators provide supports for student learningin online contexts by presenting the online learning support roles (OLSR) fra-mework, an approach using both automated analytics and qualitative interpret-ation. The framework builds on previous research that underscored the criticalrole of adult mentors in providing supports needed to take advantage of digitallearning opportunities in a school-based context (Barron et al. 2014). In this
© 2014 Taylor & Francis
*Corresponding author. Email: [email protected]
Learning, Media and Technology, 2014http://dx.doi.org/10.1080/17439884.2015.975722
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
study, we examined use data from educators and students in three middle-school English Language Arts classrooms with a focus on digital literacies,looking primarily at the moves made by educators in the online environment.We argue that the OLSR framework offers a method to make sense of dataabout educators’ online behavior and their interactions with learners. For organ-izations that are exploring the integration of online learning environments intotheir models, this framework provides a lens to inform data-driven professionaldevelopment efforts and instructional improvement. For researchers of learningenvironments and massively open online courses, we offer a framework forcoding use data that can reveal patterns of specific educator actions andstudent learning. It can serve as a guide for designers in creating software fea-tures that intentionally enable users to play certain learning support roles forlearners. In describing both the method and organizational context of the frame-work’s development, this paper also contributes an example of research thatattends to multiple dimensions of data use (Coburn and Turner 2011).
1.1. Online learning environments
In 2008, an estimated 75% of public schools had one or more students enrolledin a fully online or blended course, and 66% estimated that their online offer-ings would grow (Picciano and Seaman 2009). These ‘blended’ models aimto leverage the advantages of face-to-face, supervised settings and onlinetools, while providing students with some element of control over time,place, path, and pace of their own learning (Staker and Horn 2012). Open-source platforms offering teachers a customizable online classroom space areincreasingly popular, including Moodle and Edmodo.
Some of these environments created intentionally for K12 learning incorpor-ate a variety of interactive and social features, allowing students to go beyondretrieval and submission of information, moving to collaborative production,processing, critiquing, and understanding. We refer to these spaces as onlinesocial learning networks. These sites are intentionally designed for learning(at home, in school contexts, in after-school or summer programs) while imple-menting affordances of more informal networks that have been shown to encou-rage and spark informal interest-driven learning (e.g., gaming sites and specialinterest forums, Gee 2005; Jenkins et al. 2009).
Despite the rapid scaling up of these environments and potential outcomes,research about learning through massively open online courses and blendedlearning opportunities is still somewhat limited, and has mostly focused onmetrics that are aligned with more traditional, lecture-based instructionalmodels, reporting findings related to low course completion rates, viewing ofcourse content, and grades received (e.g., Ho et al. 2014; Perna et al. 2013).Several researchers have pointed out the inadequacy of these approaches andhave called for the conceptualization of new learning analytics that are more rel-evant to social learning models that emphasize personalization, collaboration on
2 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
projects, and learning as participation and becoming (DeBoer et al. 2014; Eisen-berg and Fischer 2014; Pea 2013).
Within the more informal networked community contexts, less traditionalresults have been documented that are believed by some to be importantpieces of the twenty-first century skillset, including managing information;directing learning pathways; collaborating on projects, discussion, and critiquearound common artifacts; and building collective intelligence (Barron et al.2014; Jenkins et al. 2009).
1.2. Building equitable opportunities through online mentorship
Although opportunities for online learning and networks can extend learningenvironments out from school or clubs into home and communities, researchhas repeatedly shown that only a subset of people participate in ways thatcan lead to these positive outcomes (Hargittai and Walejko 2008; Jenkinset al. 2009). Today’s digital divide is identified not in terms of access to tech-nology, but in terms of who has exposure to experiences that increase digitalliteracy and further opportunities, support, and knowledge. Factors such associoeconomic status, user skill, gender, and a knowledgeable social networkare important pieces of this system. Parents and other home connections canbe highly instrumental in guiding learning and participation (Barron et al.2009), but for many families, parent technological knowledge and co-participation are not the norm (Warschauer and Matuchniak 2010). There isevidence, however, that informal spaces such as after-school programs and com-munity centers can provide knowledgeable human social networks for youngpeople who can supplement those provided by parents (Barron et al. 2014).The National Research Council’s (Eccles and Gootman 2002) description offeatures of settings that have value for positive youth development highlightsthe importance of creating programs that capitalize on building connectionsbetween youth and others who can support them. Evidence has also beenfound to support the potential of positive mentoring relationships on youths’social-emotional, cognitive, and identity development (Rhodes et al. 2006).Online spaces have the potential to support and supplement these face-to-faceopportunities, making connections to mentors possible outside of the boundariesof scheduled programming. Consequently, the effective design and implemen-tation of online social learning networks in formal and informal contexts havegreat potential for creating equitable learning opportunities for all youth.
If these types of networks are to play a significant role in moving towardequitable participation, a solid understanding of how adults operating in theseenvironments can effectively support youth learning is needed. Fortunately,educators are increasingly joining social networking sites for personal, pro-fessional, and instructional use (MMS Education 2012). This willingness toengage is promising; however, we cannot assume that educators are preparedfor the challenge of navigating a new environment that may have different
Learning, Media and Technology 3
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
rules and roles, and require new pedagogical approaches from those of tra-ditional face-to-face classrooms. Recent surveys have suggested that themajority of contemporary teachers have serious concerns about the relationshipbetween new technologies, including social networking sites, and students’ability to learn (Purcell et al. 2012; Rideout 2012). This situation finds manyteachers who may have misgivings about student use of media and the internetnavigating their own role and approach within these novel environmentswithout guidance or support. The enormous amount of data that are collectedfrom these online environments presents a great opportunity to understandteacher behavior and student learning. This research aims not only to contributeto research and design agendas, but also to provide guidance that practitionersand other stakeholders (e.g., students and program leaders) need to make effec-tive use of online environments to support student learning.
2. The Digital Youth Network
Our research is situated in the context of the Digital Youth Network (DYN), amix of school, after-school, and online spaces designed to provide youthwith opportunities to develop traditional and digital literacy within a supportiveand interactive community. Some DYN implementations are organized intoformal school classroom instruction, while others are more informal drop-inopportunities in after-school and library spaces. Established in 2005, DYNefforts have focused on designing and integrating digital media curriculum,social practices, professional development, and online environments towardthe goal of developing digitally savvy urban minority youth, with a focus onthose from undeserved communities. The program has also been particularlyaware of designing programs and environments that engage girls as well asboys, purposefully building skills, expertise, and experiences for youngfemales in an effort to counter the persistent national trend of gender disparitiesin computing fields (Dubow 2011; Margolis and Fischer 2002).
DYN provides professional development for the adult mentors it employs inits programs. DYN mentors are typically hired with expertise as artists anddigital media professionals, and are expected to continually develop skills asteachers, artists/technicians in their disciplines, and as adults capable of build-ing relationships with youth. Professional development activities organized formentors are varied and consist of face-to-face meetings, online collaborativelearning, and self-paced learning all grounded in topic-driven modules (e.g.,blogging, critique and feedback, and social learning networks). Professionaldevelopment is designed to help mentors encourage youth to pursue interests,provide useful feedback, exercise fairness when critiquing student work, andmanage professional relationships.
Previous research within a school-based implementation ofDYN highlightedthe central position of DYN educators for student’s technology-related learning,experiences, and development, more so than adults at home. Educators were
4 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
aware of their critical position within DYN and revealed complexities and ten-sions that went hand in hand with this status, including difficulty finding abalance between being a teacher and an evaluator alongside being a friendand a mentor (Barron et al. 2014).
2.1. iRemix social learning network
iRemix is theDYN online environment, and instance of an online social learningnetwork. iRemix has become increasingly central as programming scaled up andface-to-face time has become more distributed. The closed social network onlyallows youth learners who have participated in DYN programming to join.iRemix has an interface and functionality similar to popular online socialnetwork communities; users are able to create a profile page, link to peers,and share and critique work (see Figure 1).
Participation by mentors is a key aspect of the system’s design. Adult edu-cators (including DYN mentors and school-based teachers) purposefully useiRemix to seed assignments, content, comments, and posts that promotedesign, tinkering, challenges, questioning, learning, and meaningful conversa-tion. They can also use iRemix functionality to evaluate student work. Particu-larly useful for blended classroom settings, the platform incorporates acurriculum management system to support instructors in building out com-ponents of the site such as assignments, linked resources, and assessments,around their specific face-to-face contexts.
Figure 1. Screen capture of a mentor’s profile page in the iRemix online social learn-ing network.
Learning, Media and Technology 5
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
Students using iRemix in formal classes are expected to take part in therequired online activities, such as posting assignments, participating in discus-sions, and responding to feedback. Students are also encouraged to use iRemixas an extension of classroom activities, and to develop as individual and collec-tive community participants over time and outside of required time. There areopportunities for students to view and comment on each others’ work, ask ques-tions of mentors and their peers, use learning resources embedded on the site,and create and join groups and discussions around common work, interests,or topics. The design of the environment is particularly focused on supportingyouth in creating, sharing, and talking about digital artifacts such as videos,photos, podcasts, games, and graphics.
While iRemix can be used intentionally to lead students toward certain learn-ing goals in more formal learning environments, it can also serve as a platform ininformal environments that are interest driven. As such, we view iRemix as a toolto build affinity spaces (Gee 2005) in which participants are engaged not onlywith the creation and sharing of artifacts but also with people, a connected com-munity of participants, experts, and mentors. Thus, while adults hold key pos-itions in the network and are important for structuring and facilitating learningin iRemix, the system is designed to foster a student-driven and student-centeredcommunity that reflects student voice and interests. Students influence patternsand push for new information and behaviors, as they start new groups and dis-cussions, share particular media, or create memes of commenting practice.
2.2. Using iRemix data in professional development: a history
Using data collected from classroom observations, interviews, surveys, and theiRemix platform to inform professional development and mentor practice hasbeen an ongoing strand of work pursued by DYN (Barron et al. 2014; Nacuet al. 2012). As the number of DYN mentors has grown and iRemix has becomemore integrated into activities, the need to make effective use of iRemix datalogs has become more pressing. The log data captures user actions in thesystem, such as logins, viewing artifacts, posting new artifacts, and posting com-ments. This log also records how actions occur between types of actors in thesystem – for example, documenting when an educator makes a commentrelated to a student’s post, and when students are responding to each other.
Our current framework for making sense of the iRemix data is informed byearlier efforts to make these data useful for understanding student and mentoractions in the online environment. In these early attempts, we created data dis-plays using the categories of participation (e.g., number of logins in a giventime period, and number of minutes spent on the site), consumption (viewsof posted artifacts), contribution (sharing artifacts and commenting), and pro-duction (posting original work). While mentors had expressed an appreciationand appetite for visualizations focused on student participation, creating rep-resentations of mentors’ online actions that were easily accepted and used by
6 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
mentors was a more difficult endeavor. Some of the displays were used duringprofessional development conversations with the goal of developing sharednotions of effective online practice for mentors that could provide a foundationfor practice improvement. While this approach highlighted differences in theuse of iRemix among mentors and across contexts, it did not stimulate therich conversation hoped for by the design and development team. Instead,the visualizations created tension within the mentor community becausementors felt that the categorization did not adequately reflect their onlineactivity. In particular, they remarked that seeing the data categorized by ‘count-ing’ participation, contribution, production, and consumption logs tended tovalue doing ‘more’ online (e.g., more logins and most comments), withoutreflecting the substance or intention of their online actions.
They also resisted the concept of identifying a single online behavior patternthat would represent best practice. Because mentors had recently started to bepaid for hours worked supporting youth online, this concern that the data didnot properly reflect their efforts was especially sensitive and important tobetter understand. Instead of professional development conversations focusingon reflection and improvement, mentors expressed concerns related to oversightand evaluation. The questions raised focused on quantification over quality –
for example, how many minutes are mentors required to spend online? Howmuch and how often are mentors expected to post comments?
We view the insights gained from these earlier attempts at using visualiza-tions of iRemix data as key to this research, because they surfaced the need tobetter reflect mentors’ intentions and to describe mentors’ actions in a waythat allows for more variability and complexity inherent in their work. Theyalso highlighted the need to build an analytical framework that is more groundedin mentor perspectives and the larger organizational context (Coburn and Turner2011). Thus, our subsequent efforts to analyze and represent online social learn-ing network data in useful visualizations more intentionally involved mentors incollaborative analysis and development, which is described next.
3. Methods
Given the central role of educators and the increasing reliance on interactionswithin the online iRemix space, we persisted in attempts to document educatoractivity online to learn more about what was happening and reflect on the poten-tial for redesign. While our current work examines the larger blended learningcontext, looking at both face-to-face and online actions, here we focus ourobservations on the online space.
3.1. Coding framework and analysis
Our framework for educator roles online is based on the work by Barron et al.(2009) that used case studies of eight highly technologically engaged middle-
Learning, Media and Technology 7
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
school students from Silicon Valley to identify the influential roles played byparents in their learning about technology and computers. These roles includedTeacher, Collaborator, Learning Broker, Resource Provider, Learner, Non-technical Support, Employer, and Monitor. Barron et al. (2014) suggestedthat these roles were not context specific and could be applied to other typesof learning and environments. Barron extended these roles to adult mentors informal and informal school-based settings. A main interest of this study wasto extend the basic framework of parent roles to online learning environments.As a strategy for grounding the extension and revision of the original frame-work, we collaborated with mentors who had had significant experienceworking in formal learning contexts integrating the use of an online social learn-ing network. Together with DYN mentors as our design partners, we examinedand discussed these roles in the context of online environments. First, DYNmentors shared verbal examples and reflections from their own face-to-faceand online interactions with students across various contexts. Next, archiveddigital histories of DYN mentors were explored on iRemix, excerpting screen-shot examples and discussing as a group. Through several cycles of collabora-tive review and discussion, and category refinement, a set of learning supportroles that apply to adult educators within the DYN OSLN were defined. Theseincluded the existing roles of Learning Broker, Learner, Monitor, and ResourceProvider, and the new roles of Audience, Encourager, Evaluator, SocioculturalFriend, Model, and Promoter. In addition, we adapted the role of Teacher anddefined an Instructor role. The full set of roles is summarized in Table 1. TheBarron et al. role of Collaborator (play a role in a group project alongsideyouth) was an important face-to-face interaction discussed by mentors, but wehave not yet found evidence of this in the online environment.
iRemix was intentionally designed to collect and report use data, includinglogins and particular actions [including read (viewing work or posts), create(posting media or starting a blogpost), comment (adding a comment to postedwork or discussion), and rate (formally assessing posted work)]. Authors deter-mined what pre-existing logged actions could be automatically coded as a roleand what needed to be interpreted by hand. For example, while all instances ofeducators reading a student post were automatically coded for the role of Audi-ence, educator comments on student work and educator posts to the site wereautomatically logged but needed content interpretation to determine the typeof role evident.
We utilized a method of automatic and manual coding to enable quantitativeand qualitative analyses of the iRemix use data (Chi 1997). Two authors codedthe online educator activities on iRemix in a school-based, classroom contextduring a three-month class unit, from 17 March through 20 June 2012. Therewere 1864 actions logged as adult actions for students during this time; 1469actions were automatically coded and 395 actions were determined to needhuman interpretation. Due to the nature of comments and posts, actions suchas those could be coded as more than one role. Two authors coded a subset
8 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
(25%) of the actions that needed human interpretation (educator comments andposts) and reached a reliability of 86.4%. Coding discrepancies were discussedand agreements about coding and modifications to the coding scheme werereached to mediate ambiguity in the coding scheme going forward. A File-Maker Pro collaborative database with a portal into iRemix was used to codethe activity instances.
Table 1. Coding scheme for the OLSR framework.
Role definition Examples from iRemix
Audience. View what youth are doingonline
(A) View or read student work
Encourager. Encourage youth aboutwork or participation
(H) Comment: Jamie this is pimp, the detailsfrom the shoes to the chain are nice. GreatJob!!
Evaluator. Provide grades, ratings,badges, or other formal assessments
(A) Rate student work(A) Award or create badge
Sociocultural Friend. Exhibit personalapproachability/friendship/mentorship, including social posts,off-topic conversation
(A) Post a status update(A) Edit/create profile page(H) Comment: LOL! I love Boondocks, too!
Learner. Learn from youth (H) Comment: Thanks for sharing. I did notknow this before.
Learning Broker. Connect youth withlearning opportunities (people,activities, etc.)
(H) Comment: check [how to do this] withMs. Ammond or Mr. Vireo.
Model. Share own creative work/process
(H) Blogpost: [Animation created byeducator]
Monitor. Impose or suggest rules ofbehavior online
(H) Comment: This is not appropriate and Idon’t expect to see content like thisanymore. If I see it again you will loseyour privileges permanently!
Promoter. Showcase youth participantwork
(A) Feature student work on front page ofsite
(H) Blogpost: [Embed class-created e-zine]Resource Provider. Provide learningresources (how-to guides, links,embedded media, etc.)
(H) Blogpost: Use this link. [link](H) Posted document:ExportingFromiMovie.doc
Instructor. Directly teach a concept orskill or provide an assignment.Provide prompts and/or feedback tofurther student thinking or work
(A) Create a debate(H) Blogpost: Create a story based on acharacter’s trajectory of choices. It mustbe 3 paragraphs and include characternames, settings and a creative plot.
(H) Comment: I’d love to see a step by stepon how you made this piece.
Notes: (A) Indicates automatically coded actions. (H) Indicates actions that require interpretationand were coded by hand.
Learning, Media and Technology 9
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
3.2. Participants
The DYN instance reported here blended face-to-face school day classes withonline participation. The DYN classes focused on traditional and digital litera-cies and were taught by two DYN literacy mentors who specialized in Englishlanguage arts and two DYN media mentors who specialized in digital media.They co-taught the face-to-face classes with the formal classroom teacher.Mentors were required to spend time on iRemix in addition to their face-to-face classroom time. Two additional DYN online mentors were responsiblefor iRemix interactions only and were not present in the classroom. Becausethe formal classroom teachers were not involved online, they are not includedin this study. In all, there were six educators who participated in iRemix as partof these classes (50% female). The two literacy mentors were former classroomteachers of social studies and English Language Arts, both in their 40s. Themedia and online mentors were younger, were practitioners of their writingand media arts crafts, and had primarily worked in less formal youth programs.The literacy and media mentors had participated in regular DYN professionaldevelopment for over four years, while the online mentors were newer to theorganization. We looked at educators’ online interactions with students inthree sixth-grade classrooms (N = 79, 46% female) at a public charter K-8school. The school was located in an urban environment, with 725 students,99% of whom were African-Americans, and 86% of whom were receivingfree or reduced lunch benefits (one measure of socioeconomic status).
4. Results
Educator logins ranged from 19 to 97, with an average of 58 logins during thethree-month period. The number of logins was not an indicator of number ofactivities enacted in iRemix. There were 1864 educator to student actionslogged during this time. In the following, we share data patterns that we haveidentified.
4.1. The audience role: a window into students and their work
In today’s classrooms, teachers need to deeply understand not only the content,but also the personalities, interests, and abilities of their students (Darling-Hammond 2008). This practice and ability are called out and assessed inrecent teacher evaluation efforts (e.g., Danielson 2007). DYN educators usediRemix as a window into student work and knowing what students weredoing outside of the classroom. An overwhelming majority of the actionslogged by educators online reflects the Audience role (1448 instances acrosseducators, representing 78% of the total role actions logged), includingviewing student-created media assignments, profile pages, personal blogposts,and interactive discussions. What is not yet clear is the extent to which students
10 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
were aware of their adult audience and if it motivated their participation on thesite. To the student viewing the user interface, there was no indication of thenumber of views on a post or an artifact of work, unless a more visibleaction accompanied the view, such as a comment left on the page. AlthoughAudience reflects the majority of logged educator to student actions, studentswere most likely unaware of the adult educator views of their profiles andtheir work in iRemix.
4.2. Connections with individual students
The OSLN offered opportunities to support students individually, somethingthat has been documented to be especially important in urban schools(Brown 2004), but that is often difficult within the real-time complexities ofthe face-to-face classroom environment. Reading student work (Audiencerole), providing formal assessments (Evaluator role), and commenting onstudent work (multiple roles) represent roles played by an educator for an indi-vidual student as opposed to the entire group. Although there is evidence ofeducators enacting roles for a group of students, such as submitting statusupdates and posting an assignment or a resource for the class, the vast majority(96%) of the actions logged online were targeted toward an individual. Evenwhen the Audience role is removed from the count to reflect those roles thatare more visible to students, roles played for individual students remain themost common pattern in the online space (86%).
4.3. A range of roles played, but few played regularly
We then looked only at those actions that were visible to students in iRemix; thatis, those educator actions that left a trace (i.e., the 22% of logged actions forstudents that were not Audience). Of these, Instructor and Encourager wereemployed most often (71% and 50% of the visible actions, respectively). Acommon educator to student interaction reflected a blend of the roles of Instruc-tor and Encourager, involving a student posting work and an educator respond-ing with an encouraging statement followed by suggestions or prompts foradditional work or thinking. The following example was part of a poetryactivity designed by the classroom teacher:
Student post: Theres nothing as great as a dear old sunset / setting in thewest / its as beautiful as a rainbow / the sun is so close tothe ground / yellow orange pink purple blue / it is sopretty / you know that night is coming / a sunset is asign of night / sunset…
Mentor comment: I really like how you compared a sunset to a rainbow. Youshould add more descriptions. Describe how the day isdifferent during sundown. Very nice job, so far.
Learning, Media and Technology 11
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
The role of Resource Provider was evident in 19% of the visible actions, andothers were seen infrequently (see Figure 2), although there were a numberof features purposefully built into the iRemix environment to foster specific edu-cator–learner interactions believed to be generative to student participation. Forexample, educators were able to formally assess student work by creating andadministering learning badges and by rating artifacts against formal rubricassessments, but the Evaluator role occurred only twice, reflecting both creatinga new type of badge and a formal rating of student work. Likewise, thoughiRemix included a ‘Feature’ button to allow educators to push student workout in a prominent section of the home page, the Promoter role only occurredtwice.
All six educators played the role of Instructor, Encourager, and LearningBroker at least once. Although educators employed other approaches (thetotal roles educators played ranged from four to eight), they did not regularlyuse the range of roles possible in the system, nor were they exhibiting therange of roles that were apparent in face-to-face classrooms. Further researchis needed to determine why. User interface usability issues or a lack of aware-ness of certain features might explain these findings (the iRemix site is regularlyupdated as new features are added and improvements are made). Or, educatorsmay have made deliberate decisions about interaction, such as believing evalu-ation of student work to be something best done face to face.
4.4. Similarities among educators with similar responsibilities
Looking at the roles played by individual educators, we found that the two lit-eracy mentors looked similar to one another, and that this was also true for the
Figure 2. Percentage of educator to student actions that reflected each type of learningsupport role.
12 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
two DYN writing mentors who only interacted with youth online (see Figure 3).The literacy mentors primarily exhibited the Instructor and Resource Providerroles. The two online DYN writing mentors also displayed the Instructor roleprominently, but were much more likely to play the Encourager role. Althoughmore research is needed with larger data sets, we are curious about the role pat-terns. The online-only mentors may have felt the need to build a trustingrelationship with students as they commented on their work and promptedfor revisions or further thinking. The face-to-face mentors were using thetool primarily to offer guidance and provide further resources for learning,perhaps not feeling the need to build trust and rapport with students onlinebecause they had done so within the face-to-face environment.
4.5. Gender patterns for educators and students
Though we have a small sample of educators, the overall pattern of learningsupport roles played by adults in the online learning environment in the DYNcontext revealed that the three female educators evidenced more onlineactions for students counted as a learning support role. Males ranged from 54to 270 logged role actions (M = 145.33, SE = 95.57), while females rangedfrom 247 to 645 (M = 476.00, SE = 95.57).
Figure 3. Roles played by individual educators with different formal responsibilities.
Learning, Media and Technology 13
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
We also found that female students had more different roles played for themby adult educators. For each student, we created a sum count of the number ofvisible roles (i.e., not Audience) ever played at least once for the student withiniRemix across mentors. However, we found that girls were also contributingmore to the site, including blogposts, media artifacts, comments, and otherposts (M = 47.50, SE = 5.03) than boys (M = 28.55, SE = 5.94), and ANOVAshowed this difference to be significant (F(1, 77) = 5.93, p < .01). We wantedto analyze the number of roles played for students controlling for the numberof student posts, and chose to use a general linear model. It is important topoint out that this is not experimental data, but rather observational. Totalroles played was the dependent variable and gender was the categorical predic-tor variable, with the number of media artifacts posted on iRemix being the cov-ariate. In this case, the categorical predictor variable and the covariate were notindependent of one another. Although this violates an assumption of ANCOVA(analysis of covariance), for observational data these results still reveal a moreaccurate estimate of the relationship between gender and total roles played thannot adjusting for the number of media artifacts posted (Keppel 1991). In ouranalysis, there was a main effect of gender (F(1, 77) = 22.87, p < .000) on themean difference at any given value of the covariate, with girls receivingmore unique learning roles online from educators (M = 2.11, SE = 0.18) thanboys (M = 1.90, SE = 0.22). Interestingly, when the roles are looked at individu-ally, there are no significant differences for boys and girls for any role exceptEncourager [49% of boys and 72% of girls were encouraged by an adult edu-cator at least once within iRemix (chi-square analysis, p < .01)]. It is not yetclear if there is a pattern over time that can suggest causal effects. That is,were educators playing more roles for girls, and encouraging them in particular,which in turn prompted more work from the girls? Time is an important dimen-sion we intend to incorporate in our future implementations of this frameworkanalysis.
5. Reflecting on practice with the OLSR framework: a follow-up onprofessional development
Following the analysis of iRemix data presented in this paper, we reengagedDYN mentors. The study results were shared and several professional develop-ment meetings were organized around understanding the framework, looking atexamples in their own online student interactions, and reflecting on theircomplex roles as artists and teachers who work in both face-to-face andonline settings with youth. As with our earlier attempts at sharing data inwhich mentors were invited to co-develop the framework, we aimed to builda shared understanding of effective online teaching with the use of the OLSRframework. To address this goal, we created a professional developmentmodule, entitled ‘Online Participation 101’. In a set of activities, mentorswere prompted to question their assumptions about teaching in online spaces,
14 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
discuss and critique the OLSR framework, use the framework to code their owninteractions with students, and consider ways in which the roles did or did notreflect their own practice. Mentors who participated in this work found that theframework generally captured and validated the variety of roles they play asDYN mentors, although a few new roles also emerged. For example, mentorsproposed an additional role to acknowledge the task of ‘on-ramping’ – therole of scaffolding and promoting student engagement with the online environ-ment itself. Also, mentors questioned the label of the Sociocultural Friend role.While the notion of engaging in off-topic and friendly behavior was familiar tomentors, this label was not intuitive.
By engaging mentors in the reflections and discussions prompted by the‘Online Participation 101’ professional development module, a commonlanguage of learning emerged across the team of mentors, professional develop-ment leaders, researchers, and designers. Another area of future work is explor-ing shifts in educator practice resulting from the introduction of the OLSRframework and the new shared language and understandings.
6. Discussion
This study was motivated by the increasing presence of online learning environ-ments and the concern for equity of participation for all youth. We posit that thedata that can be collected from such systems allow for enormous opportunitiesto understand how learning occurs online and how to best support it. Impor-tantly, we believe that these data must be made accessible and usable to prac-titioners in order to develop models of effective online teaching and learning. Inthe following, we highlight some of the interesting findings, issues, and oppor-tunities for further study that have come out of this work.
6.1. Revealing patterns of educators’ online interactions with youth
While we cannot make claims about student learning from this analysis, theOLSR suggests specific ways that educators interacted with students iniRemix. Audience, Instructor, and Encourager roles were most prominent,with audience accounting for most actions logged by mentors. It was alsoshown that mentor actions online were most likely an interaction with an indi-vidual student as opposed to the entire group. These findings are important, asthey suggest that the online environment can offer educators an alternative orsupplemental space to become familiar with individual youth and their work,and to provide supports for learning for those youth around that work, andthat educators welcome these opportunities to do so. These practices haveimplications for the design of equitable online environments that have thepotential to supplement learning spaces where the educators do not have thetime or accessibility to attend to individual youth face to face, and for thoseyouth who do not have access to high quality learning opportunities nearby.
Learning, Media and Technology 15
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
When looking at how individual educators played these roles differently,the framework helps to reveal varying profiles. For example, one educatorplayed the Encourager role in 40% of actions while another evidenced thisrole in 7% of actions. We also found that the literacy mentors tended to playthe Resource Provider much more than the online-only mentors, and thatfemale educators were playing more roles overall. Although we do not havesignificant data on student learning outcomes that result from different roles,we do have data on the development of student portfolios of work, withfemale students evidencing more work posted online over time, and indicationsthat female students were receiving more roles played for them, including thatof Encourager.
These outcomes offer themes to pursue, including learning outcomes andgender differences for both students and educators in the online environment,a better understanding of how the Audience role is used and received, andimplementing the dimension of time into our analysis of the data logs to beable to draw more causal inferences. We intend to compare patterns of educatorand student activities to explore the impact of adult roles on student work andparticipation and look at the variation between educators. For example, how dolearning support roles influence student creation and revision of work? We alsowant to expand this work to examine other types of interactions in the onlineenvironment (e.g., student–student, educator–educator, educator–community,and student–community). Looking at patterns over time, future work willalso aim to identify any causal links between educator support roles andstudent outcomes.
6.2. Further development of the OLSR framework
There were many learning support roles that were rarely present online in thisstudy. Of particular interest are the roles that emphasize social and relationalinteractions, and have the potential to build rapport with students. Forexample, the Sociocultural Friend role in which educators exhibit personalapproachability, social posts, or off-topic conversation was not often evident.Five of the six educators played the role at least once, but the role was lessthan 10% of all interactions for each educator. Another role that can helpbuild relationships is Encourager; this role was one the most frequentlyplayed roles but was not prominent across educators. We may also argue thatthe Learning Broker role, in which the educator connects students with otherpeople or learning opportunities, is a relational action. To interpret thisfinding, one must have a complete picture of the learning environment andthe intentionality of the educator with respect to the types of interactions theyhave with students in the online space. Do teachers perceive the face-to-faceenvironment as more appropriate for social interactions, especially in contextsin which they spend time with students on a daily basis? And similarly, do edu-cators who are only interacting with students online have to take on a larger
16 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
range of roles, including that of rapport building, because there is no othercontext for exchange? As suggested from our data, we conjecture that sociallearning support roles will be played differently in contexts in which mentorsonly interact with youth online versus blended learning environments,suggesting the deliberate design of features that support different rolesneeded in different interaction contexts.
The relative lack of support for these types of virtual relationship-buildingraises questions about the social or emotional support needed and expected inan online learning environment for youth. Some have argued that it is in factthe social and relational aspects of teaching – nurturing social relationships,expressing energy and humor, and establishing an expressive connection withthe students – that are more important in the online space (Coppola, Hiltz,and Rotter 2002; Guasch, Alvarez, and Espasa 2010) since educators interact-ing with students online cannot rely on their expressive skills to build and main-tain relationships with students (Major 2010). Thus, while this study did notexamine the relationship between online learning support and learning out-comes for students, one area to investigate is the importance of socialsupport online. The key position of mentors and the social support they pro-vided in prior DYN research were clear; and part of the motivation of thisstudy was to understand how actions observed in mentoring may play out inthe online space. The importance of exposure to and experiences in participa-tory cultures and community for building skills, developing interests, andbecoming creators and contributors is well documented (Hargittai andWalejko 2008; Ito et al. 2009; Jenkins et al. 2009). Thus, being able to get apulse on how social support is distributed to students is one important way toaddress participation inequities.
6.3. Building a ‘system of meaning’ to build and shape data use
How can the OLSR framework help make data from online environmentsactionable for practitioners? We suggest that this framework serves as a‘system of meaning’ that offers a productive set of categories to shape howonline data are used and interpreted (Coburn and Turner 2011). Whether it isused as an anchor for professional development discussions or an analytical fra-mework for making sense of quantitative online use data, the OLSR frameworkhas utility for connecting teacher behavior to pedagogical goals. Going beyondparticipation metrics such as counts of logins and time spent online, theOLSR aims to illuminate types of behaviors that suggest their instructionalintension.
6.4. Data visualizations using OLSR to support professional development
This study suggests that making educators’ patterns of learning support rolesplayed online available in data visualizations, for example, as shown in
Learning, Media and Technology 17
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
Figure 2, can be useful for professional development purposes. Similar to theBarron et al. (2009) learning support roles, we believe that these roles can begeneralizable to other online learning contexts to understand educator–student interactions. One way that data displays like these can be useful is byusing them as a tool for reflection. For example, how should an educatorpresent her personality in the online environment? Is it a space to exhibitfriendly and encouraging behavior as she might in the face-to-face environ-ment? Or, might the online space be viewed as a ‘grading tool’ – a space tosupport the task of giving feedback and grading in an efficient manner that isconvenient for the instructor? Using visualizations that can reflect the learningsupport roles can be useful for informing conversations that educators need tohave about formal role expectations, responsibilities, and building effectivepractices. For example, differences observed across educators can be used todiscuss contextual factors that educators take into consideration in theironline behavior. The variability and complexity inherent in teaching can berevealed and examined. Educators may not be cognizant of their use patternsor the ways in which they may be differentially interacting with students. Assuch, data about learning support roles played for students can enable prac-titioners to attend to issues of equitable participation.
As we have seen in our professional development work, educators are waryof log data being used to tell the whole and only story about their teaching. Weacknowledge that the data logs can only reveal one aspect of the learningenvironment – especially in blended learning environments. Thus, such datashould be used to point us (as researchers and practitioners) to other datasources with which to triangulate, so that we can better document and under-stand other aspects of the learning environment (e.g., teacher–student face-to-face interactions). Furthermore, the method we employed to code some typesof educators’ actions automatically has the potential to provide practitionerswith immediate access to usable data about their practice. As we continue todevelop our data analysis process, we will seek ways to code more rolesautomatically. For example, the inclusion of a URL or other internallearning resource in a mentor’s comment can be used to automatically applythe Resource Provider code. By creating ways to quickly and easily seepatterns of online behavior, such data visualizations can inform qualitativeapproaches such as interviews and classroom observations to learn moreabout educator decisions and choices in the roles that they play both inperson and online.
6.5. OLSR framework to evaluate and improve the design of socialnetwork platforms
Just as we view the OLSR framework as useful for understanding educatorpractice online and how learning is supported equitably, we also believethat it can be instrumental in shaping the technological infrastructure of
18 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
iRemix as an OSLN. As technological capabilities advance and the OLSR fra-mework develops, we can use it as a guide for improving the design of thesocial network platform features that works toward the goal of supportinginteractions characteristic of affinity spaces. For example, to support engage-ment between experts and novices around the creation of artifacts, we canevaluate the user interface used to provide feedback for its ability to enablethe roles of Encourager or Resource Provider. In our current work, we areexploring the design of specific features that are intentional in helping educa-tors play learning support roles.
6.6. Methodological approach of the OLSR framework and itsapplication to other contexts: opportunities and limitations
In this study, we have offered a particular methodological approach that webelieve is viable and useful for informing efforts to improve instruction utiliz-ing online-only or blended models. With a concern for equity, we sought todevelop a method to reveal the nature of educator–learner interactions usinglarge sets of use data available from online learning environments. Buildingfrom the parent support roles framework of Barron et al. (2009) and collabor-ating educators, we identified some new roles and made modifications toothers to reflect the goals and activities of our particular context. As describedin the results earlier, our framework of OLSR and our method of using auto-matic and manual coding were indeed useful for revealing patterns that speakto equity – for example, gendered differences in learning support roles playedby educators. These findings encourage us to continue developing the OLSRframework by exploring its use in other contexts and pursuing research ques-tions that investigate educator actions, learner outcomes, and the design ofonline environments. In future explorations of the framework, we expectthat the conceptualization of roles will likely shift, as we learn more abouteducators’ intentions and pedagogical approaches in collaboration withpractitioners.
This study has also highlighted a few challenging aspects of the method thatwill require further work. For example, 25% of use logs we examined requiredhand coding to determine the learning support roles played, a task accomplishedby two researchers. This approach was viable given the early, exploratory stageof the work, and the size of the data set. For larger data sets, and in cases inwhich practitioners would need to have timely access to data in order to beuseful, the need to hand code use logs is a formidable challenge. Another chal-lenge that remains is developing effective ways of presenting data that areuseful to practitioners to reflect on and improve practice. As we saw in ourearly attempts at providing data to mentors for practice improvement,meeting practitioners’ needs and concerns is no simple task. Certainly, contin-ued implementation, reflection, and iteration are required. For researchers, theOLSR framework provides a lens for studying participation in online
Learning, Media and Technology 19
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
environments that is aligned with social learning. We have still more work to doin developing useful data displays and tools for educators.
7. Conclusion
This work is part of a larger initiative to understand interactions betweenmembers of an OSLN over time and we believe that the findings will be gen-eralizable beyond iRemix. This first study specifically looks at the differentroles that adult educators play to support students in an online environmentused in a formal classroom setting, although we believe its findings can beapplied in informal online learning environments as well. Building from ourprior efforts to use log data from the iRemix social learning network in pro-fessional development with teachers, we constructed the OLSR frameworkfor analyzing educator actions collaboratively with practitioners, an approachthat provided a grounded foundation and will help to support buy-in forfuture work in blended learning contexts such as school-based classes and infor-mal after-school programs and community organizations. The framework isintended to illuminate the kinds of moves they made online in terms of educatorto student interaction, and a method that utilizes automatic coding when poss-ible and human interpretation as needed. As features of online social learningnetworks such as iRemix change and technological capabilities improve, theremay be new opportunities to enable new types of interactions. However, weexpect that the nature of support roles for learning will be similar acrossformal and informal contexts. Our current work continues to explore the appli-cation of the OLSR framework in both informal and formal settings, both on-and offline, and – like the present study – involves educators as partners toexamine and revise it.
Understanding the learning support roles that are being played and notplayed and different generative outcomes that emerge from these directactions can help researchers and designers to create online supports fordesired actions and outcomes. For educators and professional developmentleaders, this work also offers a language and framework to share ideas and find-ings during professional development sessions, aiding discussions and under-standing the complexities of taking on multiple roles intentionally andenacting certain behaviors at key moments to achieve certain instruction and/or interactive goals. Above all, it is an exploration of the online learningenvironment as a space that can provide youth with a knowledgeable socialnetwork to support their learning and development.
AcknowledgementsWe would like to thank the Digital Youth Network research, design, and implemen-tation staff for their dedication to children and learning and for their contributionsand encouragement for this work. We would also like to thank the anonymousreviewers for their insightful comments and critique.
20 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
FundingThis research is supported by the John D. and Catherine T. MacArthur Foundation; theBill and Melinda Gates Foundation; and the National Science Foundation [grantnumber RECIIS-1325004].
Notes on contributorsDenise C. Nacu is an Instructor in the College of Computing and Digital Media atDePaul University. She is a designer and researcher focusing on building frameworks,tools, and data visualizations to enable teaching and learning in online social learningnetworks and blended learning environments.
Caitlin K. Martin is a Senior Researcher for the Digital Youth Network at DePaul Uni-versity and for the youthLAB research group at Stanford University. Her focus is on thedesign, implementation, and assessment of environments and materials that offer youthfrom diverse communities opportunities to deeply learn, understand, and use digitalmedia and technology.
Nichole Pinkard is an Associate Professor of Interactive Media, Human ComputerInteraction, and Education at the College of Computing and Digital Media at DePaulUniversity. She is also the founder of the Digital Youth Network. Her current scholarlyinterests include the design and use of pedagogical-based social learning networks,digital media literacy learning outcomes, and ecological models of learning.
Tené Gray is the Director of Professional Development and Operations for the DigitalYouth Network at DePaul University. Her focus is on the design and delivery of highquality professional development for mentors, teachers, and after-school program pro-viders in the areas of digital media literacy, connected learning, badge design and inte-gration, mentoring, and balanced literacy.
References
Barron, B., K. Gomez, N. Pinkard, and C. K. Martin. 2014. The Digital Youth Network:Cultivating Digital Citizenship in Urban Communities. Cambridge: MIT Press.
Barron, B., C. K. Martin, L. Takeuchi, and R. Fithian. 2009. “Parents as LearningPartners in the Development of Technological Fluency.” International Journal ofLearning and Media 1 (2): 55–77.
Brown, D. F. 2004. “Urban Teachers’ Professed Classroom Management Strategies:Reflections of Culturally Responsive Teaching.”Urban Education 39 (3): 266–289.
Chi, M. T. 1997. “Quantifying Qualitative Analyses of Verbal Data: A PracticalGuide.” The Journal of the Learning Sciences 6 (3): 271–315.
Coburn, C. E., and E. O. Turner. 2011. “Research on Data Use: A Framework andAnalysis.” Measurement: Interdisciplinary Research and Perspectives 9 (4):173–206.
Coppola, N., S. Hiltz, and N. Rotter. 2002. “Becoming a Virtual Professor: PedagogicalRoles and Asynchronous Learning Networks.” Journal of ManagementInformation Systems 18 (4): 169–189.
Danielson, C. 2007. Enhancing Professional Practice: A Framework for Teaching.Alexandria, VA: ASCD.
Learning, Media and Technology 21
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
Darling-Hammond, L. 2008. “Teacher Learning That Supports Student Learning.” InTeaching for Intelligence, edited by B. Z. Presseisen, 91–100. Thousand Oaks,CA: Corwin Press.
DeBoer, J., A. D. Ho, G. S. Stump, and L. Breslow. 2014. “Changing ‘Course’:Reconceptualizing Educational Variables for Massive Open Online Courses.”Educational Researcher 43 (2): 74–84.
Dubow, W. 2011. NCWIT Scorecard: A Report on the Status of Women in InformationTechnology. Boulder, CO: National Center for Women & Information Technology.
Eccles, J., and J. A. Gootman, eds. 2002. Community programs to promote youth devel-opment. Washington, DC: National Academies Press.
Eisenberg, M., and G. Fischer. 2014. “MOOCs: A Perspective from the LearningSciences.” Learning and Becoming in Practice: 11th International Conference ofthe Learning Sciences (ICLS) 2014, edited by J. L. Polman, E. A. Kyza, D. K.O’Neill, I. Tabak, W. R. Penuel, A. S. Jurow, K. O’Connor, T. Lee, and L.D’Amico. Boulder, CO.
Gee, J. P. 2005. “Semiotic Social Spaces and Affinity Spaces.” In Beyond Communitiesof Practice Language Power and Social Context, edited by D. Barton and K.Tusting, 214–232. Cambridge: Cambridge University Press.
Guasch, T., I. Alvarez, and A. Espasa. 2010. “University Teacher Competencies in aVirtual Teaching/Learning Environment: Analysis of a Teacher TrainingExperience.” Teaching and Teacher Education 26 (2): 199–206.
Hargittai, E., and G. Walejko. 2008. “The Participation Divide: Content Creation andSharing in theDigitalAge.” Information,Communication&Society11 (2): 239–256.
Ho, A. D., J. Reich, S. Nesterko, D. T. Seaton, T. Mullaney, J. Waldo, and I. Chuang.2014. HarvardX and MITx: The First Year of Open Online Courses (HarvardX andMITx Working Paper No. 1).
Ito, M., B. Sonja, B. Matteo, d. boyd, R. Cody, B. Herr-Stephenson, and H. A. Horst.2009. Hanging Out, Messing Around, and Geeking Out: Kids Living and Learningwith New Media. Cambridge: The MIT Press.
Jenkins, H., R. Purushotma, M.Weigel, K. Clinton, and A. Robison. 2009.Confrontingthe Challenges of Participatory Culture: Media Education for the 21st Century.Cambridge: MIT Press.
Keppel, G. 1991. Design and Analysis: A Researcher’s Handbook. Englewood Cliffs,NJ: Prentice-Hall.
Major, C. 2010. “Do Virtual Professors Dream of Electric Students? College FacultyExperiences with Online Distance Education.” Teachers College Record 112 (8):2154–2208.
Margolis, J., and A. Fisher. 2002. Unlocking the Clubhouse: Women in Computing.Cambridge: MIT Press.
MMS Education. 2012. Survey of K-12 Educators on Social Networking, OnlineCommunities, and Web 2.0 Tools. Report from MMS Education, Newtown, PA.
Nacu, D., N. Pinkard, R. Schmidt, and K. Larson. 2012. “Remixing iRemix: DataVisualizations to Understand Learning and Development in Online SocialLearning Networks.” Presentation at the Digital Media and Learning conference,San Francisco, CA, March 1.
Pea, R. 2013. “Learning Sciences and Learning Analytics: Time for a Marriage.”Lecture presented at the EDUCAUSE Learning Initiative annual meeting,Denver, CO, February 5.
Perna, L., A. Ruby, R. Boruch, N. Wang, J. Scull, C. Evans, and S. Ahmad. 2013. “TheLife Cycle of a Million MOOC Users.” Presentation at the MOOC ResearchInitiative conference, December 5.
22 D.C. Nacu et al.
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14
Picciano, A. G., and J. Seaman. 2009. K-12 Online Learning. A 2008 Follow-up of theSurvey of US School District Administrators. Report for the Sloan Consortium.
Purcell, K., L. Raine, A. Heaps, J. Buchanan, L. Friedrich, A. Jacklin, C. Chen, and K.Zickuhr. 2012. How Teens Do Research in the Digital World. Washington, DC:Pew Internet & American Life Project.
Rhodes, J. E., R. Spencer, T. E. Keller, B. Liang, and G. Noam. 2006. “AModel for theInfluence of Mentoring Relationships on Youth Development.” Journal ofCommunity Psychology 34 (6): 691–707.
Rideout, V. 2012. Children, Teens, and Entertainment Media: The View from theClassroom. White paper report for Common Sense Media.
Staker, H., and M. B. Horn. 2012. Classifying K-12 Blended Learning. Watertown,MA: Innosight Institute.
Warschauer, M., and T. Matuchniak. 2010. “New Technology and Digital Worlds:Analyzing Evidence of Equity in Access, Use, and Outcomes.” Review ofResearch in Education 34 (179): 179–225.
Learning, Media and Technology 23
Dow
nloa
ded
by [
205.
178.
115.
194]
at 1
3:23
02
Dec
embe
r 20
14