science of social learning (learning solutions 2016)

Sharin’ Ain’t “Social”The Science of Social Learning

Greg BybeeVP, Learning Solutions, NovoEd

greg@novoed.com

March 17, 2016

Introduction

Principles of Andragogy

Quick NovoEd Introduction

Principles of Social Learning

Research on Team Learning

Other Best Practices

Closing Thoughts

Agenda

I’m an education technologist

● McKinsey & Company Consultant● Coursera, Product Manager● Renren, Education Business Development● NewSchools Venture Fund, EdTech Fellow● NovoEd, Learning Products (2013-present)

At NovoEd, I lead our learning solutions team

● Learning experience design (and ISD)● Learning solution strategy consulting● Course and program operations and facilitation● Customer success and support

I have a diverse background in training

● MA-Ed & MBA @ Stanford● Chair, Student Achievement @ Leadership High● Instructor @ 辰熙中英文学校 (China)● School Building @ ILAE in Ethiopia● Education Market Consultant and Volunteer

Hi, I’m Greg

Nice to meet you, too!

Introduction

Closing Thoughts

Agenda

Group Activity!!

● Form groups of 4-8.

● Answer the question: “what makes for an effective adult learning experience”○ Focus not on outcomes of the

learning, but the definition of the learning process

○ You’ll have 8 minutes.

● Add your ideas as text here:http://padlet.com/novoed/ls16

Weren’t you supposed to tell us that?

Source: Learning Solutions Session, March 17, 2016

Social collaborative

- Metacognition, teaching- Multiple perspectives- Peer-to-peer adaptivity- Felt accountability

Construction and Inquiry

- Problem-Based- Project-Based- Role plays, scenarios- Case method

Engaging

- Intrinsically Motivated- Relevant, authentic

My list(Okay admit I took more than 8 minutes...)

Experiential

Coherent

Self-directed, autonomous

Mastery, goal attainment, progression

Applied, relevant, authentic, purposeful

Engaging, fun, captivating

Inquiry-based, discovered

Constructed, created, explored

Scaffolded, workshopped

Interactive, multimodal, multisensory

“Knowing, being, doing”

Accountable, measured, transparent

Practiced, reinforced, rehearsed

Leveled, ZPD, personalized, adaptive

Chunked, cognitive load, working memory

Coherent, sequenced, schematicCurricular Design

- Sequencing, chunking- Scaffolded, coherent

Malcolm Knowles’ Theory of Andragogy

Assumptions1. Self-concept2. Experience reservoir3. Readiness to learn4. Orientation to learning5. Motivation to learn

Therefore, learning must be:1. Self-directed2. Experiential3. Relevant and authentic4. Problem-based

Sources: Knowles, M. (1984). The Adult Learner: A Neglected Species. & Kearsley, G. (2010). Andragogy: The theory Into practice database.

Source: Pappas, C. (2013). “The Adult Learning Theory.” eLearning Industry. Online.

But this training is costly...

Airfare – Average of $800 per person

Hotel – 3 nights @ $200 per night $600

Catered Meals – 3 days @ $150 per day = $450

Labor – $500 per day (assuming $125k/yr) = $1,500

Transportation – $200 per person across 3 days

Group Activity – $300 per person

Room Rental of $2500 per day x 3 = $7500

Total Investment before a facilitator = $103,750

Facilitator = $15,000 per day x 3 = $45,000

---------------------------------------------------------------------------

Total Cost Per Person: $5,800

Source: Altman, Ian (2014) How Much Does Sales Training Cost? Forbes. Online.

...and it cannot scale across the organization

Companies spend 78% more on Executives than Emerging Leaders (and 360% more per Leader)

Source: Bersin by Deloitte (2014) Leadership Development Factbook 2014: Benchmarks and Trends in U.S. Leadership Development.

Online learning offered a solution, but didn’t deliver

My Background

Shameless Plug

Closing Thoughts

Agenda

Solutions for learning experience design and implementation

Online platform for experiential learning

NovoEd develops talent online.

We power the best talent development programs

Principles of NovoEd all learning

Intuitive, User Friendly

Experience Orientation

Pervasive and Social Learning

Data-Driven, Evidence-Based

These drive 20x higher engagement C

Traditional Social

Stanford Professor Chuck Eesley conducted this research in 2014 based on data from Technology Entrepreneurship (12/13) with 26,248 students.

Figure 1. Completion Rate By Social Model in MOOCs

Learners are more engaged when they learn together. 68

95%+ User Satisfaction

60+ Net Promoter Score

Source: Survey of NovoEd learners (January 2016)

My Background

Closing Thoughts

Agenda

What is social learning?

The use of peer-to-peer (social) interaction as a pedagogical technique to drive learning.

Examples:

● 1:1 tutoring● Reciprocal Teaching● Think-Pair-share● Jigsaw ● Group work● Collaborative brainstorming

Source: I made this up last night after talking to another speaker.

Social learning is more than (just) sharing.

Communities visible to themselves with real identity

Peer-to-peer feedback, coaching, and mentorship

Collaboration in teams on relevant projects

Discussions with forums, and messaging

User generated content sharing with discovery

Why is social learning valuable?

Theories

● Relatedness motivation (Deci & Ryan)

● Felt accountability (Sutton & Rao)

● Social context of learning (Bandura)

● Cooperative learning (Dewey, Ross, et al)

● Social constructivism (Piaget)

● Joint inquiry (Dewey)

● Shared reflection of ELM (Kolb)

● Diversity of Learning Styles (Kolb)

● Mastery learning (Bloom)

● Community of Inquiry (Peirce, Dewey, Garrison, et al)

● Social development theory (Vygotsky)

● Truly adaptive learning (Bybee)

But you don’t have to subscribe to a single theory to leverage the techniques.

Source: I compiled this list last night - send me any I missed!

The benefits of social learning

Communities visible to themselves

Peer-to-peer feedback and coaching

Collaboration in teams on projects

Discussions with forums, messaging

User generated content sharing

● Social obligation, transparency, “felt accountability”● Self-regulating and adapting● Build networks, the new “content management system”

● Metacognition in providing feedback or mentorship● Personalized and adaptive; “network of 1:1 tutors”● More relevant guidance, “learning at speed of trust”

● Inquiry-driven, constructivist, PBL● Practice, rehearsal, interactivity● Self-directed and autonomous, right in ZPD

● Share diversity of skills, experiences, and perspectives● Crowd sourced, rapid response time● Relevant and authentic dialog drives connectedness

● Provides multiple representations & perspectives● Application focused, authentic, and relevant● Accountability and social proof

My Background

Closing Thoughts

Agenda

My Background

NovoEd Introduction

Social Learning

Team Learning

Best Practices

Closing Thoughts

Impact of Teams

Felt Accountability

Team Size

Team Selection

Team Heterogeneity

Scaffolding Teamwork

Context and Methodology

Chuck EesleyAssistant Professor,Stanford University

● Teaching online since 2012● Morgenthaler Faculty Fellow ● 2010 Best Dissertation Award,

Academy of Management● Ph.D., MIT; BS, Duke University

Technology EntrepreneurshipChuck Eesley, Stanford UniversitySeptember 16, - November 17, 2013

● Eight-week course offered on NovoEd

● Team-based, experiential pedagogy

● 8 assignments, each a part of a team-based project to find and evaluate a startup idea

● Taught on NovoEd 11+ times and at Stanford since 2004

View the course at https://novoed.com/venture17

Multivariate regressions, descriptive statistics, and t-tests of difference in means

● n = 26, 248 students● Students self-selected into

experimental groups● Dependent Variables:

engagement and satisfaction measures

● Independent variables: whether the student participated in a team, if they worked individually, and if they had a mentor

● Control Variables: demographics, engagement level, and others

For more information, please contact Professor Eesley at cee@stanford.edu.

Researcher Context Methodology

16x higher completion with peers and feedback

Individual with Discussions

Teams Teams with Feedback

Chuck Eesley, Stanford Professor, conducted this research in 2014 based on data from Technology Entrepreneurship (12/13) with 26,248 students.

Figure 2. Completion Rate By Social ModelRESULTS:

5x more sign-ins by individuals in teams A

Individual with Discussions

Teams Teams with Feedback

Chuck Eesley, Stanford Professor, conducted this research in 2014 based on data from Technology Entrepreneurship (12/13) with 26,248 students.

Figure 3. Sign-Ins Per User, By CohortRESULTS:

Agenda

My Background

NovoEd Introduction

Social Learning

Research on Teams

Best Practices

Closing Thoughts

Impact of Teams

Felt Accountability

Team Size

Team Selection

Team Heterogeneity

Scaffolding Teamwork

Rao & Sutton (2013) suggest felt accountability drives intrinsic motivation and engagement

Sutton and Rao discuss NovoEd and the value of felt accountability

in Scaling Up Excellence

"[NovoEd] built in numerous clever and easy-to-use social features to create a peer-powered network

that would link, organize, evaluate, and mentor students."

Huggy Rao Robert Sutton

Agenda

My Background

NovoEd Introduction

Social Learning

Research on Teams

Best Practices

Closing Thoughts

Impact of Teams

Felt Accountability (Stanford)

Team Size (NovoEd)

Team Selection (NovoEd)

Team Heterogeneity (NovoEd)

Scaffolding Teamwork (Presidio)

8 public MOOCs with self-formed teams on NovoEd from 2014-15

● Stanford GSB - Scaling Up● Stanford - Tech Entrepreneurship● Stanford - Creativity: Music● IDEO/ACUMEN - Design Kit ● Princeton- Global History Lab● Princeton - Making Gov't Work● Maastricht - Project-Based

Learning

3866 teams total

Important Note: Learners self-selected into various team sizes

Andrew LinfordInstructional Programs,NovoEd

● Coro Fellow (2013-14)● Instructor, Ministry of Education,

Singapore (2011-13)● BA, Stanford University

Retrospective cohort data analysis and descriptive statistics.

● n1 = 10, 315 studentsn2 = 3,866 teams

● Dependent Variables: percentage of assignments completed

● Independent variables: team size

● Control Variables: none

● Analysis done Sept 2015

For more information, please contact Andrew at andrewl@novoed.com

Note: Team size one is off chart (1734). Average exclude team size of 1. Including that, average is 2.7.

Source: NovoEd Analysis (2015). 8 MOOCs, 3,866 teams, 10,315 students

Learners prefer smaller teams, average is 4

Figure 4. Team Size PreferencesRESULTS:

Source: NovoEd Analysis (2015). 8 MOOCs, 3,866 teams, 10,315 students

Empirically, the optimal team size is 7.

Figure 5. Team Size vs. Assignment Completion

RESULTS:

Agenda

My Background

NovoEd Introduction

Social Learning

Research on Teams

Best Practices

Closing Thoughts

Impact of Teams (Stanford)

Team Size (NovoEd)

3 private offerings of Presidio Institute’s Introduction to Cross Sector Leadership: Building Teams

420 learners total

Important Note: In two of the courses, learners self-selected into teams (220 learners). In the other course (200 learners), learners were assigned teams by the teaching team based on meeting availability times.

● n1 = 420 learners

● Dependent Variables:

○ Learners engaging in course-wide discussions

○ Learners chatting with team in private work space

○ Learners messaging other learners

○ Learners commenting on assignment submissions

● Control Variables: course size

● Analysis done Feb 2016

Andrew LinfordManager, Support and Technical OpsNovoEd

Alison GoldManager of Leadership EducationCourse InstructorPresidio Institute

AndrewAlison

Autonomous Team Formation Increases Engagement

RESULTS:

Source: NovoEd and Presidio Institute Analysis (2016). 3 Courses, 420 learners

Agenda

My Background

NovoEd Introduction

Social Learning

Research on Teams

Best Practices

Closing Thoughts

Team Size (NovoEd)

● Focused on organic teams formed by the students and analyze preferences of students when selecting a team.

● Analyzed the differences between randomly-assigned and self-selected teams.

● Analyzed signals to discern which activities had the greatest correlation with course completion.

● n1 = 11 courses● n2 = 24,000 active learners

For more information, please contact Milad at m.eftekhar@gmail.com.

Milad EftekharData Science Intern, NovoEdCo-authored with Amin Saberi and Farnaz Ronaghi

● Former data researcher at Microsoft and Stanford

● Ph.D., University of Toronto● BS & MS, Computer

Engineering, Sharif University of Technology

● Analysis conducted on 11 public MOOCs with teams offered summer 2014.

● Courses were four to eight weeks long focusing on various business topics.

● Enrollments ranged from 200 to more than 25,000.

● Demographic data comes from self-completed student profiles. Data includes age, education, gender, and location.

Working Paper: “Team Formation Dynamics: A Study Using Online Learning Data.”

Learners preferred teams with similarly aged members.

Figure 6. Homogeneity by age in team selection

Source: Eftekhar, et al (2015) Team Formation Dynamics: A Study Using Online Learning Data.

RESULTS:

Figure 7. Age is a homophilic preference.

Age homophily resulted in more successful teams.

Average Age Distance

RESULTS:

Figure 8. Distance is a homophilic preference, particularly longitude due to timezones.

Individuals prefer to join teams in similar time zones.

Average Distance (km)

RESULTS:

Figure 9. Education Level is a homogenous preference

Teams preferred and worked best with similar education levels

Average Education Distance

RESULTS:

Figure 10. Skill diversity is a heterogenic preference

Successful teams have diverse skill sets.

Skill Entropy

RESULTS:

Note: Courses tended to 2/3 male, 1/3 female.

Learners prefer gender mix, though 2/3 identify as male

RESULTS: Figure 11. Gender mix by course

Results are consistent across all courses.

Team preferences are rational.

Characteristic Most Effective

SimilarAge

SimilarLocation

SimilarEducation Level

DiverseGender

DiverseSkill Set

Preference

Similar

Diverse

RESULTS: Table 1. Team characteristic preferences and effectiveness

Agenda

My Background

NovoEd Introduction

Social Learning

Research on Teams

Best Practices

Closing Thoughts

Team Size (NovoEd)

How did you run your teams?

Did any group...

● Use an ice breaker?

● Assign roles (timekeeper, note taker)?

● Divide into smaller groups?

● Google or use external sources?

● Students randomly assigned to two cohorts / groups.

● Group 1 was given team guidelines during week 2, with a suggested agenda and roles for the experience. Group 2 had no instructions.

● The post course survey then asked how well teams functioned and final projects were assessed.

● n1 = 97 studentsn2 = 103 students

For more information, please contact Drew at drew@novoed.com

Drew RemikerInstructional ProgramsNovoEd

Alison GoldManager of Leadership EducationCourse InstructorPresidio Institute

● Course: Introduction to Cross Sector Leadership: Building Teams by the Presidio Institute

● Offered as private beta to members from United Way, Points of Light, Americorps, Kresge Foundation, ProInspire, WYMAN, and others

● Providing structure for team meetings and roles will result in a higher self-reported quality of teamwork and completion of final team assignment compared to organic team development.

Researchers Context Methodology

Alison Drew

Source: Presidio Institute (2015) Introduction to Cross Sector Leadership: Building Teams, Team Scaffolding Study. “Guidelines for First Team Meeting”

Example of teamwork scaffolding

EXHIBIT:

Source: Presidio Institute (2015) Introduction to Cross Sector Leadership: Building Teams, Team Scaffolding Study.

Collaboration scaffolding improved completion and teamwork quality

Characteristic Group 1 Group 2

Quality of teamwork 4.03 / 5.0 3.79 / 5.0

Completed final assignments

8 / 10 7 / 9

Figure 12. Team scaffolding results

Marginal improvement in self-reported teamwork quality and final assignment completion

RESULTS:

My Background

Closing Thoughts

Agenda

My Background

NovoEd Introduction

Social Learning

Research on Teams

Best Practices

Closing Thoughts

Notifications

Course Champions

6 public MOOCs offerings before and after new notifications offering was released.

● Stanford GSB - Scaling Up Without Screwing Up

● Stanford - Tech Entrepreneurship Part 2

● Agder - Success Unleashed

41,657 teams total

Important Note: Almost nothing changed within the courses when offered a second time with the notifications feature available, although were slightly smaller. Learners self-selected teams in all courses.

● Coro Fellow (2013-14)● Instructor, Ministry of Education,

Singapore (2011-13)● BA, Stanford University

● n1 = 41,657 learners

● Dependent Variables: number of learners

● Independent variables: selection of different social indicators

● Control Variables: course size

● Analysis done March 2016

Notifications

● Personal alerts about ongoing social activity (including responses in discussion forums), assignment reminders, and more

● Daily Course Digests with assignment reminders and an overview of course activity

● Easy reminders about and access to teaching team communication

Huge Impact on Submissions; Positive Impact on Other Social Activities

RESULTS:

Source: NovoEd Analysis (2016). 6 Free Public MOOCs, 41,657 learners

Figure 13. Social Activities Pre/Post Notifications

Agenda

My Background

NovoEd Introduction

Social Learning

Research on Teams

Best Practices

Notifications

Course Champions

● Program: Philanthropy University

● September - December, 2015

● Course champions chosen from active learners in the middle of the program (beginning of November)

● 7 courses, 200,291 unique learners

Lisa BrefiniPhilanthropy University TACourse Operations SpecialistNovoEd

Retrospective learner and course champion data analysis.

● 7 courses, each with 5-12 course champions

● Comparison - course (and course champions) social activity before and after course champions are selected

● Analysis performed December 2015

Researchers Context Methodology

Lisa Andrew

Course Champions at work

Course Champions increased course social activity

RESULTS:

Source: NovoEd (2015). Philanthropy University Courses from 2015.

Course Champions Selected

Figure 14. Weekly Discussion Activity

My Background

Closing Thoughts

Agenda

● That experience must be engaging, driven by inquiry, social, and coherent.

● Adults require self-discovery and (extra) practice to overcome preexisting schema

● Knowledge acquisition < skill development < behavior change. Increasingly experiential.

● Online learning is no different. Scale the best of offline learning, not the worst (lectures).

● Social techniques include discussions, collaboration, sharing, feedback, community.

● Peer learning enables personalization, metacognition, and motivation.

● Teams and group work is the most critical.

CLOSING THOUGHTS

Learning must be experiential.

Learn Together.

http://novoed.com

Thank you! Please feel free to contact me!

Greg BybeeVice President, LearningNovoEd

greg@novoed.com

@gregbybeehttp://linkedin.com/in/gregbybee

Email me for slides or the research briefs. I’d also love to help you implement in your org.

science of social learning (learning solutions 2016)

Education

systole learning solutions

ekg life science solutions

adecco learning solutions

ms solutions - separation science...

science standards of learning science curriculum guide...

opentext learning solutions

totara learning solutions update | andy kirk, totara...

puppets: talking science, engaging science, learning...

ai & machine learning solutions -...

ms solutions - separation science

sage grouse initiative science to solutions grazing … ·...

statements of learning for science · 2007-01-09 ·...

learning 2.0-web2.0 applications for learning solutions

science learning

climate science & solutions professional science master’s...

ibm learning solutions

digital solutions content learning services tvet …...

3ds learning solutions

artificial intelligence machine learning & data science ·...

pvtd2014 learning solutions