open badges mooc session 12: james willis
DESCRIPTION
#openbadgesMOOC Session 12: Design Principles Documentation Project / Open edX and Beyond Project Presenter: James WillisTRANSCRIPT
Open Digital Badges
Design Principles Documentation Project
Open edX and Beyond Project
James E. Willis, III, Ph.D.Research Associate
30 organizations awarded grants to develop badge
content in the 2012 Badges for Lifelong Learning
Initiative
Collaborators and Partners
Rebecca Itow
Daniel Hickey
Katerina Schenke
Cathy Tran
Nate Otto
Christine Chow
K-12
Middle School
Secondary
College Stu-
dents
Educa-tors/ Other
Vocational/ Adult
Common Core
Local StandardsPartner
s for 21C
S
None Identified
School or University
After School
Teacher PDIn-for-
mal & Other
Adult & Career
Mu-seum
Extra-Curricu-
lar
Accredited Con-text
Accredited Badges
None
Other
Earners
Setting
Standards
Accreditation
Regardless of where you start, it is likely you will end up somewhere other than your intended destination. That’s okay. Systems are living things and your system needs to be flexible. You need to embrace a bit of chaos in its design.
--Carla Casilli
Most of the knowledge generated when designing complex systems evaporates as features evolve and teams dissolve.
--Phillipe Kruchten
Badge Functions
Recognizing LearningSkills, achievements, experiences, & practicesIndividual, peer, social
Assessing LearningSummative, formative, transformative, & transcendent
Motivating LearningIntrinsic, extrinsic, & participatory
Studying LearningResearch of, for, & with digital badges
Intended Practices- Ideas outlined in original proposals
Enacted Practices- Intentions unfolding in world
Formal Practices- Practices after funding ends
General Findings Across Project
General Findings
Digital Badges are Different
Claims and Evidence are Hard to Define
Many projects struggled to specify claims
Many projects struggled to specify evidence
{
"uid": "f2c20",
"recipient": {
"type": "email",
"hashed": true,
"salt": "deadsea",
"identity":
"sha256$c7ef86405ba71b85acd8e2e95166c4b111448
089f2e1599f42fe1bba46e865c5"
},
"image": "
https://example.org/beths-robot-badge.png",
"evidence": "
https://example.org/beths-robot-work.html",
"issuedOn": 1359217910,
"badge": "https://example.org/robotics-
badge.json",
"verify": {
"type": "hosted",
"url": "https://example.org/beths
-robotics-badge.json"
}
}
General FindingsOpen Digital Badges are Really Different
Info Circulates in Social Networks
Projects recognized that states are increased
Validity gets crowd-sourced
COPPA, FERPA, and other concerns worried many projects
General FindingsIt’s Not (Just) About Badges
Badges are Part of an Ecosystem
More successful projects integrated badge functions into their learning ecosystem
Less successful projects tried to build an ecosystem around badges
SPECIFIC FINDINGS Across Project
Badges Can Really Work
- but not everywhere.
4 Projects Suspended
1 Project Still Trying
7 Projects Partial
18 Projects Implemented
Sheryl Grant’s Starting PointsBadge Systems = Tech + Content + Badges
Bucket Start With Need to Build
Layered Tech + Content Badges
Integrated Tech Badges + Content
Responsive Content Badges + Tech
New Build Badges + Tech + Content
Badges First Badges + Tech + Content
Layered Integrated Responsive New Build Badges First
0
1
2
3
4
5
6
7
8
9
10
11
12
Implementation Status by Bucket
ImplementedPartialPursuingSuspended
K-12
Middle School
High School
Post-Se
con...
Adult0
2
4
6
8
10
12
14Implementation Status by Earners
ImplementedPartialPursuingSuspended
Layered Integrated Responsive New Build Badges First
0123456789
101112
Badge Status by Bucket
Proposed
Different
None
Layered Integrated Responsive New Build Badges First0
1
2
3
4
5
6
7
8
9
10
11
Ecosystem Status by Bucket
Proposed
Different
None
Discussion of One Segment of the DPD Project: Adult Learners
Proposed ecosystem with scaled back practices
Proposed ecosystem with fewer practices
Different badges and ecosystem
Proposed badges and different ecosystem
Still building proposed badges
Proposed badges
Practices Formalized or Proposed & Not Enacted
Category Principle Practice
Recognizing Recognize diverse learning Recognize diverse learning
Use badges to externally communicate learning
Use badges to externally communicate learning
Promote discovery Discover learning opportunities
Assessing Align activities to standards` Internal standards
Use formative functions of assessment
Peer assessment
Use mastery learning Judged by human experts and computer
Use rubrics Rubrics developed for specific artifacts
Promote “hard” and “soft” skills Combine collaborative learning and discreet skills
Motivating Provide privileges Peer mentorship
Display badges to the public Learners can choose to share badges with public
Build outside value for badges Real-life application of knowledge
Proposed & Not Enacted or Unproposed & Formalized
Category Principles General PracticeRecognizing Have experts issue badges Credentialed via accredited
entity and community
Have experts issue badges Credentialed via community
Seek external backing of credential
Externally endorsed
Assessing Use rubrics Rubrics developed for specific artifacts
Motivating Provide privileges Provide prizes
Use different types of assessments
Peer Assessment
Stimulate competition Use point system
Study Study impact of badges Research of badges
Practices Formalized, Proposed & Not Enacted or Unproposed & Introduced
Category Principles General Practice Specific Practice
Recognizing Learning
Use badges as a means of external communication of learning
Endorsement and translation of skills
Promote discovery Discover learners Promote earners’ skills to employers
Assessing Learning
Use leveled badge systems Competency levels Translation of skills
Motivating Learning
Build outside value for badges
Evidence of outside opportunities
Potential to gain employment
Recognize identities Target a specific group Veterans
Engage with community Involvement in digital community
Network with community members
Studying Learning
Study Badge impact Research of badges Data collection and analysis
Practices Proposed & Not Enacted
Category Principles General Practice
Recognizing Use badges to externally communicate learning
Use badges to externally communicate learning
Assessing Use leveled badge systems Competency levels
Enhance validity with expert judgment
Use human experts
Use formative functions of assessment
Provide peer and expert feedback
Involve students at a granular level Learning pathways and badge design
Motivating Set goals User created badges
Use different types of assessments Peer assessment
Principles and Practices Across Projects
Most Challenging RECOGNITION Principles & Practices
Principle Practice Failures/ Attempts
Seek external backing Externally endorsed 6/10Externally valued 6/8
Use badges to externally communicate learning
Use badges to externally communicate learning
4/12
Have experts issue badges Have badges accredited by accredited entity and community
3/10
Align badges to standards Align to internal standards 2/6
Align to national or international standards
2/12
Align to community standards
1/4
Most Challenging ASSESSMENT Principles & Practices
Principle Practice Failures/ Attempts
Enhance validity with expert judgment
Use human experts and computers
4/10
Align assessments to standards and create objectives
Common Core State Standards
3/8
National/state standards 1/8Internal standards 1/9
Use e-portfolios Foster discussion around artifacts
2/3
Open to the public 1/2Local to community 1/4
Use formative functions of assessment
Peer feedback 2/4
Most Challenging MOTIVATION Principles & Practices
Principle Practice Failures/ Attempts
Provide privileges Provide internships 4/6
Provide prizes 3/6
Provide peer mentorship 3/9
Provide new activities 3/11
Use different types of assessment
Peer assessment 4/7
Self assessment 1/3
Expert assessment 1/3
Build outside value for badges
Evidence for outside opportunities
3/9
More Info on the DPD Project: DPDProject.info
Open edX and Beyond Project
“Practical Numerical Methods with Python” MOOC taught by Prof. Lorena Barba,
GWU
Graduate students in on-campus course are simultaneously enrolled
CRLT: technology (facilitating coding in Open edX) and pedagogy (purposeful implementation, evidence, and assessment)
Practical Numerical Methods with Python
Dr. Barba realized that Open edX requires authentication; she proposed a solution: to link directly to GitHub.
May be the first time digital badges have linked to GitHub for evidence.
Challenges and OpportunitiesTechnical
Finding the ‘seams’ in Open edX coding to build a badges API connection
Assuring individual identity verification and management
Keeping open materials within the evidence of outcomes
Pedagogical
Assessing student progress in specific, cumulative skills learned
Aligning outcomes for replication in future edX and Open edX MOOCs
Collaborations to Overcome Challenges
On-ground expertise and collaboration is vital for short-term solution, and to set the stage for the long-term issuance of badges.
Will locate the ‘seams’ in Open edX platform
Commitment to open badges development
Will use API to link to Open edX
Python MOOC: Goals
Short-term goal: by mid-November, a series of badges will be available.
Long-term goal: by spring 2015, instructors in Open edX and edX will be able to seamlessly offer badges.
Present collaboration at Open edX conference in Cambridge, MA in mid-November.
On-Going and Future Goals
Facilitate widespread use of digital badges in higher education
• From fully online courses to hybrid courses and stand-alone open modules.
• From a single platform (edX) to the other major platforms.• From unaccredited MOOCs to for-credit courses, co-
curricular learning, faculty & staff learning.• From email-based & informally verified badges to
cryptographically-signed and formally-verified badges.• From credentialing of isolated learning experiences to
sophisticated dossiers with dramatic impact on college admission and employment.
Publish findings, challenges, collaboration notes, and future opportunities
Questions?