webinar introduction to vocareum virtual labs
TRANSCRIPT
Introduction to Vocareum Virtual Labs
March 26, 2020
Webinar
Sanjay SrivastavaCEO
David LinBusiness Development
David Joyner, PhDGeorgia Tech
Agenda
• Vocareum Introduction - Sanjay Srivastava - 10 mins
• Teaching Computer Science at Scale - David Joyner - 30 mins
• Q&A
Mission
Deliver virtual labs and assessments
securely, on budget, and at scale
( incl. skills-based exams)
(infrastructure, tools, content)
(user roles, policies, spend)
(~ 500K learners)
Academic Partners
… over 600 institutions using Vocareum Labs today …
University of Illinois
RMIT University
Indiana University
Delft University
Supporting broad range of subject areasData Science
Fully hosted notebook solutions;Pre-installed packages
Programming
Broad range of languages; Standard IDE, spyder, eclipse.
Engineering
Stream desktop & applications; License flexibility, Raspberry Pi
Big Data
Hadoop processing;Run MapR on Clusters
Machine Learning
Any ML framework; Dedicated GPU
Cloud Computing
Policy controlled, budgeted; Aggressive resource mgmt
Database
MySQL, PostgreSQL;Neo4j, DynamoDB
Full Stack
Front- and back-end services;Business logic; Databases
DevOps & Cyber Security
Range of tools and infrastructure;Linux; Sudo/Root/Admin
C/C++
Assessment Options
ManualGrading
AutomatedGrading
In-ContextFeedback
PeerReview
MasteryLearning
CompetitionLeaderboard
Using diverse architecture to support labs• Web Applications
• Streaming Linux Applications
• Batch Compute Clusters
• Cloud Infrastructure
• BYO Applications + Infrastructure • Network of Docker Containers• Network of Virtual Machines• AppStream
Open source web applications/frameworks• Jupyter Notebook
• R Studio
• Terminal
• MySQL
• MongoDB
• Neo4J
• Postgres
Supporting a variety of languages:• C, C++, Python, R, Java, Scala, Go, PHP, etc.
• Hadoop applications, including Spark
• ML frameworks - TensorFlow, PyTorch
• Eclipse Theia
• Cloud 9
• Node
Open source desktop applications • MySQL Workbench
• Eclipse
• Rattle (for R)
• Spyder
• Firefox browser
• pgAdmin
Others can be easily added upon request
Learning Management & Assessment
• Enrollment
• Hosting Course Material
• Grading
• LTI integrations
Supporting Exams• Synchronous exams with 500 students• Asynchronous exams (90 minutes over 3 days)• Scheduled proctored certification tests• Exams with a passcode provided by proctor• Check whether learners came from a specific IP
range• Deliver random tests from a bank of problems• Built in plagiarism detection tests.
“I can see how it is very challenging to scale up the system so all students can access and work on the exam at the same time.
In previous years, we used our own online platform to handle students’ submissions. But, we found Vocareum to be much more maintainable and has better features. Using Vocareum really helps us: we know who submitted, at what time they submitted, and then we can run an auto-grading job immediately.”
Dr. Oka Kurniawan, SUTD
CONFIDENTIAL
Student Lab Interfaces
Supporting Advanced Coursework & Research• Long running jobs• Large data sets• Additional resources
• Cloud - AWS• Databases - MySQL, Mongo, Neo4j
• Aggregated labs + tools
Deploying Vocareum
Hands-on Labs / MOOC
Labs for learners to use tools, technology and services in managed environment (policy, budget, etc)
Skills-based Exams
Administer proctored skills-based certification exams, with automatic scoring
Training / Bootcamps / Hack-a-Thons
Course material + lab environments to support instructional training events
Research
Team-based lab environment with data security, grant management, etc.
Syndication
Course material + labs packaged and distributed to Universities, partners, etc.
Virtual Campus Labs
Upgrade on-prem computer labs with purpose-built cloud labs and tools
Security
Public Subnet Private Subnet
servers
distributed file system
object storage
proxy server
internet gateway
NAT gateway
Customer VPC
REST API
CONFIDENTIAL
Thank you!
Vocareum Webinar SeriesDavid JoynerMay 28, 2020
CS1 Online & At Scale
Goal:Scale up the availability of
intro-level computer science education.
Solution:Georgia Tech's online CS1301: Introduction to Computing in Python
course, available as both a massive open online course (MOOC) and a for-credit offering.
CS1301Introduction to
Computing
CS1301Introduction to
Computing Online
Course Videos
Integrated Exercises
Programming Exercises
Adaptive Textbook
Problem Sets
Proctored Exams
Recitations & TAs
Course Videos
Integrated Exercises
Programming Exercises
Adaptive Textbook
Proctored Exams
Recitations & TAs
~500 2-minute videos
~500 integrated multiple choice and text entry exercises
~200 integrated programming exercises with live feedback
Textbook with ~1000 multiple choice and text entry exercises
~150 programming and ~500 multiple choice/ text entry problems
Proctored exams occurring within a browser-based IDE
Traditional in-person recitations, graders, and teaching assistants
Problem Sets
Course Videos
Integrated Exercises
Programming Exercises
Adaptive Textbook
Proctored Exams
Recitations & TAs
Free MOOC
Verified MOOC
Certificate MOOC Series
Georgia Tech Course
Problem Sets
Free MOOC
Verified MOOC
Certificate MOOC Series
Georgia Tech Course
Opened February 2017, Free
Opened February 2017, $100
Opened July 2018, $400
Opened January 2017, Tuition
Honors Program Course
Dual Enrollment Course
Opened August 2017, Tuition
Free MOOC Series Opened July 2018, Free
Opened January 2018, Free*
Video CoursePlatform
Assignment PlatformProctoring Platform
Adaptive TextbookPlatform
Course Content
Partnerships
20% Headshot 80% Screencapture
All Course Content Available at bit.ly/CS1301PC
Video Content
Programming Problems Multiple Choice & Text Entry
All Course Content Available at bit.ly/CS1301PC
Exercises
Integrated Exercises
All Course Content Available at bit.ly/CS1301PC
Random Test Cases
All Course Content Available at bit.ly/CS1301PC
Sample Answers
All Course Content Available at bit.ly/CS1301PC
Timed, Proctored Tests
All Course Content Available at bit.ly/CS1301PC
Total Volume:Over 500 coding problems
All Course Content Available at bit.ly/CS1301PC
Automated Evaluation
All Course Content Available at bit.ly/CS1301PC
Phineas (Global
Autograder)
Student Code
Ferb (Local Autograder)
Grade
Feedback
Results
2017 2019
J F M A M J J A S O N D J F M A M J J A S O N D J F M
2018
Georgia Tech Course
Free MOOCFree MOOC (Series)
Honors Program CourseDE Course
Verified MOOCCertificate MOOC (Series)
A M J J A S O N D
0
200
400
600
800
1000
1200
1400
1600
1800
Enrollment Over Time (Georgia Tech)
2017 2018 2019
0
50000
100000
150000
200000
250000
Enrollment Over Time (All Versions)Georgia Tech Free MOOC Verified MOOC Free MOOC Series Verified MOOC Series
2017 2018 2019
Georgia Tech, 2069
Free MOOC, 678
Verified MOOC, 239
Free MOOC Series, 3300
Certificate MOOC Series, 5564
Total Course Completers by Version
Research Findings
Students in CS1301 online class perform as well as1 or better than2 students in a
traditional version of CS1301 in a pre-test/post-test setup.
1: Joyner, D. A. (2018). Toward CS1 at Scale: Building and Testing a MOOC-for-Credit Candidate. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale. London, United Kingdom. ACM
Press.
2: Joyner, D. A. & McDaniel, M. (2019). Replicating and Unraveling Performance and Behavioral Differences between an Online and a Traditional CS Course. In Proceedings of the ACM Global
Computing Education Conference (CompEd). Chengdu, China. ACM Press.
Research Findings
Students who previously failed or withdrew from a CS class prefer the online version,
and perform better in the online version than the traditional version1.
1: Joyner, D. A. (2018). Intelligent Evaluation and Feedback in Support of a Credit-Bearing MOOC. In Proceedings of the 19th International Conference on Artificial Intelligence in Education. London, United
Kingdom. Springer.
Research Findings
Students rate the online version of CS1301 as higher quality than other college courses
they have taken1.
1: Joyner, D. A. (2018). Toward CS1 at Scale: Building and Testing a MOOC-for-Credit Candidate. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale. London, United Kingdom. ACM
Press.
Awards
Bronze Award: Computing & IT Discipline2019 Reimagine Education Awards
Class Central Top 100 Course ofAll-Time (2019)
For more:bit.ly/CS1301PC
(the whole course in MOOC format)
LucyLabs.gatech.edu(research on the course)
DavidJoyner.net(me!)
Questions?