Provisioning Canvas:Manual,Batch,or Transactional?

Brent Shawbs@ucf.edu

Senior Systems ProgrammerCenter for Distributed

LearningUniversity of Central Florida

Provisioning Canvas 18 June 2014Page 2 / 253

• 59,770 students (Spring 2014)• 1,965 faculty & adjuncts

• 12 regional campuses• 2013/4 budget: $1.45 B • 180 buildings on 1,415 acres

• Classes started in October, 1968

About the University of Central Florida

Provisioning Canvas 18 June 2014Page 3 / 253

About Me

• Yes, bs@ucf.edu really is my email address

• UCF’s first LMS administrator in 1998 (when WebCT was in Beta for the first time)

• Worked for WebCT Inc. for a year• I’m a lazy programmer.• This is boring. Please laugh.

Provisioning Canvas 18 June 2014Page 4 / 253

Words of Wisdom

“I can build one of the best networks in the

world…

“Until people want to start using it.”

-Anonymous Network Architecture

Instructor

Canvas is Awesome!

But do we HEROs really have to add actual users?!?

Brent Shawbs@ucf.edu

Senior Systems ProgrammerCenter for Distributed

LearningUniversity of Central Florida

Provisioning Canvas

The Provisioning Triad ™Which (1) COURSE / SECTION

is being taught by which (2) INSTRUCTORto which (3) STUDENTS ?

If you don’t have that information at a minimum,

this presentation won’t help (much).

More information is better!18 June 2014Page 6 / 253

Provisioning Canvas 18 June 2014Page 7 / 253

The SIS has The Triad ™

• Colleges / Departments Build Class Schedules– Create Classes and Sections (1)– Assign Instructors (2)

• Students Enroll in Classes (3)

• Implicit in the Triad are users.– “Triad” is cooler than “Quartet”

Provisioning Canvas

The SIS has More Information• Start / Stop dates of classes• Terms• Crosslisting• TAs / Observers, other roles• Changes in information !!!• Adds / Drops

– Courses– Enrollments

18 June 2014Page 8 / 253

Provisioning Canvas

Provisioning: SIS Information LMS

• Information has to get from the SIS to the LMS:1. Manually2. Batches3. Transactions

• Some combination of manual, batches and transactions

18 June 2014Page 9 / 253

Provisioning Canvas

Manual

• Often involves someone typing directly

into Canvas• Complete Free for All!

– Allow people to create their own user accounts

– Allow users to create their own classes / sect.

– Allow users to self enroll – Allow instructors to enroll students

• Administrator manually matching your SIS

18 June 2014Page 10 / 253

Provisioning Canvas

ManualPros

• Hands off for administrators

• Valuable for sub-accounts

• Allows for flexibility– Personnel training not

contained in SIS• Great for “one offs”

– Pragmatic solutions– Gets the job done

Cons• Wild Wild West• Little control• Auditing / Reporting is

difficult ($$$)• Difficult to capture

changes in data• Not scalable

18 June 2014Page 11 / 253

Provisioning Canvas

Batches• Chunks of data processed all at once

– As little as one record– As much as 249,000 users

• In Canvas this means a CSV file– CSVs can be uploaded into Canvas 2 ways

• Manually through the GUI• Programmatically through the APIs

• CSVs Allows for snapshots and incrementals

18 June 2014Page 12 / 253

Provisioning Canvas

BatchesPros

• CSVs are a simple format

• Easy to export data from SIS to a CSV file

• Can be run multiple times a day

• Can be uploaded manually or with APIs

• Greatest return on investment

Cons• Not “immediate”

• CSV files will be queued in the order they are received

• Not granular

• Catching errors is more difficult

18 June 2014Page 13 / 253

Provisioning Canvas

Transactions

• Triggered by a change in the SIS– Could be a new item– Course be a change

• A single “piece” of information is sent to the LMS immediately using the API– Success or failure is immediately

known18 June 2014Page 14 / 253

Provisioning Canvas

TransactionsPros

• Immediacy

• Allows for better error handling

Cons• Potentially lots of API

call need to be made in a short period of time – 3k / hour limit

• Harder to program– Must catch errors– Failures won’t be fixed

in a snapshot

18 June 2014Page 15 / 253

Provisioning Canvas

Technical Details for Provisioning

Implementation• Planning• SISIDs• Canvas CSV Files• Batches• Transactions• A word about pagination• APIs for batches / CSV uploads• APIs for transactions (reference only)

18 June 2014Page 16 / 253

Provisioning Canvas

Planning for Your Provisioning• Time spent planning is never wasted• Get to know your data

– What are the quirks in the data?• Which do you have to account for?• Which can you ignore?

– Trick question! You shouldn’t ignore any of the quirks

• Get to know your data source– Who will get you The Triad?– How often is data available?

18 June 2014Page 17 / 253

Provisioning Canvas

Planning for Your Provisioning

• What do users expect?– Students want it now!!!

• Transactions for enrollments?– Instructors want it correctly.– Registrars want it officially.– SIS administrators want it

conveniently.

18 June 2014Page 18 / 253

Provisioning Canvas 18 June 2014Page 19 / 253

SISIDs: Choose Wisely• SIDID = Student Information System

ID– Unique identifier

• Automatically generated by your SIS

• SISID is the link between objects in your LMS and your SIS

• Lots of things have SISIDs in an LMS:– Users– Courses and Sections– Terms

Provisioning Canvas

SISIDs: Examples• Our User SISID looks like this:

0111614• Our Course SISID looks like this:

1470-UCF01-GEO-1200-SEC0001• Every section gets a course and section

– The section SIS ID is: 1470-UCF01-GEO-1200-SEC0001

– The course SIS ID is: 1470-UCF01-GEO-1200-SEC0001

18 June 2014Page 20 / 253

Provisioning Canvas

Canvas CSV Files

• In theory CSV files are standard• In practice CSV files are different

• https://canvas.instructure.com/doc/api/file.sis_csv.html

18 June 2014Page 21 / 253

Provisioning Canvas

Canvas CSV FilesThings that can be added to Canvas using CSV

files

1. Users 2. Sub-accounts 3. Groups4. Group memberships5. Terms6. Courses7. Sections8. Enrollments9. Crosslisting

• That order is important• Not all of them have to

be created• The Triad would be:

0. Sub-accounts • (manually created?)

1. Users2. Courses3. (Sections)4. Enrollments

18 June 2014Page 22 / 253

Provisioning Canvas

A Word About Pagination• For API calls that return large data sets

Canvas returns chunks of data• Ex: An API call with1,000 results would

take 10 API calls returning chunks of 100

• Create a wrapper to automatically handle pagination of API calls

https://canvas.instructure.com/doc/api/file.pagination.html

18 June 2014Page 23 / 253

Provisioning Canvas

APIs for Batches - Upload

• Upload a single CSV-- OR --

• Upload a ZIP file containing several CSV files

https://canvas.instructure.com/doc/api/sis_imports.html#method.sis_imports_api.create

18 June 2014Page 24 / 253

Provisioning Canvas

APIs for Batches - Upload

Request (POST with Attachment)

POSThttps://webcourses2c.beta.instructure.com/api/v1/accounts/89347/sis_imports

18 June 2014Page 25 / 253

Provisioning Canvas

POST https://webcourses2c.beta.instructure.com/api/v1/accounts/89347/sis_imports Authorization: Bearer <REALLY_POWERFUL_TOKEN_GOES_HERE>Content-Length: 58095Content-Type: multipart/form-data; boundary=xYzZY --xYzZYContent-Disposition: form-data; name="import_type" instructure_csv--xYzZYContent-Disposition: form-data; name="attachment"; filename="2014-06-16-13:19:12-2014-06-16-12:00:04-CANVAS_MEMBER.CSV"Content-Type: text/csv course_id,user_id,role,section_id,status1515-UCF01-BMS-6001-SEC0001,0111614,teacher,1515-UCF01-BMS-6001-SEC0001,active 

18 June 2014Page 26 / 253

Provisioning Canvas

APIs for Batches - Upload• Response (JSON)

– Returns a creation object{        'ended_at' => undef,          'created_at' => '2014-06-16T17:32:13Z',          'progress' => 0,          'updated_at' => '2014-06-16T17:32:13Z',          'data' => {'import_type' => 'instructure_csv' },          'id' => 5167983,          'workflow_state' => 'created'        };

18 June 2014Page 27 / 253

Provisioning Canvas

A Word About batch_mode

• The “batch_mode” parameter is available for SIS CSV file uploads.

• Don’t use it unless you know exactly what it does. (It overwrites everything.)

• Make sure you test it on BETA / TEST.

• More information about batch_mode:– http://guides.instructure.com/m/

4214/l/80122-what-are-sis-imports

18 June 2014Page 28 / 253

Provisioning Canvas

A Word About Stickiness

• Stickiness in Canvas is the idea that changes to information made through the GUI will not be overwritten by SIS imports.

• Useful for manual tweaks via CSV file

• Not all fields respect stickiness

18 June 2014Page 29 / 253

Provisioning Canvas

A Word About StickinessNot all fields are “Sticky” – These are the Sticky fields

Account name

Abstract_courseNameshort_nameenrollment_term_id

Coursenamecourse_codestart_atconclude_atrestrict_enrollments_to_course_datesenrollment_term_idworkflow_state

18 June 2014Page 30 / 253

Sectioncourse_idnamestart_atend_atrestrict_enrollments_to_section_dates

Enrollmentstart_atend_at

Termnamestart_atend_at

Groupname

Usernamesortable_nameshort_namePseudonym (Login)unique_id (login_id, username...)

Provisioning Canvas

APIs for Batches - Status

• Now that you’ve uploaded it, you’ll want to see if it finishes

https://canvas.instructure.com/doc/api/sis_imports.html#method.sis_imports_api.show

18 June 2014Page 31 / 253

Provisioning Canvas

APIs for Batches - Status

• Request (GET)– Use the SIS Upload ID from the status

object returned from the file upload

GEThttps://webcourses2c.beta.instructure.com/api/v1/accounts/89347/sis_imports/5167983

18 June 2014Page 32 / 253

Provisioning Canvas

APIs for Batches - Status• Response (JSON)– In Progress

{   "created_at": "2014-06-16T18:06:07Z",    "ended_at": null,    "updated_at": "2014-06-16T18:06:08Z",    "progress": 1,    "id": 5167983,    "workflow_state": "importing",    "data": {        "import_type": "instructure_csv",        "supplied_batches": ["user"] }}

18 June 2014Page 33 / 253

Provisioning Canvas

APIs for Batches - StatusCOMPLETED{"created_at": "2014-06-16T07:00:11Z","ended_at": "2014-06-16T10:31:18Z","updated_at": "2014-06-16T10:31:18Z","progress": 100,"id": 5167983,"workflow_state": "imported_with_messages","data": { "import_type": "instructure_csv",

"supplied_batches": ["user"],"counts": {"accounts": 0, "terms": 0,

"abstract_courses": 0,"courses": 0, "sections": 0, "xlists": 0,"users": 249999,"enrollments": 0, "groups": 0,"group_memberships": 0,

"grade_publishing_results": 0}},"processing_warnings": [["attachment_3703675020140616-22926-1h0xf.CSV", "user 3600989 has already claimed 3619372's requested login information, skipping"]]}

18 June 2014Page 34 / 253

Provisioning Canvas

APIs for Batches - Show All

• Canvas will tell you about all your SIS imports

https://canvas.instructure.com/doc/api/sis_imports.html#method.sis_imports_api.index

18 June 2014Page 35 / 253

Provisioning Canvas

APIs for Batches - Show All

• Request (GET)

GEThttps://webcourses2c.beta.instructure.com/api/v1/accounts/89347/sis_imports

18 June 2014Page 36 / 253

Provisioning Canvas

APIs for Batches - Show All

• Response (JSON)

• List of JSON objects– In Progress– Completed

18 June 2014Page 37 / 253

Provisioning Canvas

APIs for Transactions

• Far too many to give examples

• I’ll list the APIs that could be used for provisioning Canvas

18 June 2014Page 38 / 253

Provisioning Canvas

APIs for Transactions - Users

• Create a user• Update user settings• Edit a user• Delete a user• Merge user into another userhttps://canvas.instructure.com/doc/api/users.html 18 June 2014Page 39 / 253

Provisioning Canvas

APIs for Transactions - Sub-accounts

• List accounts• Get a single (sub)account• Get the sub-accounts of an

account• Update a (sub)account• Create a new sub-accounthttps://canvas.instructure.com/doc/api/accounts.html

18 June 2014Page 40 / 253

Provisioning Canvas

APIs for Transactions -Groups

• Get a single group• Create a group• Edit a group• Delete a group• Invite others to a grouphttps://canvas.instructure.com/doc/api/groups.html 18 June 2014Page 41 / 253

Provisioning Canvas

APIs for Transactions -Terms

• List Enrollment Terms

(That’s all- can’t create or edit with APIs)

https://canvas.instructure.com/doc/api/enrollment_terms.html

18 June 2014Page 42 / 253

Provisioning Canvas

APIs for Transactions -Courses• Create a new course• Conclude a course• Get course settings• Update course settings• Get a single course• Update a course• Update courseshttps://canvas.instructure.com/doc/api/courses.html 18 June 2014Page 43 / 253

Provisioning Canvas

APIs for Transactions -Sections• List course sections• Create course section• Cross-list a Section• De-cross-list a Section• Edit a section• Get section information• Delete a sectionhttps://canvas.instructure.com/doc/api/sections.html 18 June 2014Page 44 / 253

Provisioning Canvas

APIs for Transactions -Enrollments

• Enroll a user• Conclude an enrollment

– Deleted Active– Active Concluded– Concluded Deleted– Active Deleted

https://canvas.instructure.com/doc/api/enrollments.html

18 June 2014Page 45 / 253

Provisioning Canvas

APIs for Transactions -Crosslisting

• Cross-list a Section• De-cross-list a Section

https://canvas.instructure.com/doc/api/sections.html

18 June 2014Page 46 / 253

Provisioning Canvas

Combination of Techniques• Manual:

– Handle things not in the SIS (Training…)– Manage some sub-accounts

• Allows for delegation of responsibility– Crosslisting

• Batch:– Users: creation & changes - Once a day

– Course / Section Creation - Multiple times a day

• Transaction:– Enrollments

18 June 2014Page 47 / 253

Questions?Comments?Suggestions?

Brent Shawbs@ucf.edu

Provisioning Canvas 18 June 2014Page 49 / 253

If you liked it …

Please tell people you saw Brent from the

University of Central Florida

Provisioning Canvas 18 June 2014Page 50 / 253

If you didn’t like it …Please tell people you saw someone’s

random Central Florida vacation pictures…