metaverse t o mooc: scaling virtual worlds in the cloud?

METAVERSE TO MOOC:

SCALING VIRTUAL WORLDS IN

THE CLOUD?C.J. DaviesColin AllisonIain Oliver

John McCafferyAlan Miller

motivation

• MOOCs are open and massive– cope with tens of thousands of learners

• Open Virtual Worlds (OWV) are open and small– can support hundreds at best, often less

• MOOCs and OVWs are complementary educationally – MOOCs consist of static resources for download / streaming /

consumption– OVWs allow for constructivist multi-user interaction

• Can the Cloud be used to scale OVWs for MOOCs?

3

structure of this talk• Overview of Open Virtual Worlds at St Andrews

– example from STEM area of computer networking education

• What is meant by “Open” and “Massive” ?• Methodology

– design of a benchmark and testbed

• Measurements– metal, virtual machines, Amazon ec2

• Comments and Conclusions

4

Open Virtual Worlds @ St Andrews• STEM education

– Internet routing– 802.11 wireless protocols– Algorithm animation and visualisation

• Cultural Heritage and Education– Digital Tourism, Digital Preservation, Historic Scotland– national school curriculum, Education Scotland– Archaeology fieldwork training

• Mobile Cross Reality (see talk on Tuesday, 16:00, Heights)

• Novel User Interfaces– Xbox 360, Kinect, commodity-based CAVEs

STEM education example

Internet Routing Protocols

6

Internet Routing• Hierarchical

– billions of nodes• Internet organised into Autonomous Systems

– AS usually organised into regions• Routing between AS: exterior routing

– usually Border Gateway Protocol (BGPv4)• Routing within AS: interior routing

– Link State or Distance Vector

7

Fife and Tayside regional network

8

RNEP: Regional Network Entry PointPOP: Point of Presence

9

Link from RNEP1 to Abertay is broken (interactively by student)

10after watchable protocol exchanges the forwarding table for RNEP1 changes

imagine: an internet core as a hypercube (k=4) rather than a mesh

11

k=4: visualisation gets tricky in 2D

h t t p : / / b l ogs . c s . s t - a nd r ews . a c . u k / openv i r t u a lwo r l d s 12

0000 0001

0010

0011

0100 0101

01110110

1000 1001

1101

11111110

1100

1010 1011

hypercube k=4

http://www.cs.berkeley.edu/~demmel/cs267-1995/lecture11/Hypercube4D.gif 13

k=4: 3D virtual world visualisation

14

routing island complements other learning resources

• two lecture theatres (DV and OSPF)– displays content from youtube, web pages and other

media• document centre (internet standards docs etc)

• pre-canned simulations of textbook examples– Peterson & Davie– Tanebaum & Weatherall– Kurose & Ross

• multiple sandbox areas– build your own network

15

document centre

16

popular youtube video of djikstra’s algorithm

17

OSPF example from Peterson and Davie

18

Kurose & Ross

Fig. 4.27

19

What is meant by “Open” ?

20

UNESCO, 2012• “Open Educational Resources (OERs) are any type of

educational materials that are in the public domain or introduced with an open license.

• The nature of these open materials means that anyone can legally and freely copy, use, adapt and re-share them.

• OERs range from textbooks to curricula, syllabi, lecture notes, assignments, tests, projects, audio, video and animation.”

21

How Open are MOOCs and OVWs?

• The UNESCO definition is far more open than most open source licenses– “anyone can legally and freely copy, use, adapt and re-

share them”• MOOC components seem to meet this• OVWs:

– anyone can visit or download an OVW, and then interact with it

– they can’t necessarily see or take away the underlying code or graphical design

22

What is meant by “Massive” (i)

• MOOCs– tens of thousands of registered learners

• aside: less than 10% of participants complete a course– asynchronous, one-way: mostly download of prepared

resources• video streaming, slides, docs, etc.

– interactive features• asynchronous text-based interactive forums• online MCQs

23

What is meant by “Massive”? (ii)

• Open Virtual Worlds based on OpenSim– synchronous interaction, user (avatar) driven, dynamic

updates to shared environment– at very best hundreds of concurrent avatars– also depends on number of prims and complexity of code

• Routing Island grinds to a standstill with 12 pro-active users carrying out experiments

• Cathedral mega-region good up to ~ 80 pro-active avatars

24

Scalability and Variance in Load

• For asynchronous MOOCs the load can vary but as interaction is always asynchronous frustrated users can simply go away and leave a download running or try again later

• For synchronous OVWs a transient peak demand can bring a region to a standstill and/or crash the server

• If an OVW was incorporated as a learning resource in a MOOC it would not cope

25

OVWs: coping with variance in load• OVWs are synchronous but typical load may be low e.g. less

than 5 avatars

• A high load e.g. more than 50 avatars may be caused by a scheduled event

• MOOC access would have to be regulated like an art exhibition – by ticket and time

• Still need to increase the capacity for such scheduled events

• The Cloud offers pay per use scalability – a good match?

26

testing scalability: methodology• design benchmark

– calibrate bot and human behaviour– establish a close match and use that pattern

• build testbed and conduct experiments– use bots to facilitate exploration of parameter space

• QoE parameters– Frame Time found to be the best measurable discriminator as to

load and performance– Frames per Second

• ideally at least the refresh rate of the display device e.g. 60 fps• in practice 30 fps or better acceptable

27

Walk-2 best fit for human behaviour

tests

• 5 – 100 bots in increments of 5• Each bot executes a pattern of behaviour for 10

minutes that matches typical human controlled avatar

• Each run repeated three times

29

platforms and virtualisation

• Cathedral Island• Metal: Quad core i7, 8GB

– Xen: dom0 and domU – KVM– Virtual Box

• Amazon ec2 extra large (M1: quad core, 16GB)

30

frames per second v number of avatars

h t t p : / / b l ogs . c s . s t - a nd r ews . a c . u k / openv i r t u a lwo r l d s 31

frame time (ms) vs number of avatars

h t t p : / / b l ogs . c s . s t - a nd r ews . a c . u k / openv i r t u a lwo r l d s 32

Comments on the Cloud for OVWs

• Scaling up is easy once image of OVW is created– simply change the underlying AWS machine type

• Disappointing performance from tests to date, but more powerful machine types are becoming available

• Still useful to know that for $20 you can run an OVW session for 50 students for 2 hours without owning any server hardware!

33

Comments on OVW Scalability

• Number of Concurrent Avatars is only one view of scalability

• Other approaches include replicating regions and limiting the number of avatars on each replica– no longer a single large multi-user interactive environment– but, preserves interactive learning resource functionality

• Fundamentally re-think the architecture e.g. distributed scene graph

34

Conclusions• OVWs and MOOCs complementary educationally• OVWs would need to be scheduled with tickets and

times if made available as MOOC resources• Cloud is potentially good fit for scheduled sessions of

known loads• Loads can be predicted using benchmark and testbed• Current Cloud virtual machines do not scale or

perform better than dedicated commodity hardware• There are different approaches to OVW scalability

35

THANK YOU!

36

comments and collaborations welcome

Colin Allisonca@st-andrews.ac.uk