georgetown university february 12, 2007

Use of the Blackboard Enterprise Learning System Page 1 of 28

Georgetown University February 12, 2007

Preface This analysis of the use of the Blackboard Enterprise Learning System for the Spring term 2006 is intended to facilitate the dialog with faculty and instructional technologists engaged in business continuity planning. In colleges and universities business continuity planning for information technology begins with sustaining the computing and communications capabilities; George Washington University was an early implementation and Georgetown University has similar capabilities. The planning continues by adding infrastructure—such as podcasts and Webcasts, audio and video conferencing, and access to digital content—used by faculty during a period when classroom and library facilities are not available. The third phase is the most difficult—changing learning and teaching methods to be effective when dependent upon eLearning via the Internet. This analysis may provide some additional insight to early discussions. Georgetown University is proud of the rich interaction between students and faculty, the extensive materials and services available from its library, and the ability of its students. It is challenging to even imagine an environment that provides a similar opportunity for learning when students and faculty have no classrooms. Washington, DC has experienced buildings that had to be evacuated for several weeks because of anthrax or several days because of large protests. And less than two miles from the Pentagon on September 11th. Various parts of Georgetown have experienced days without electricity as “manholes exploded” from deteriorated infrastructure. But the experience of our colleagues at Loyola University during and following Hurricane Katrina—that they have openly, honestly and willingly shared with the higher education community—suggests it is possible that some event could make the Georgetown campus unavailable to faculty and students for an extended period of time. This analysis is a contribution to “Thinking about the Unthinkable.”1

1 It To use Herman Kahn’s phrase from his 1962 book of the same title.



1.0 Introduction 1.1 The University Georgetown University comprises four undergraduate schools, three graduate and professional schools, professional development programs and certificates, medical residencies and other programs predicated on the liberal arts tradition at the heart of the institution.2 The university includes

• Georgetown College Arts, humanities, languages, social sciences and sciences

• Edmund A. Walsh School of Foreign Service (SFS) International affairs, regional and comparative studies

• Robert Emmett McDonough School of Business (MSB) Business administration, MBA programs, executive education

• School of Nursing and Health Studies (SNHS) Nursing, human science, health systems and management

• Graduate School of Arts and Sciences Liberal arts, interdisciplinary programs, biomedical, and public policy

• School of Medicine (SOM) General medicine and medical specialties; medical research

• Law Center Constitutional, criminal, and international law; corporate finance

• School of Continuing Studies (SCS) Part-time degrees, professional development and summer studies

2 See Ref. 10.



The Spring 2001 and Spring 2006 enrollments for of the College and Schools are shown in Table 1.1. These enrollments provide some perspective on the potential uses of a learning system.3 Spring 2000 Spring 2006 Georgetown College 3,210 3,152 School of Foreign Service 1,144 1,218 School of Business 1,213 1,202 School of Nursing and Health Services

300 473

Graduate School of Arts and Sciences

1,925 2,538

Graduate School of Medicine 388 661 School of Medicine 709 757 Law Center 2,505 2,402 School of Continuing Studies 494 5204

Total 11,888 12,923 Table 1.1 – Spring 2001 and Spring 2006 Enrollments

1.2 eLearning at Georgetown University The Blackboard learning system is used by the college and all of the schools. Faculty may also be using online services from textbook publishers or using other online services in their teaching.5 Information about the possible use of these resources is not included in this analysis. There are three organizations with the University that support eLearning. Center for New Designs for Learning and Scholarship Since 2000, CNDLS has supported faculty and graduate students with tools, resources, and opportunities for new learning environments. We began with a mission to bridge a historic gulf between pedagogy and technological advances, and today CNDLS integrates a teaching and learning center with the latest educational technology.6 CNDLS encourages postsecondary educators to critically review their teaching practices. We help them implement their big ideas, support their research, and incorporate technology into their course design. We consult with them as they refine new educational approaches and design programs for assessment so that they understand their

3 These enrollments were developed from data in the Registrar’s enrollment database by Jim Siantz. The Spring 2001 data is from Georgetown University Official Enrollment Statistics as of January 29, 2001 Office of the Registrar; Spring 2006 data is from the Official Enrollment Statistics as of January 31, 2006, Office of the Registrar.

4 Includes 25 students from Georgetown University Qatar.

5 Results of national surveys of faculty are provided in Ref. 8. Because of the preparation of its students and the methods of instruction, and the university-wide availability of Blackboard supported by the Center for New Designs in Learning and Scholarship (CNDLS) and the Medical Schools’ Faculty and Curriculum Support Center (FACS), it is likely Georgetown University faculty use of publisher’s online services is very limited, if at all. 6 See Ref. 7.



impact on students. We're proud to have participated in a variety of projects that successfully uncover students' learning process. Because of the early focus on the use of learning technology, faculty use of the technology has moved beyond “early adopters” to the “early majority.”7 Faculty and Curriculum Support (FACS) Center, School of Medicine

The School of Medicine’s Faculty and Curriculum Support (FACS) Center describes their role:8

The Faculty and Curriculum Support (FACS) Center of Dahlgren Library has a division of Instructional Design and Multimedia Development. The staff in this division provide advanced consultation, classroom instruction, and individual assistance in a small multimedia computer center to help faculty incorporate technologies into the medical school and nursing curricula.

The staff also support Blackboard, ColdFusion web database development, PDA and handheld devices, and computer-based course materials. Scholarly Information Services, University Information Systems9 Although not a formal organization within University Information Systems, Scholarly Information Services, headed by UIS’ Principal Technologist, directly supports pedagogical and research activities at Georgetown. This responsibility has expanded to include support of the Blackboard system, integration of learning systems with library and enterprise administrative systems, and liaison with organizations active in this area. These include Internet 2 and the Shibboleth project, Sakai Foundation, Open Source Portfolio Initiative, JA-SIG and its uPortal software, the Moodle Foundation, UK’s Joint Information Systems Committee, the Postsecondary Electronic Standards Council, the IMS Global Learning Consortium and Educause and its Learning Initiative. The Principal Technologist and staff also actively participate in Common Solutions Group (CSG), the Blackboard Product Advisory

7 Using terminology from Geoffrey Moore’s “Crossing the Chasm” published in 1991. See Ref. 9.

8 The Web page from which this quotation was taken January 1, 2007 is no longer available. Similar information was available from dml.georgetown.edu/resources/8491.html on February 12, 2007; the link to FACS no longer works.

9 Now named the Scholarly Systems Group.



Board, the Association for Computing Machinery (ACM), the Data Warehousing Institute (TDWI), and the Association of Jesuit Colleges and Universities Conference on IT Management (AJCU-CITM).10 This organization facilitates the move of instructional technology from research, through development and into implementation. To ensure long-term sustainability, Scholarly Information Services participates in open standards development and implementation. 1.3 The Blackboard Learning System at the University

The Center for New Designs for Learning and Scholarship describes the use of the Blackboard learning system:11

Blackboard delivers a powerful, easy-to-use course management system that enables instructors to provide their students with course materials, communication tools, online assessments, and a dedicated academic resource center on the Web.

Blackboard offers a course site framework with sections for announcements, course information (syllabus, attendance policy, etc.), and course documents (handouts, assignments, etc.). In addition, the program has a built-in feature to create online tests and surveys. These tests can be automatically graded with the results recorded in an online grade book.

Blackboard also has several built-in communication tools including a threaded discussion forum and a real-time chat program, complete with an interactive whiteboard. All of these features are under the control of the instructor, making it easy to develop and maintain a robust online course site.

CNDLS offers a number of Blackboard Training Classes, as does FACS at the Medical Center.

The Blackboad learning system was installed in 1998, made widely available to faculty in 2000, and has been effectively serving a continually increase number of courses. CNDLS summarizes this history writing:12

10 See Ref. 11.

11 See Ref. 1.

12 See Ref. 2.



Since Fall 1998, Georgetown faculty have used the Blackboard Course Management system to extend the learning environment, including posting course materials, hosting online class discussions, and quizzing students.

2005-2006 Academic Year Statistics

• Over 11,000 unique student, faculty, and staff users • 2,303 active course sites (Fall 2005, Spring 2006, and

Summer 2006 available course sites) • 1,456 Blackboard help requests handled by the Blackboard

Support Team • 31 Blackboard Training Courses taught

The historical growth in active users of the Blackboard learning system are shown in Figure 1. The growing number of courses is shown in Figure 2.

Active Users of Blackboard Learning System

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

1998-

1999

1999-

2000

2000-

2001

2001-

2002

2002-

2003

2003-

2004

2004-

2005

2005-

2006

2006-

2007

Academic Year

Num

ber

of A

ctiv

e U

sers

Fall Semester

Spring Semester

Summer Session

Active users include students, faculty, and staff authorized to use the Blackboard Learning System.

Fig. 1 Active Learning System Users by Term



Active Courses Using the Blackboard Learning

System

0

500

1,000

1,500

2,000

2,500

1998-

1999

1999-

2000

2000-

2001

2001-

2002

2002-

2003

2003-

2004

2004-

2005

2005-

2006

2006-

2007

Academic Year

Num

ber

of A

ctive C

ours

es Fall Semester

Spring Semester

Summer Session

Academic Year

In Spring 2006 Georgetown University offered 3,413 course offerings. In Blackboard a course offering is called a “course,” subsequently the same terminology is used here. For example, if there were, say, three sections of the same course, this would be one course and three course offerings. In Blackboard this can be represented as three ‘courses’ with separate enrollment, or with later versions, it can be a single course if all students are enrolled in that course. In addition to courses, Blackboard also is used by organizations. The use of Blackboard by organizations is not included in this analysis. As shown in Table 1.2, in Spring 2006 2,640 Blackboard courses had student enrollments in the Blackboard learning system. There were a total of 51,690 course enrollments. 1,057 had “content” where content count was defined as “The sum of the following content areas: documents authored in Blackboard, links to deployed assessments, links to deployed course surveys, assignments, links within courses to document, documents, links to resources external to Blackboard, files, and lessons. This data element was used to distinguish between courses that use Blackboard primarily for collaboration from those where course management information or course content is delivered by Blackboard.13 Courses with enrollment, but without “content” appear to be courses where there is enrollment and the learning system may or may not have been used for instruction or learning. Course with content, but without enrollment, may

Fig. 2 Active Learning System Courses by Term

13 The data elements (variables), description, count, and averages are given in Appendix B.



represent courses that were planned to use the learning system, but subsequently did not. The number of courses classified by content and enrollment are shown in Table 1.3. This analysis focuses on Blackboard courses with content and with enrollment.

Course Size

Courses with Enrollment

Courses with Content

Course Enrollments

1-5 692 55 1,521 6-15 839 328 8,635 16-35 774 420 17,388 36-60 221 141 9,998 61-100 63 52 4,743 101 or more 51 44 9,405

Total 2,640 1,040 51,690 only one student 321 8 321

Table 1.2 – Courses by Course Enrollment, Spring 2006

Enrollment With Without Total

With 1,040 17 1,057 Content Without 1,600 756 2,356 2,640 773 3,413

Table 1.3 – Number of Courses with Content or Enrollment

Course enrollments using the same classifications of Table 1.3 are given in Table 1.4.

Course

Enrollments With 31,613 Content Without 20,077

51,690 Table 1.4 – Course Enrollments in Courses With and

Without Content The distribution of courses and course enrollment by course size is shown in Figure 3. This suggests the 12, 923 students and 31,613 course enrollment yields an average number of course enrollments per student of 2.4 courses. These are courses with enrollment and content.



The difference in distribution between courses and enrollment comes from the number of small classes that have few enrollments and the very few classes that have large enrollment. The difference in these two distributions becomes important in developing a strategy to maximize the use of the learning system when classrooms, seminar rooms, and faculty offices are not available—a scenario for continuity planning. For planning purposes, a relationship is shown between the number of courses and course enrollment in Figure 4 for course enrolling 1 to 60 students.

Similarly the relationships between total student enrollment and course size is shown in Figure 5. The cumulative student enrollments are also shown. The polynomial

Distribution of Courses and Enrollment

by Blackboard Course Enrollment

0%

5%

10%

15%

20%

25%

30%

35%

40%

101 or more61-10036-6016-356-151-5

Number of Students Enrolled per Course, Spring 2006

Perc

ent

Courses

Enrollment

Number of Courses by Enrollment

0

10

20

30

40

50

60

70

0 5 10 15 20 25 30 35 40 45 50 55 60

Students Enrolled per Course, Spring 2006

Num

ber

of C

ours

es

Fig. 3 Distribution of Courses and Course Enrollment by Course Size

Fig. 4 Distribution of Courses by Course Enrollment



trendline for student enrollment by course size showing first increasing—as class size increases—and then decreasing student enrollments because fewer courses are offered at larger course size. The maximum enrollment—only by class size—is courses enrolling 32 students. The strategy to serve the largest number of students is to begin with courses having the larges class sizes.

Student Enrollment by Course Enrollment

0

100

200

300

400

500

600

700

800

0 5 10 15 20 25 30 35 40 45 50 55 60

Course Enrollment, Spring 2006

Stu

dent E

nro

llment

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

20,000

Cum

ula

tive E

nro

llment

Student Enrollment

Cumulative Enrollment

Poly. (Student Enrollment)

Fig. 5 Total Student Enrollment by Course Size and the Cumulative Total Enrollment



2.0 Use of the Blackboard Learning System 2.1 Blackboard Features Blackboard has an environment rich in features. The data set identifies 21 major features. Departmental use of these features is shown in Figure 6. A feature if considered “used” if any course in the department uses this feature. The purpose here is to identify which features are most widely used. The aggregated feature “Any Course Content” identifies courses that use any of several different features. This shows that every course with “content” includes documents.

Department Use of learning System Features

0% 20% 40% 60% 80% 100%

Internal linksDeployed surveys

Question pools

Available surveysDeployed assessments

Calender entriesAvailable assessments

Groups

ForumsMessage

Staff informationDropbox

External linksGradebook

Folders

AnnouncementsFiles

DocumentsAny Course Content

Percent of Departments, Spring 2006

Figure 7 shows the number of features used in a course. It shows the number of courses distributed by the number of features used. Most courses use few features, but there are some that use a number of different features reflecting a mature use. 8.2% of courses use 10 or more features, 36.3% use 5 or more. This suggests that campus-wide training should focus on the most frequently used features—documents, files, announcements, folders, gradebook, external links and dropbox. Other features would be offered separately for faculty who need the additional features.

Fig. 6 Use of Blackboard Features by Department



In addition to Blackboard features, e-mail, instant messaging, and Web 2.0 Internet applications are also used. The extent of that use is not yet documented.

Blackboard Features Used per Course

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

2018161412108642

Features Used in Courses Using Blackboard, Spring 2006

Perc

ent of C

ours

es

Class size could be a proxy for methods of instruction. The data was groups into six ranges of course size: 1-5 students, 6-15, 16-35, 36-60, 61-100 and more than 100. These were chosen to represent the characteristics of the physical facilities used for each class size. The use of the more frequently used features is shown in Figure 8. An alternate presentation of the same data is shown in Figure 9.

Use of Blackboard Features by Course Size

0%

20%

40%

60%

80%

100%

Doc

umen

tsFile

s

Ann

ounc

emen

ts

Folde

rs

Gra

debo

ok

Ext

erna

l lin

ks

Forum

Dro

pbox

Pe

rce

nt

of

Co

urs

es

1-5

6-15

16-35

36-60

61-100

101 or more

All

Fig. 7 Number of Courses by the Number of Features Used

Fig. 8 Use of Blackboard Learning System Features by Class Size



Use of Blackboard Features by Course Size

0%

20%

40%

60%

80%

100%

Documents Files Announcements Folders Gradebook

Perc

ent of C

ours

es U

sin

g

1-5

6-15

16-35

36-60

61-100

101 or more

As expected courses with small course size—1 to 15 students—tend to use documents and messages and forums proportionally more than courses with larger course size. Use of all of the features is given in Appendix A. The amount of content used in courses—using the aggregated course content value—varies from 1 to 302 items; the average for the 1,057 courses is 29.3 content items. For Spring 2006, there were 30,925 content items. The distribution of the use of content is shown in Figure 10.

Number of Courses by Content Items per Course

0

20

40

60

80

100

0 25 50 75 100 125 150 175 200 225 250 275 300

Content Items per Course

Num

ber

of C

ours

es

0

6,000

12,000

18,000

24,000

30,000

Conte

nt Item

s, A

ll C

ours

es

Number of Courses

Cumulative Content

Power (Number ofCourses)

Fig. 9 Use of Blackboard Learning System Features by Class Size

Fig. 10 Use of Content Items per Course



2.2 eLearning Content Perhaps a more useful view is the percent of courses by ranges of number of content items. This distribution is shown in Figure 11; the percentages may be more useful in planning.

Percent of Courses by Content Items

0%

5%

10%

15%

20%

25%

30%

35%

40%

1-10 11-

20

21-

30

31-

40

41-

50

51-

60

61-

70

71-

80

81-

90

91-

100

101-

110

111-

120

121-

130

131-

140

141-

150

more

than

150

Number of Content Items per Course

Perc

ent of C

ours

es

One-third of the courses have more than 30 items; less than 10%–9.6%–have more than 70 content items per course. Most of the content items are documents. The distribution of documents per course is shown in Figure 12. The 1,044 courses used 26,744 documents; this is 86.5% of all content.

Fig. 11 Distribution of Courses by Number of Content Items per Course



Number of Documents per Course

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1-10 21-30 41-50 61-70 81-90 101-110 121-130 141-150

Number of Documents

Perc

ent of C

ours

es, S

pring 2

006

Percent of Courses

Cumulative Percent

The average number of documents

per course is 25.6. The maxium

was 302. 1.2% had none.

2.3 The Blackboard Gradebook The Blackboard Learning System includes a gradebook. The gradebook can be used in conjunction with activities—that is the grade for each activity is recorded, or it can be used only for final grades. At Georgetown University the final course grade will be transferred to the SunGard SCT student information system at the end of the term.14 27.1% of the courses using the gradebook; 9.1% appear to use it only for the final course grade as shown in Figure 13.15 The distribution of courses using the gradebook is shown in Figure 14.

Fig. 12 Distribution of the Number of Documents per Course

14 The University is changing the student system from an earlier version to the Banner student information system

15 If there was only one grade, this was assumed to be the final grade for the course.



Use of Blackboard Gradebook

3.7%

96.3%

0%

20%

40%

60%

80%

100%

Gradebook only With course content

Num

ber

of C

ours

es


0%

2%

4%

6%

8%

10%

12%

0 5 10 15 20 25 30

Number of Graded Activites per Course, Spring 2006

Perc

ent of G

radebook U

sers

Note: 27% of Blackboard

courses use the

Blackboard gradebook.

For planning purposes, the cumulative distribution is shown in Figure 15.

Fig. 13 Use of Blackboard Gradebook

Fig. 14 Percent of Gradebook Users by Number of Graded Activities per Course




0%

20%

40%

60%

80%

100%

0 5 10 15 20 25 30

Number of Graded Activities, Spring 2006

Cu

mu

lative

Pe

rce

nt

Usin

g

4% of gradebook users have

more than 30 graded activities;

the maximum was 85

Figure 15 – Cumulative Percent of Gradebook Users by

Number of Graded Activities per Course

Fig. 15 Cumulative Percent of Gradebook Users by Number of Graded Activities per Course



3.0 Scope and Limitations 3.1 Scope of the Analysis The data and the analysis represent how the Blackboard Learning Systems was used during the Spring term 2006, but does not relate that use to either the broader learning environment or to the method of instruction. The analysis is descriptive. Faculty who use Blackboard may find these data and presentations useful in planning future use, but neither the data or the trends are prescriptive or even suggestive. The methods of instruction depend upon discipline and level of the subject matter, and the style and experience of faculty. This analysis is a single term of the nine years that Georgetown University has been using the Blackboard Learning System. The number of courses using Blackboard has grown suggesting more faculty are increasingly using Blackboard over time and likely have developed more effective use. 3.2 Limitations In addition to the Blackboard Learning System itself, students have e-mail and instant messages, may also be provided with others content—especially podcasts and webcasts, and may have been assigned to use other resources available on the Internet. This use of eLearning supplements the environment provided by the University—the library, media laboratories, and study areas. These resources, in turn, influence the use of Blackboard. Blackboard can be used to “manage” the learning process through a series of assignments. Learning is external to Blackboard, the student’s progress can be recorded in Blackboard. The number of “items” in the gradebook suggest this use of Blackboard may be expanding. The analysis then is a point in time of a continually changing use of Blackboard; Blackboard’s use will continually evolve as more is learned about learning itself and faculty incorporate new or different learning activities into their teaching.



4.0 Georgetown University Without a Campus 4.1 The Challenge It if difficult to conceive of a university—especially Georgetown University—without a campus. Without class rooms, faculty offices, library or many of the other facilities that faculty, staff, and students depend upon. But business continuity planning requires thinking about this option. The rich interactions among faculty and students may be impossible to replicate without a campus. There are limits to communication. University Information Systems is reviewing alternatives to enhance this community if required by an emergency. Although podcast and Webcast and audio and video conferencing technologies are available, extending them to the 13,000 students and 60,000 course enrollments is daunting. Although it is feasible to support video conferencing for some courses, maintaining video conferencing simultaneously for even a substantial fraction of more than 3,000 courses is much more challenging. 4.2 An Alternative The cost of developing course materials so the course can be taught within minimal faculty involvement represents a major investment in content that also is costly to maintain and, in many disciplines, obsolesces quickly. Likely the best alternative would, in conjunction with other colleges and universities, to make available facilities that Georgetown faculty, in a time of emergency, could create podcasts and webcasts for Georgetown classes. Where appropriate, this permits faculty to focus on creating additional content week-by-week during the emergency. If this is done, then Blackboard continues to supplement rather than replace instruction. As the use of Blackboard continues to increase both in number of courses using Blackboard and in the content and activities, Blackboard becomes more effective and more important. To maximize students served—the number of course enrollments available, the strategy is to focus on maintaining instruction for high enrollment courses. This



strategy is illustrated in Figure 16 where course enrollments are shown based on course size. 500 courses represent 57% of instruction; 1,000 77% and 1,500 91%. The Spring 2006 use of Blackboard illustrates how current practices, if maintained during an emergency the 1,040 courses has 78% of course enrollments. Those 1,040 courses with content have 61% of the course enrollments. The relationship of the two is shown in Figure 17.

Cumulative EnrollmentLargest Course Enrollments First

0

10,000

20,000

30,000

40,000

50,000

60,000

0 500 1,000 1,500 2,000 2,500 3,000 3,500

Number of Courses

Cours

e E

nro

llments

, S

rpin

g 2

006

Course Enrollments, Blackboard Learning

Systems, Spring 2006

0

10,000

20,000

30,000

40,000

50,000

60,000

0 500 1,000 1,500 2,000 2,500 3,000 3,500

Courses

Cum

ula

tive C

ours

e E

nro

llment

With Content

All Courses

Fig. 16 Cumulative Enrollment Based on the Number of Courses

Fig. 17 Cumulative Enrollment in Courses and in Courses Supported by Blackboard



4.3 An Emerging Strategy To ensure business continuity with a reasonable investment, the emerging strategy is:

1. Encourage further utilization of Blackboard as a learning management system. This provides the infrastructure for faculty and students to continue even though at geographically disparate locations.

2. Record current instruction as webcasts or podcasts with complementary visuals. This provides the repository of content that could be the basis for class content during an emergency whether or not it has a current role in instruction.

3. Accelerate use of the Blackboard learning system as a supplement to classroom instruction so, in conjunction with the recordings could support instruction during an emergency in a significant number of courses. Focus on the largest within each school or college first.

4. Continue the current research in the effective implementation and use of audio and video conferencing so the technology would be available and supported during an emergency.

5. Create additional documentation so faculty and students could learn, without further contact with the university, how to teach and learn during an emergency.

6. Continue the dialog to revise the emerging strategy so that it reflects a coherent and effective set of priorities rather than the simplistic strategy illustrated here.



References

1. Blackboard Course Management System. Retrieved January 1, 2007 from http://cndls.georgetown.edu/resources/8491.html

2. Blackboard Course Management System 2005-2006 Academic Year Statistics. Retrieved January 1, 2007 from http://cndls.georgetown.edu/blackboard/numbers.html

3. Blackboard Learning System: Instructor Manual, Release 6. (2003). Blackboard Inc., July 8, 2003.

4. Blackboard Learning System: Student Manual, Release 6. (2002). Blackboard inc., September 24, 2002.

5. e-Learning Competitive Landscape. (2004a). Blackboard Inc., October, 20 2004.

6. The Blackboard Academic Suite: Overview of Product Capabilities. (2004b) Blackboard Inc., December 4, 2004.

7. Center for New Design for Learning and Scholarship. (n.d.). Retrieved January 1, 2007 from http://cndls.georgetown.edu/index.cfm?fuseaction=main.about

8. Farmer, James (2006). Faculty Selection and Use of Publisher-Provided Textbooks and Supplementary Materials in the United States. instructional media + magic, inc., Washington, DC, December 15, 2006.

9. Moore, Geoffrey A. (1998). Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customers. Collins [now HarperCollins] (paperback) revised edition August 20, 2002, HarperInformation [now HarperCollins]1991 (hardcover), New York.

10. Schools and Programs. (n.d.). Retrieved January 1, 2007 from http://www.georgetown.edu/home/learning.html

11. UIS Leadership Team: Charles Leonhardt, Principal Technologist, Office of Information Services. (n.d.) Retrieved January 1, 2007 from http://uis.georgetown.edu/deprtments/staff/leonhardt.html



Appendix A – Use of Blackboard Features by Course Size

1-5 6-15 16-35 36-60 61-100 101 or more

All

Documents 7.8% 38.5% 53.7% 62.9% 38.5% 86.3% 38.9% Files 7.4% 36.9% 52.3% 62.4% 36.9% 84.3% 37.8% Announcements

4.2% 19.8% 31.1% 41.2% 50.8% 62.7% 22.4%

Folders 4.3% 20.3% 27.1% 28.1% 20.3% 54.9% 19.9% Gradebook 1.4% 10.1% 15.1% 22.6% 10.1% 23.5% 11.2% External links

1.7% 11.0% 13.6% 9.0% 11.0% 19.6% 9.4%

Forum 2.6% 11.8% 10.6% 7.7% 11.8% 11.8% 8.6% Dropbox 1.3% 9.2% 11.1% 14.9% 9.2% 5.9% 8.4% Messages 1.9% 10.5% 10.3% 5.4% 10.5% 11.8% 7.8% Groups 0.1% 1.1% 3.5% 6.8% 1.1% 21.6% 2.6% Available assessments

0.6% 1.8% 2.8% 5.0% 1.8% 5.9% 2.2%

Deployed assessments

0.6% 1.5% 2.5% 4.5% 1.5% 5.9% 2.0%

Calendar 0.4% 1.1% 1.0% 0.9% 1.1% 3.9% 0.9% Available surveys

0.0% 0.6% 0.5% 2.3% 0.6% 3.9% 0.6%

Question pools

0.1% 0.1% 0.4% 2.3% 0.1% 2.0% 0.5%

Deployed surveys

0.0% 0.4% 0.3% 0.9% 0.4% 0.0% 0.3%

Internal links 0.0% 0.2% 0.3% 0.0% 0.2% 0.0% 0.2% Lessons 0.0% 0.0% 0.0% 0.9% 0.0% 2.0% 0.1% Assignments 0.0% 0.0% 0.1% 0.0% 0.0% 0.0% 0.0% Tasks 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%

Table A1 – Use of Blackboard Features by Course Enrollment Spring 2006



1-5 6-15 16-35 36-60 61-100 101 or more

Documents 85.7% 91.7% 91.3% 100.0% 100.0% 81.8% Announcements 28.6% 69.4% 72.1% 70.6% 70.6% 90.9% Folders 35.7% 43.1% 50.0% 70.6% 70.6% 54.5% Gradebook 28.6% 30.6% 37.5% 41.2% 41.2% 18.2% Groups 14.3% 8.3% 15.4% 5.9% 5.9% 18.2% External links 28.6% 37.5% 33.7% 35.3% 35.3% 63.6% Messages 21.4% 25.0% 22.1% 29.4% 29.4% 18.2% Forum 42.9% 31.9% 30.8% 29.4% 29.4% 9.1% Dropbox 0.0% 4.2% 9.6% 11.8% 11.8% 27.3% Deployed assessments 7.1% 19.4% 13.5% 5.9% 5.9% 18.2% Available assessments 7.1% 13.9% 15.4% 11.8% 11.8% 11.8% Calendar 7.1% 6.9% 7.7% 0.0% 0.0% 0.0% Surveys 0.0% 2.8% 5.8% 5.9% 5.9% 0.0% Question pools 0.0% 11.1% 4.8% 0.0% 0.0% 18.2% Lessons 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Deployed surveys 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Internal links 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Available assignments 0.0% 0.0% 1.0% 0.0% 0.0% 0.0% Tasks 0.0% 2.8% 0.0% 0.0% 0.0% 0.0%

Table A2 – Use of Blackboard Features by Course Enrollment Spring 2001



Appendix B – Content Planning Data

Blackboard Feature Courses Using Average Items per Using

Course

Maximum Items per Using

Course Any Course Content 100.0% 29.3 302 Documents 98.8% 25.6 302 Files 95.6% 26.7 324 Announcements 55.1% 5.6 50 Folders 50.0% 9.6 103 Gradebook 27.1% 11.8 85 External links 23.7% 15.1 257 Dropbox 20.3% 62.9 1,321 Message 17.6% 94.5 1,425 Staff information 20.9% 1.8 11 Forums 19.4% 5.3 28 Groups 6.2% 5.6 38 Available assessments

5.7% 11.7 135

Deployed assessments

5.0% 6.5 34

Calender entries 2.4% 27.1 502 Surveys 1.5% 1.0 1 Question pools 1.4% 25.9 153 Deployed surveys 0.7% 1.0 1 Internal links 0.4% 2.8 7 Created lessons 0.3% 7.7 11 Assignments 0.2% 4.5 8 Student home pages 0.0 1.0 1

Table B1 –Blackboard Features and items Used in Courses



Use of the Blackboard Learning System by Early Adopters

It appears those who first used the Blackboard Learning System—the “early adopters”—are more intense users than the “early majority.” A comparison of use in 2001 from Table A2 and in 2006 from Table A1 is shown in Table C1.

1-5 6-15 16-35 36-60 61-100 101 or more

Documents -77.9% -53.2% -37.6% -37.1% -19.0% 4.5% Announcements -24.4% -49.7% -41.0% -29.4% -19.8% -28.2% Folders -31.4% -22.8% -22.9% -42.5% -29.3% 0.4% Gradebook -27.1% -20.4% -22.4% -18.6% -6.3% 5.3% Groups -14.1% -7.3% -11.9% 0.9% 3.6% 3.4% External links -26.8% -26.5% -20.1% -26.2% -21.0% -44.0% Messages -19.5% -14.5% -11.8% -24.0% -19.9% -6.4% Forum -40.3% -20.1% -20.2% -21.7% -21.5% 2.7% Dropbox 1.3% 5.0% 1.5% 3.2% 10.5% -21.4% Deployed assessments -6.6% -17.9% -11.0% -1.4% 0.5% -12.3% Available assessments -6.6% -12.1% -12.5% -6.8% -5.4% -5.9% Calendar -6.7% -5.9% -6.7% 0.9% 1.6% 3.9% Available surveys 0.0% -2.2% -5.3% -3.6% -5.9% 3.9% Question pools 0.1% -11.0% -4.4% 2.3% 4.8% -16.2% Lessons 0.0% 0.0% 0.0% 0.9% 0.0% 2.0% Deployed surveys 0.0% 0.4% 0.3% 0.9% 0.0% 0.0% Internal links 0.0% 0.2% 0.3% 0.0% 0.0% 0.0% Assignments 0.0% 0.0% -0.8% 0.0% 0.0% 0.0% Tasks 0.0% -2.8% 0.0% 0.0% 0.0% 0.0%

Table C1 - Change in the Use Features from Spring 2001 to Spring 2006

As expected, early majority users tend to use fewer features, especially in courses with the least enrollment. The use of features is expected to increase as faculty become more accustomed to using the learning system, as the learning system is better adapted to faculty users, or students use the learning content more frequently. Use, then, is expected to increase for the “early majority” as then gain experience and the environment changes because of learning technology.



Appendix D – Course Data Used in the Analysis

Percent

Count Data Element Description

3,413 DEPT Department: The four letter department code (truncated from Course_ID) 3,413 COURSE_ID Course_ID: The Course ID of the course as stored in Blackboard. 3,413 COURSE_TITLE Course_Tame: The title of the course as stored in Blackboard. 3,413 TERM Term: The Term (Spring, Summer, Fall and Year - Truncated from Course_ID). 3,413 AVAILABLE_IND Available_ind : Is the course available to students at the time of snapshot? 3,413 ALLOW_GUEST_IND Allow_guest_ind : Is the course available to guests? 0 CARTRIDGE_PK1 Cartridge_ind : Was a course cartridge used in this course?

17.5%

596 ANNOUNCEMENT_COUNT

Announcement_Count : Number of announcements.

31.0%

1,057 CONTENT_COUNT Content_Count : The sum of the following content areas: AOI WYSIWYG, test link, survey link, assignments, courselinks, documents, external links, files, lessons

0.0% 0 AOI_WYSIWYG_COUNT AOI_wysiwyg_Count : Number of AOI WYSIWYG documents. 1.6% 53 TEST_LINK_COUNT Test_link_Count: Number of links to assessments. This is essentially a deployed

assessment. 0.2% 7 SURVEY_LINK_COUNT Survey_link_Count : Number of links to surveys. This is essentially a deployed survey. 0.1% 2 ASSIGNMENT_COUNT Assignment_Count : Number of links to assignments. 0.1% 4 COURSELINK_COUNT Courselink_Count : Number of internal course links to content items. 30.6

% 1,044 DOCUMENT_COUNT Document_Count : Number of items added using the Add Item option of adding content.

7.4% 251 EXTERNAL_LINK_COUNT

External_link_Count : Number of External Links (URL's) added to a course.

0.0% 0 FILE_COUNT File_Count : Deprecated, number of files uploaded to a course. Not in use any longer, and current course with a number > 0 are probably copies from older semester courses.

15.6%

532 FOLDER_COUNT Folder_Count : Number of folders created in a course.

29.6%

1,011 COURSE_CONTENTS_FILES_COUNT

Course_content_files_Count: Number of data files attached to documents or other course structures



0.1% 3 LESSON_COUNT Lesson_Count : Number of Lessons created in course. Lesson is a content type available from the pulldown menu when adding content to a course.

1.8% 60 ASSESSMENT_COUNT Assessment_Count : Number of Assessments/Tests in the Assessment Manager. Not necessarily in use.

6.7% 227 FORUM_COUNT Forum_Count : Number of discussion board forums created. 6.0% 205 MSG_COUNT Msg_Count : Total number of messages across all discussion board forums. 8.7% 297 GRADEBOOK_ITEM_COU

NT Gradebook_item_Count : Number of columns in the gradebook. Includes the Total and "Weighted Total" column, so only include courses with gradebook_item_Count > 2 when considering the gradebook usage of a course.

6.5% 222 DROPBOX_COUNT Dropbox_Count : Number of files in the Digital Dropbox. 0.4% 15 POOL_COUNT Pool_Count : Number os question pools in the Pool Manager, not necessarily in use. 2.0% 69 GROUP_COUNT Group_Count : Number of groups created. 0.5% 16 SURVEY_COUNT Survey_Count : Number of surveys in the survey manager, not necessarilly in use. 7.4% 251 URL_COUNT URL_Count : The same as the number of external links external_link_Count. 0.0% 0 TASK_COUNT Task_Count : Number of Tasks that have been created. 0.8% 26 CALENDAR_COUNT Calendar_Count : Number of Calendar entries for this course. 6.5% 222 STAFF_COUNT Staff_Count : Number of Staff Information pages that have been created. 0.1% 2 HOMEPAGE_COUNT Homepage_Count : Number of students homepages that have been created. 76.2

% 2,601 INSTRUCTORS Instructors : Number users with instructor role.

77.4%

2,640 STUDENTS Students : Number users with student role.

1.1% 36 COURSE_BUILDERS Course_Builders : Number users with course builder role. 6.8% 233 TAS TAs : Number users with TA role. 0.2% 6 GRADERS Graders : Number users with grader role. 0.4% 12 GUESTS Guests : Number users with guest role.