1

Collaborative Technology Adoption: A Case Study of

Success and Challenges

Steven E. PoltrockMathematics & Computing Technology

Phantom WorksThe Boeing Company

Presentation to the 2003 International Symposium onCollaborative Technologies and Systems (CTS 2003)

January 22, 2003

2

Agenda

Collaboration technology in a social context Case study: A data conferencing system Adoption rate and uses of data conferencing Studies of 5 early adoption teams A model of innovation diffusion (technology adoption) A survey of 194 data conference users Results and conclusion

3

Complex Systems Are Built through Teamwork

4

Teamwork Across Major Boeing Sites Requires Collaboration Technology

5

Collaboration Technology Supports Diverse Collaboration Activities

Communication

Information sharing

Coordination

Real time Asynchronous

• AV conferencing• Telephone• Chat• Broadcast video

• Whiteboards• Application sharing• Meeting facilitation• Collaborative virtual environments (CVEs)

• Floor control• Session management

• E-mail• Voice mail• FAX

• Document management• Threaded discussions• Knowledge repositories• Team workspaces• Program repositories

• Workflow management• Case tools• Project management• Calendar & scheduling

6

Collaboration Occurs in, and Is Shaped by, a Social Context

Team:Team: A group organized to work togetherA group organized to work together

Organization: Organization: A number of persons or groups having A number of persons or groups having specific responsibilities and united for a specific responsibilities and united for a specific purposespecific purpose

Community: Community: A group or class having common interestsA group or class having common interests

Program / Project: Program / Project: An undertaking requiring concerted An undertaking requiring concerted efforteffort

7

Collaboration Technology Supports Diverse Collaboration Activities

Communication

Information sharing

Coordination

Real time Asynchronous

• AV conferencing• Telephone• Chat• Broadcast video

• Whiteboards• Application sharing• Meeting facilitation• Collaborative virtual environments (CVEs)

• Floor control• Session management

• E-mail• Voice mail• FAX

• Document management• Threaded discussions• Knowledge repositories• Team workspaces• Program repositories

• Workflow management• Case tools• Project management• Calendar & scheduling

TeamsTeams

8

Collaboration Technology Supports Diverse Collaboration Activities

Communication

Information sharing

Coordination

Real time Asynchronous

• AV conferencing• Telephone• Chat• Broadcast video

• Whiteboards• Application sharing• Meeting facilitation• Collaborative virtual environments (CVEs)

• Floor control• Session management

• E-mail• Voice mail• FAX

• Document management• Threaded discussions• Knowledge repositories• Team workspaces• Program repositories

• Workflow management• Case tools• Project management• Calendar & scheduling

TeamsTeams OrganizationsOrganizations

9

Collaboration Technology Supports Diverse Collaboration Activities

Communication

Information sharing

Coordination

Real time Asynchronous

• AV conferencing• Telephone• Chat• Broadcast video

• Whiteboards• Application sharing• Meeting facilitation• Collaborative virtual environments (CVEs)

• Floor control• Session management

• E-mail• Voice mail• FAX

• Document management• Threaded discussions• Knowledge repositories• Team workspaces• Program repositories

• Workflow management• Case tools• Project management• Calendar & scheduling

Teams Organizations Communities

10

The Challenge of Collaboration Technology Adoption

Technology adoption is a slow, phased process

Physical distance is an obstacle to adoption – People learn from neighbors– Organizational mandates have limited

range Collaboration technologies require universal

adoption but have inherent limiting properties– Tragedy of the Commons– Critical mass– Difficulty of learning infrequent features– Visibility of performance

Early adopters

Late adopters

Time

11

Data Conferencing Technology: A Case Study

Developed and piloted Boeing’s data conferencing infrastructure in 1997

Transitioned the technology to a support organization Studied its use by Boeing teams and provided feedback Evaluated new technologies as they emerged

12

Data Conferencing Infrastructure

CorporateIntranet

PublicInternet

Instructions& Downloads

DirectoryServer

ConferenceServer

SGI Meeting

MS NetMeetingSun ForumHP Visualize

13

Conferences Hosted on Workstations

CorporateIntranet

PublicInternet

Instructions& Downloads

DirectoryServer

ConferenceServer

SGI Meeting

MS NetMeetingSun ForumHP Visualize

14

Conferences Hosted on a Server

CorporateIntranet

PublicInternet

Instructions& Downloads

DirectoryServer

ConferenceServer

SGI Meeting

MS NetMeetingSun ForumHP Visualize

Growth of Data Conference Use

16

Examples of Data Conferencing Uses at Boeing

Overcoming distance

– Add distant participants to a face-to-face meeting

– Virtual meetings with no face-to-face participants

In teams and small groups

– Team meetings to review progress

– Document collaboration

– Telecommuting from home or other sites

In organizations

– Training

– Computer assistance

– Presentations In communities of practice

– Distributed meetings

– Presentations

– Demonstrations

17

Observational and Experimental Studies of Early Adopter NetMeeting Usage

In the first six months of deployment we observed 4 teams that used similar approaches

– NetMeeting to show and interact with information

– Teleconferencing for voice

– 2 meeting configurations Face-to-face meetings with remote participants at

desktops Virtual meetings with no meeting room

For a 5th team, we manipulated their physical and technology configurations

18

General Benefits and Problems

Application-sharing was the most used feature and clearly added value– Audio-conferences alone were unsatisfactory – Access to last-minute changes, e.g. microphotograph– Shared reference improved efficiency of distributed teams

Problems coordinating interactions– Meetings started late, technology use limited– Interaction hardest for remote members– Difficulty in knowing who was present, identifying remote speakers,

coordinating participation “Are they pausing for a comma, or a period?”

– Could not make sense of others’ on-line behaviors– Face-to-face meetings were markedly different: side-discussion, story-telling,

spontaneity “I hear the voice, but there is a vacancy for the whole human being.”

New roles emerged in successful team: meeting and technology facilitators

19

Use of Technology Was Minimal

Some did not acquire NetMeeting in this early phase of deployment– Heterogeneous computing and support environments, little or no help,

and resistance to learning new technologies– One heard that a lot of time would be wasted getting synchronized

Mostly presentation mode – One person shared, others just observed– No instances of using collaboration, whiteboard, or file transfer– Chat used in one group

Even this limited use was difficult– Frequent problems connecting to a session– Cannot find the shared window under the NetMeeting interface– When email or calendars were shared, others were surprised that this

was possible

20

Technology Enabled Greater Participation from More Locations

Barriers are diminished for some

– D at main site: Does anyone in this room understand what he’s saying?

– Remote site: I do.

– D: You’re not in this room.

– Remote site: I’m in the global room. Face-to-face or virtual is a choice for some

– Evolution: face-to-face mixed virtual

– Scientific team collected data

21

0

5

10

15

20

25

30

35

402.

8.19

95

2.10

.199

5

2.12

.199

5

2.2.

1996

2.4.

1996

2.6.

1996

2.8.

1996

2.10

.199

6

2.12

.199

6

2.2.

1997

2.4.

1997

2.6.

1997

2.8.

1997

2.10

.199

7

2.12

.199

7

2.2.

1998

2.4.

1998

2.6.

1998

2.8.

1998

2.10

.199

8

Meeting Date

Nu

mb

er

of

Att

en

de

es

Audio

conferencing

NMFace-to-face meeting

Scientific Team Attendance

22

0

1

2

3

4

5

6

7

8

9

10

2.8.

1995

2.10

.199

5

2.12

.199

5

2.2.

1996

2.4.

1996

2.6.

1996

2.8.

1996

2.10

.199

6

2.12

.199

6

2.2.

1997

2.4.

1997

2.6.

1997

2.8.

1997

2.10

.199

7

2.12

.199

7

2.2.

1998

2.4.

1998

2.6.

1998

2.8.

1998

2.10

.199

8

Nu

mb

er

of

Sit

es

Meeting Date

NMAudio conferencing

Face-to-face meeting

Number of Scientific Team Sites

23

Staff Meeting Experiment

Weekly meeting held for years 18-question post-meeting survey covering meeting

productivity, process, technology use Meetings surveyed: 4 in room, 2 split between CRs, one

with manager and OA in one room,others in CR Experimenters took notes on meeting process

24

Satisfaction: FTF versus Distributed Meetings

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

CR CR-CR CR-CR CR CR-Office CR CR-Office CR

25

Given these Challenges, How Did Data Conferencing Achieve Such Widespread Adoption?

The typical drivers for adoption of innovations are:

– Management mandate

– Collocated colleagues

– Opinion leaders or change agents What were the drivers at Boeing?

– Management used it but few mandated its use

– The IT organization provided it but did not advertise or advocate it

26

Rogers’ Model of Innovation Diffusion

Key properties of innovations

– Compatibility: is it needed and does it fit existing work and systems?

– Observability: how can people “observe” across distance?

– Relative advantage: i.e. over other technologies in use? Five stages of adoption

– knowledge of the innovation

– persuasion to use it

– decision to adopt it

– implementation of the innovation

– confirmation that adoption was appropriate Distance can be a barrier

27

Barriers to Early Adoption

Barriers to decision, persuasion, and implementation

– Lack of support from managers

– Different platforms

– Team participation was part-time

– No local technical support

– Lack of peer pressure

– Discouragement at local site

28

One Site Required an Exception Process

Management’s concerns– “You might say, what’s the harm in using it? On the surface, there’s no harm, but

the really damning thing of what happens is that people then begin to alter or implement new processes and procedures surrounding that capability on that product.”

– “Somewhere, someone, would need to place a value on collaboration. What’s the payback? What are you gonna get out of it to offset the costs?”

Some thought that management could control its use– “Without approval, you will not be able to enter any meetings even if you have

NetMeeting loaded.” A champion supported its adoption while accepting management’s limits

– “I particularly find it rewarding to hear back from people when they are using the more powerful tools of NetMeeting to collaborate on documents together as I think this product can change the way we work together as a company.”

Critical mass became a compelling force– “I was told yesterday that I was the only one who did not have NetMeeting.”

29

Surveyed 194 Data Conference Users in October 2000 – February 2001

0

5

10

15

20

25

30

< 3 3 - 6 6 - 12 12 - 18 18 - 24 over 24

Months since adoption

Per

cen

tage

30

Frequency and Duration of Usage Are Independent

Infrequent users

(less than once a week)

Frequent users

(2/week to daily)

Total

Late majority

(6 mos or less)

11 9 20

Early majority

(7-18 months)

13 16 29

Early adopter

(19 mos or more)

14 17 31

Total 38 42 80

31

They Learned about It from Colleagues

0

5

10

15

20

25

30

35

32

They Told Many Others About Data Conferencing

0

2

4

6

8

10

12

14

Number of Introductions

Fre

qu

en

cy o

f R

esp

on

se

s

“Yeah, just as a course of doing business. You ask them if they have NetMeeting. If they’d say no, you’d say, ‘well you might want to get it loaded on your PC so we can use it.’”

33

Reasons for Introduction

0

10

20

30

40

50

60

Dist ributedmeet ing

Smallgroup

Staffmeet ing

T raining Other

34

Remarkably Little Use of Most Features

0

10

20

30

40

50

60

70

80

90

100

Per

ce

nta

ge

35

Great Diversity in Conference Configuration

0

10

20

3040

50

60

70

80

Servers Call Host Non-Boeing

Per

cen

tag

e

36

Rogers’ Model of Innovation Diffusion Again

Key properties of innovations

– Compatibility: is it needed and does it fit existing work and systems?

– Observability: how can people “observe” across distance?

– Relative advantage: i.e. over other technologies in use? Five stages of adoption

– knowledge of the innovation

– persuasion to use it

– decision to adopt it

– implementation of the innovation

– confirmation that adoption was appropriate Distance can be a barrier

37

Reviewing Rogers’ Key Properties

Compatibility– Well integrated in the IT infrastructure– Increasing geographic diversity created the need

Observability– The results were immediately observable– Less observable features were rarely used

Relative advantage– Telephone alone was insufficient– Other approaches were too expensive (video) or staying

synchronized was too difficult

38

Reviewing Rogers’ Stages

Knowledge – Learned about it from local and distant collaborators

Persuasion– Necessary for participation in meetings– Opinion leaders were often at other sites

Decision to adopt– Facilitated by ready availability at no cost

Implementation– Some early adopters struggled

Confirmation– Frequent participation in distributed events– But new problems emerge such as need for security

39

Summary

A collaboration technology was widely adopted and heavily used

– There were many barriers to adoption

– There was no management mandate

– The usual adoption paths were not open

– Adoption was driven by working together across distance Adoption is a slow process even for a single person

– People most frequently learned about it while attending meetings

– Useful results are obtained by simply joining a meeting

– Most people used few features

– People misunderstood the system architecture and capabilities

40

Conclusions

Avoid collaboration technologies that require time or effort to learn

– Ensure that some value is achievable with minimal effort

– Few will invest the time required to learn an infrequently used technology

– Complex features will be used by few people Important features must be visible

– Users build mental models based on what they see Consider how one user will learn from others

– Some users learned how to join meetings by watching Provide compelling value