vsa 2011 exhibit files v.5

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

1

Knowledge, Identity, and Networks in the Informal Learning Community

Wendy Pollock, Wendy Pollock LLCCarey Tisdal, Tisdal Consulting

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

2

Improving Practice?

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

3

Three Big Ideas

• Professional knowledge and identity are defined by and shared through complex networks of people and media

• These networks, people, and media are constantly evolving and changing

• Ways of understanding networks, people, and media are evolving and changing

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

4

ExhibitFiles

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

5

ExhibitFiles

Emergence in a Changing Media

Landscape

Evaluating Social Media Designed

to Improve Practice

Lessons Learned

ExhibitFilesEmergence in a changing

media landscape

wendy.pollock@gmail.com

ExhibitFiles Development TeamWendy PollockKathy McLean Ideum, led by Jim Spadaccini Administrative HomeAssociation of Science-Technology CentersAdministrative support: Wendy Hancock  FundingNational Science FoundationGrant No. ESI-054026

Advisors and Core ContributorsDave BaileyAndrea Bandelli Joanna FisherGretchen JenningsJeff KennedyKathy KrafftTom NielsenPaul OrselliClaire Pillsbury Jenny Sayre RambergChristine ReichBeth Redmond-JonesJustine RobertsBeverly SerrellEric SiegelJason Jay StevensJoan WaltermireMartin WeissKatherine WhitneyNAME  Board and membersThe 1,950+ members of ExhibitFiles

  "Learning is least useful when it is private and hidden; it is most powerful when it becomes public and communal."  Lee S. Shulman, 1999

"One of the things that might be done is some kind of collecting activity to retrieve memories of things that were wonderful." Mike Spock

1980s 

WorkshopsConferences

Print publications

DesktopsFiles, folders

1990s 

Pages, links, webs

Early 2000s 

Web 2.0Forums

User-contributed content

 Community

"Communities of practice are formed by people who engage in a process of collective learning in a shared domain of human endeavor."  - Etienne Wenger

Collection

"Learning within [communities of interest] ... requires externalizations in the form of boundary objects." - Gerhard Fischer

Community + Collection = 

"ExhibitFiles is an online community of exhibit practitioners building a shared collection of exhibition

records and reviews."

 Simplicity

LanguageCulture

• one niche in learning ecology • flows, not walls

 • emergent design 

 • requires ongoing commitment of resources

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

23

Evaluating Social Media Designed to Improved Professional Practice

Remedial Evaluation

Summative Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

24

Design

Questions• How did the design team assume

the site would work to produce impacts?

• To what extent and in what ways did users value and use the functions provided by the website?

• To what extent did users perceive themselves as members of a community

• To what extent and in what ways were strategies successful in accomplishing the project impacts?

Methodology & Methods• Naturalistic Inquiry• Methods

– Program theory Weiss (1998)– Analysis of the website

database (N = 1357)– Comparison to an association

membership database (N = 830)– Online survey of registered

members (N = 286– In-depth interviews with the

project team and registered members (N = 18).

Remedial Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

25

Program TheoryRemedial Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

26

Target Audience

Who used ExhibitFiles?• 59.1% exhibition

professionals• 40.9% roles connected to

exhibition development (e.g. administrators, educators, evaluators)

Where did ExhibitFiles members work?

• 37.7% work in science-technology centers/museums, natural history museums, zoos or aquaria, botanical gardens/arboreta, and nature or interpretive centers

• 63.3% work in other settings including other museum types (art, history, children’s) and design firms.

Remedial Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

27

Frequency of UseRemedial Evaluation

Figure 22. How often do you visit ExhibitFiles? (N = 264)

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

28

Visit PromptsRemedial Evaluation

Figure 23. What prompts you to visit ExhibitFiles? (N = 245)

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

29

Social Connections

Who on ExhibitFiles has influenced your work?

Remedial Evaluation

Already know them, however I did connect with [name] due to fascinating case history in ExhibitFiles

I have not contacted them via ExhibitFiles - I knew them before.

I knew all of these individuals before the creation of ExhibitFiles.

[Name] is my manager and mentorWorked together

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

30

Level of Participation

ExhibitFiles (online survey)• 10.0% contributed at least

one case study• 10.0% contributed at least

one review• 90.0% neither.• Low levels of favoriting and

comment• Site somewhat difficult to

search

Neilson’s 90-9- 1 rule (2006)• 90% of users are lurkers

(i.e., read or observe)• 9% of users contribute from

time to time, but other priorities dominate their time.

• 1% of users participate a lot and account for most contributions

Remedial Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

31

Influence on WorkRemedial Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

32

Reasons for Contributing Remedial Evaluation

Figure 30. Reasons for publishing a review (N = 23)

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

33

Remedial Findings

• How could ExhibitFiles be more useful and relevant? – Email reminders to return to site– Improved search functions on site– Connect search with other sites– Greater range of contributors– More non-science contributions– More contributions from outside the US– More critical discussion/edgier content– More discussion of what didn’t work

Remedial Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

34

Revised TheoryRemedial Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

35

Focusing

• What information was most critically needed? • By whom? (stakeholders)• What is the future of ExhibitFiles?

(sustainability)• What evaluation approach will help answer

these questions? • Do we need a revised theory?

Summative Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

36

Theory -- EcosystemSummative Evaluation

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

37

Levels of EcosystemSummative Evaluation

Individual

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

38

Levels of EcosystemSummative Evaluation

Groups

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

39

Levels of EcosystemSummative Evaluation

Fields

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

40

EcosystemSummative Evaluation

Fields

More than one Fields?

Lessons Learned• not just a site - part of learning ecology - multiple

flows/stream  • breaks down traditional walls (e.g., between

novice and expert,)  • emergent design, never finished, must continue to

evolve  • caring for the emergent

collection/resource/community  requires ongoing commitment of resources