Collaborative informationsearching as learning inacademic group work

Dan Wu and Shaobo LiangSchool of Information Management, Wuhan University,

Wuhan, China, andWenting Yu

State Intellectual Property Office of the People’s Republic of China,Beijing, China

AbstractPurpose – The purpose of this paper is to explore users’ learning in the collaborative information searchprocess when they conduct an academic task as a group.Design/methodology/approach – This paper presents a longitudinal study for a three-month period on anactual task. The participants, who were undergraduate students, needed to write a research proposal in threemonths to apply for funding for a research project, including a three-hour experiment.Findings – The results show that undergraduates’ learning in the collaborative search process for academicgroup work included knowledge reconstruction, tuning, and assimilation. Their understanding of the topicconcepts improved through the process, and their attitudes became more optimistic. Besides, the learning in thecollaborative information search process also enhanced participants’ skills in communication, research, informationsearch, and collaboration. To improve learning outcomes, professional and appropriate academic resources arerequired, as well as effective division of labor, positive sharing behaviors, and use of collaborative systems.Practical implications – The future development of collaborative information search systems shouldfocus on the needs of academic research and support for elements such as instant communication andknowledge sharing.Originality/value – This paper contributes to research into searching as learning by understandingundergraduates’ collaborative search behavior for writing a proposal.Keywords Information-seeking behaviour, Collaborative learning, Collaborative information search,Collaborative queries, Group learning behaviour, Searching as learningPaper type Research paper

IntroductionLearning can include the act of acquiring new knowledge and values, modifying and reinforcingexisting behaviors and skills, and synthesizing information from different sources (Schacteret al., 2009, 2011, p. 264). It occurs in workplaces, daily life, and all other sectors of life. Paststudies have shown that web-based learning activities can greatly help students to becomeactive and self-directed learners, which often involves information searching tasks (Bilal, 2000;Hwang et al., 2008). Although searching used to be considered a process of interacting withinformation, such as information finding, selection, and use, it is, in fact, a natural learningbehavior (Yin et al., 2013). Individuals search to acquire new knowledge or to restructure existingknowledge structures to meet their information needs, as well as to support their learning.

Studies about learning while searching have long been active topics in information literacy,which draws connections between information search and learning skills. Morris and Teevan(2010) defined collaborative search as “the subset of social search where several users share

Aslib Journal of InformationManagementVol. 70 No. 1, 2018pp. 2-27© Emerald Publishing Limited2050-3806DOI 10.1108/AJIM-03-2017-0063

Received 9 March 2017Revised 17 May 2017Accepted 4 July 2017

The current issue and full text archive of this journal is available on Emerald Insight at:www.emeraldinsight.com/2050-3806.htm

This work is supported by National Natural Science Foundation of China (No. 71673204), and also anoutcome of the Wuhan University independent research project (Humanities and Social Sciences)“Human-Computer Interaction and Collaboration Team” (Whu2016020) supported by “The FundamentalResearch Funds for the Central Universities.”

2

AJIM70,1

an information need, and actively work together to fulfill that need” (p. 2). It is common inlearning activities. Users will collaborate when completing a task for which they do nothave sufficient individual knowledge and will obtain help from others who are learningindividually. Thus, users can also learn from each other and make progress together.Hence, learning in the collaborative information search process is not simply a set of searchresults, it is also a process of learning together and learning from each other. Within thiscontext, we are curious about knowledge construction and its effect on people. Moreover,as technologies develop, collaborative search systems are needed to foster and enhance thelearning experience during the search process, rather than for information searching only.

In this study, we aim to gain a further understanding of multi-member groups’ learning whenthey work as a group to complete an academic task that lasts for an extended period. We wantto understand their behavior characteristics during learning in the collaborative informationsearch process, the changes in their sentiments, and the role that the collaborative search systemplayed in this process. Thus, we propose the following research questions:

RQ1. How do undergraduates gain knowledge in the collaborative information searchprocess?

RQ2. What are the results and implications of undergraduates’ learning in thecollaborative information search process?

We analyzed the time duration of different behavior in each group, the relations betweengroup members’ queries, the use of learning resources in different stages, as well as thelabor division to respond to the RQ1. Then we studied the results of academic taskaccomplishment, the members’ sentiment changes in in different stages, and their learninggains. Besides, we investigated the evaluation of and suggestions for the collaborativesearch system, to respond to the RQ2.

This study does not only focus on the personal search behavior, but also focus onbehaviors in a more complex collaborative environment, which is based on the naturallongitudinal study. We revealed the learning phenomena in the collaborative informationseeking (CIS) process and enriched people’s understanding of CIS behavior.

The paper is organized as follows. First, we present a review of related works beforedescribing our research design and methods. Next, we report the findings and analysis of ourresearch results, and finally, our conclusions and suggestions for future works are given.

Related workComputer-supported collaborative learningCollaborative learning is a multi-disciplinary field that is relevant to areas such aseducation, psychology, sociology, computer science, and information science andtechnology. Dillenbourg (1999) argued that research on collaborative learning shouldfocus on defining four aspects: situation, interactions, process, and effects. Stahl et al.(2006) considered that collaborative learning consists of individuals in groups as well asactivities such as communication and sharing. Recent studies have tended to considergroup task performance as a part of the evaluation of collaborative learning effects,rather than focusing only on individual task performance (Dillenbourg, 1999; Stahl et al.,2006). Reynolds (2016) focused on the relationship between process and learning resultsthrough the collaborative information search and knowledge-building practice ofAmerican middle-school students in school game design, and pointed out that studenttasks, collaborative information search forms, and query results are related. Researchersalso studied the benefits of collaborative systems or platforms for collaborative learning.Social interaction is recognized as a factor in collaborative knowledge gains; Aalst (2009)found that social interactions are one of the leading factors in group tasks.

3

Collaborativeinformationsearching

From the view of information science, studies on collaborative learning cover aspects ofcollaborative search behavior and systems in collaborative learning settings. Someresearchers have explained collaborative learning in the context of information seeking. Forinstance, Kim and Lee (2014) studied the complex dynamics of knowledge construction andinformation seeking in three different stages of a collaborative research project, and theirresults showed that students’ knowledge grew as the project progressed. The study alsofound that students who collaborated in research were confident when they formed a sharedunderstanding of the topic at the beginning of the project. When conducting theirinformation-seeking activities individually, students would feel more stressed. Sormunenet al. (2013) explored cooperative and collaborative learning models, together withinformation seeking and use in work tasks. By interviewing students following their groupprojects, they found that few student groups worked closely together. Some started in asimilar way, but then divided the work into independent subtasks, working more loosely.This could waste much of the groups’ potential for collaborative learning.

Searching as learningVarious studies have been conducted, covering several dimensions. Some have explored therole of information searching in learning, and factors and concepts related to searching aslearning (Hansen and Rieh, 2016; Vakkari, 2001; Wildemuth, 2004). Zhang and Soergel(2016) studied the conceptual changes in knowledge structures that take place in differentforms, especially in the process of constructing significant meanings. Freund et al. (2016)believed that reading is the core component of the search, and studied the text on theunderstanding of the environment and the impact of learning.

The application of existing tools or models in searching as learning is another hot topic.For instance, individuals can integrate knowledge gains into the search process through conceptmaps and understand the relationship between human sense making, learning, and informationseeking (Stange et al., 2014). As a visualization tool, concept maps can be used to evaluatechanges in the user’s knowledge structure and also to evaluate the user’s search behavior (Egusaet al., 2014). Jansen et al. (2009) classified search tasks to verify whether there were some specialcharacteristics in the learning process. They found that web searchers relied more on their ownknowledge and information needs, while the searching was used mainly to check facts.

Other researchers have studied how the performance of searching as learning can beimproved and have put forward new models, systems, and other methods.A new cognitive model named “CoLiDeS+Pic” was used to simulate the web-navigationprocess for semantic information from pictures (Oostendorp and Aggarwal, 2014). Karanamet al. (2016) found “CoLiDeS+ ” could predict more paths with the correct hyperlinks and hashigher path-completion rates than “CoLiDeS,” providing accurate support during navigationand enhancing the performance of information searches. Bah and Carterette (2014) created asystem to support continuous searching: this first created typical pseudo-documents and thensorted the information from the retrieval results according to how closely it matched thetypical document. Saito and Miwa (2007) designed a learning environment that supportedlearners’ thinking activities while searching the information, and evaluated its educationaleffectiveness. This environment could visualize the learner’s search process and support twothinking activities, and could promote thinking by comparing the learner’s own processes tothose of others. As an important aspect of information seeking, relationships betweeninformation needs and types of learning during the health information-seeking process havealso been uncovered (Pian and Khoo, 2014).

From these studies, we can see that learning is embedded in the search process and in allaspects of individuals’ lives. However, existing studies have mainly focused on individualbehaviors, and more studies are needed that focus on searching as learning under morecomplex collaborative contexts.

4

AJIM70,1

Collaborative search in academic scenariosThere are a number of terminologies that are closely related to collaborative search, such as CIS(Shah, 2014), collaborative information behavior (CIB) (Reddy and Jansen, 2008), andcomputer-supported cooperative work (CSCW) (Rodden, 1991). When individuals collaborate,they often require information search in their collaborative project (Shah, 2012). Othercollaborative behaviors such as planning, sharing, editing, and synthesis are also involvedwhen undertaking academic projects. Thus, the process of writing research proposals can betreated as an intersection of collaborative searching and collaborative learning.

Several studies have been carried out on collaborative searching and behaviors in academicscenarios (Foster, 2006), or in academic groups (Leeder and Shah, 2016). Renugadevi et al. (2014)found that collaborative searches could improve the relevancy of search results. Blake and Pratt(2006) conducted an observational study of scientists to understand their CIBs when resolvingdifferent experimental results. They put forward five suggestions for information systems tobetter support users’ synthesis activities. Hyldegård (2006) investigated whether Kuhlthau’s ISPmodel, which is used to describe an individual’s information-seeking process, could be applied togroup members’ information behavior. In 2009, he conducted another experiment to gain furtherunderstandings of this issue. By analyzing library and information science (LIS) students’activities and cognitive and sentiment experiences during the task process for a projectassignment, he found that similarities and differences in behavior were between group andindividual in the ISP model, so the model could not fully comply with collaborativeproblem-solving process and the information-seeking behavior involved (Hyldegård, 2009).

In summary, there have been many studies on computer-supported collaborativelearning, searching as learning, and collaborative search in academic scenarios fromdifferent research perspectives. Researchers have mainly focused on individual behaviors insearching as learning, while we focused on more complex collaborative contexts. As forcollaborative learning, search behavior and its influences on learning effects should be givenmore attention. Furthermore, studies on collaborative search in academic scenarios shouldfocus on the promotion of searching on learning results.

MethodologyWork tasksThis study, conducted from January to March 2016, involved five groups of LISundergraduates who were working on their research proposals to apply for funding forresearch projects. In this competition for research projects, a committee evaluates whetheror not a group is successful based on the proposal. Participants in our study had the task ofwriting the research proposal, and we asked all groups to truthfully share with us theirwhole proposal-writing process, which lasted for three months. During this period, all thegroups went through the stages of formulating a research topic, searching and reviewingrelevant information, and finally, writing a research proposal. We did not impose anyrestriction on the research topics selected by the groups.

In contrast to previous studies, this study was carried out on a longitudinal basis in anactual academic setting, which enabled us to observe changes in participants’ sentiments,learning skills, collaboration skills, etc. In addition, it contributed to better observe andunderstand the behaviors of the same person(s) over time (Wildemuth, 2009). Furthermore,part of the learning results was based on the real results of the research project competition,which was more reasonable and reliable.

Task procedureThe work task for each group was divided into three stages. In the first stage (in January),group members could discuss and decide on the topic of their research proposals, and searchfor relevant information.

5

Collaborativeinformationsearching

The second stage included an experiment based on a collaborative search system,Coagmento, with each group initially writing a proposal within the prescribed time. Becausetime was limited, it was difficult for the groups to finish the task, so we did not require thecompletion of proposals. Each group had 30 minutes’ training to become familiar with thesystem, two hours to search and write the proposal, and 30 minutes to answer aquestionnaire. A group interview about participants’ collaboration in the procedure wasconducted after the experiment. Since the writing is started at this stage, resulting in moredata, we focused on the analysis of user behavior at this stage.

In the final stage, each group continued to revise and improve their proposals, and thensubmitted the final versions to committee. Following this, we conducted an interview witheach group based on their performance during the whole task.

Data collectionTo better uncover the collaborative behavioral patterns, we adopted various methods forcollecting data.

At the beginning of the work task, structured diaries were used to collect participants’background information, details of their habits when using learning resources, theirassessment of the importance of different learning resources, as well as their sentimentsregarding the task (see Appendix 1).

Following the search stage and the writing of the proposal in the three-hour experiment, weused questionnaires (see Appendix 2) to investigate participants’ sentiments, their cognition inrelation to communication and information sharing during the collaborative information searchprocess, and their evaluation of the collaborative search system’s functions.

During the three-hour experiment, all groups were asked to use Coagmento (Shah, 2010),a well-known collaborative information search system, to write the research proposal.Coagmento is a web browser plug-in based on user interaction rather than algorithmicmediation; users browse we pages and submit queries to public search engines. We chose itfor our experiment because it is open-source and supports the Chinese-languageenvironment. It can also provide functions to support information sharing, informationrating, communication, collaborative reporting, and resource management. Specifically, itcan be used to annotate, recommend, collect, and co-edit information, as well as to chat withothers. However, the chat function does not support the posting of images, so instead weused QQ (popular instant messaging (IM) software similar to MSN Messenger that thestudents use frequently in daily life). During the experiment, group members worked ontheir own computers, and could only interact with each other via Coagmento or QQ.Participants wrote their research proposals collaboratively on EditPad in Coagmento, andtheir editing behaviors were recorded. There was no break during this experiment.Participants’ searching behaviors (recorded using screen-recording software), system logs,and chat contents were also collected. The participants mainly completed their draft ofresearch proposals via Coagmento, so the data on Coagmento is more than other two stages.

After all groups had completed and handed in their final proposals, we examinedparticipants’ labor division mode, the problems and successes, and suggestions onimproving collaborative search systems, in order to understand their whole researchprocesses by group interviews (see Appendix 3). The research is conducted in the actualsettings, and there exists a committee to evaluate the proposals of groups. The evaluationswere based on the certain appraisal mechanism, so we did not conduct pre-test or post-test,adopting the final judgment of the committee.

ParticipantsWe recruited participants from groups involved in the undergraduate students’ research projectcompetition, a university-level extracurricular academic activity held at Wuhan University.

6

AJIM70,1

The organizing committee requires that each group must contain three to five people, and doesnot intervene in grouping, so the number of students in each group may not be identical.Five groups in this work were selected randomly from all the groups. In total there were20 participants, consisting of one three-member group, three four-member groups and onefive-member group. All of them were from the School of Information Management,Wuhan University, and from different majors, as shown in Table I.

The group members had known each other before our research, and they formed theirgroups spontaneously. Each group was paid ¥300 for their participation. We alsoinvestigated their habits in using learning resources (i.e. search engine, library anddatabase, PDF, web) (see Appendix 1.2).

Data analysis methodWe used quantitative and qualitative data analysis methods. We used the statisticalmethods, such as one-way ANOVA, to analyze the structured diaries and questionnairesthrough SPSS 20. The data follow the normal distribution, and we also carried out thehomogeneity test of variance.

We applied descriptive statistics for screen recordings and system logs. The interviewrecords were transcribed and analyzed thematically (Boyatzis, 1998) by ATLAS.ti 7, basedon the grounded theory.

A qualitative analysis was used to encode the interview records. We used open codingand axial coding to represent the coding of the first level and second level. We recruited twoteachers in the LIS domain to conduct this work, which helped to avoid results coding beingtoo subjective. They first classified and described the original interview records in the shortsequence of words, which were the open coding (the first level coding). Then they analyzedthe concept of these words and merged them to a same concept, which was axial coding (thesecond level coding). We tested the validity of the coding results. The consistency testmethod was used to compare the coding results of two teachers to validate the consistencyof coding. We used a formula, CA¼ (2× S)/(T1+T2), to test the consistency of codingresults. The S in formula represents the same coding results from two teachers, T1 and T2represents the total number of codes from two teachers, respectively. If the value of CA wasclose to 1, it indicated the higher consistency of coding results (Xu and Zhang, 2005).

Results analysisGroup members’ learning behaviorTime duration of different behaviors. In the experiment, this paper analyzes the periodicalcharacteristics of collaborative academic search behavior from the longitudinalangle to understand members’ various behavior in learning, i.e., the connection betweentime and behavior.

Our study analyzed the temporal characteristics of participants’ collaborative behaviorduring academic tasks from the longitudinal perspective. This was based on the data onclick-through from the screen-recording software. We used the timeline to show the processof collaborative academic searches, as shown in Figure 1. The ordinate represents the

Group no. Members Discipline Grade

1 3 Information management and information system Junior year2 4 Library science Sophomore year3 4 Information management and information system Sophomore year4 4 Library science Sophomore year5 5 Information management and information system Freshman year

Table I.Background

information ofparticipants

7

Collaborativeinformationsearching

members in each group, while the abscissa represents the time duration. The different colorsof the boxes in the figure represent the various types of behavior in the corresponding timeperiod, and the length of the boxes represents the time duration of the behavior.Unfortunately, a 30-minute delay occurred for Group 4 owing to an unexpected system fault,but members of Group 4 still finished the tasks in this period.

As can be seen from Figure 1, the participants exhibited the following behaviors in thecollaborative academic search process: searching on search engines, searching in databases,sorting out the literature, sorting out ideas, browsing the literature, discussion in groups,browsing websites, and writing. We grouped these academic behaviors into five types:discussion, searching, browsing, sorting out literature, and writing. Browsing took the mosttime, followed by searching and writing. In fact, these behaviors were not isolated, and oftenappeared alternately or simultaneously; for example, browsing websites was oftenaccompanied by searching. In the search process, participants often reviewed the results todetermine their usefulness. As for the writing process, participants would search forinformation if they encountered some problems. These behaviors also directly reflected theacquisition of new knowledge in the search process.

We chose 30 minutes as a time unit for counting the frequency of the five types ofacademic behavior mentioned in each time unit, as shown in Figure 2. The results showed thatdiscussion appeared most often in the early stage of the experiment, while searching andbrowsing appeared most often in the middle stage. In the final stage, participants focusedmainly on sorting out relevant literature and on writing. In other words, participants preferredto determine the direction first, and then to search for information. Finally, they summarizedthe existing knowledge and new knowledge to arrive at the final results.

Relations between group members’ queries. Every member of each group faced the sametask, so the number of identical or similar queries reflects whether the group’s division oflabor is reasonable and whether communication is timely to a certain degree. This couldreflect how they searched for information in the learning process.

In line with the existing research, we can divide information search into two types – lookupsearch and exploratory search – according to whether the search target and need is clear.

Searching on search engines Searching in database Sorting out the literatureBrowsing the websiteOther behaviors

Browsing the literatureWriting

Sorting out the ideasGroup discussions

G1

G2

G3

G4

G5

30 60 90 Time (minutes)Figure 1.Time duration ofdifferent types ofbehavior in theexperiment

8

AJIM70,1

The target of lookup search is generally known, which means that the core searchmechanism is to match the query and the document. In contrast, exploratory search is useroriented. There is no unique answer, and this reflects the correlation between informationsearching and learning.

We carried out a statistical analysis of query construction to understand the participants’search behavior, using data from the Coagmento log. The total number of queries for thefive groups was 57, 19, 29, 22, and 10, respectively. Apart from all members of Group 1 (23,23, 11) and a member of Group 3 (19), all participants submitted nine queries or fewer.Besides, there existed a great difference in their distribution between Groups 1 and 3 andothers, which reflected that they have weak impressions about their task.

Despite the small number of queries, there were some relationships between thesequeries. In order to reveal these relationships, we selected the identical queries and thesimilar queries submitted by the members in each group for statistical analysis; these weredivided into three categories: identical queries, semantically identical queries, and relatedqueries (Table II).

From the perspective of the search query, semantically identical queries included notonly queries with different words, but also those with different languages, such as“film search” and another semantically identical queries in Chinese. Related queries weremainly those using different search strategies to expand or narrow the search range, e.g.,“foreign creative space” and “American creative space.” Table II shows that there are moreidentical queries in Groups 1 and 4, which implies that members of these groups lackedcommunication about the search topic. If they have communicated fully, they would notsubmit the same queries, which means wasting time in the collaborative process. In contrast,the members of Group 5 would share important literature or content with one another.In addition, lookup search was mainly used to find the specific documents and webpages orthe library’s home pages, e.g., Wiki and Zhihu (a knowledge forum). The number ofexploratory searches was the highest, accounting for more than 50 percent.

35

30

25 21 20

16

11

3 3 2 1

2124

27

1922

31

3

1720

17

29

20

15

10

5

00-30min

Discussion Searching Browsing Sorting out materials Writing

30-60min 60-90min 90-120min

Figure 2.Time duration of five

types of academicbehavior in the

experiment

Group no. Total number Identical Semantically identical Related Lookup search Exploratory search

1 57 2 3 6 4 (7.02%) 53 (92.98%)2 19 0 2 2 8 (42.11%) 11 (57.89%)3 29 1 0 4 3 (10.34%) 26 (89.66%)4 22 3 2 3 7 (31.82%) 15 (68.18%)5 10 1 1 0 6 (60%) 4 (40%)

Table II.Number and type of

search queries

9

Collaborativeinformationsearching

Figure 3 shows the relationship between search queries and the topics of the tasks. Owing tothe small number of queries from Group 5, and no repeats, we did not analyze their queries.The red oval boxes in the figure indicate the topics of participants’ proposals, while the nodeoutside of the red oval boxes shows the queries submitted by these participants. Manyqueries were submitted sequentially, forming a tree-like structure. This reflects the fact thatthe groups’ knowledge building is a process of increasing the knowledge nodes, acquiringmore concepts and relationships around the core topics.

We can understand that members of a group would reformulate the queries according to thetask progress in order to obtain more relevant information. In addition, there exist connectionsbetween the queries around the search topics, involving various aspects of the task.

Use of learning resources. Use of the various resources searched was closely related to thesuccess of search tasks and the learning results. Searching for academic information couldinvolve more types of resource than searching for general information. We investigatedthe participants’ habits in relation to using learning resources through a structured diary inthe first stage (see Appendix 1.2). The structured diary used a five-point Likert scale, usingnumbers from 1 (low) to 5 (high) that show the learning resources usage frequency.The results are shown in Table III.

Web design about school sites

Film information retrieval

Organizational image and websiteCorporate identity

Group 3

iSchool university sites

iSchool unions

iSchoolinformation schools movement

iSchool movement

iSchool + identity

Research on iSchool Website

Evaluation Based on the Theory

Framewory of Enterprise Image

Recognition

Group 2

Folksonomy

Optimization of information

organization mode of “dianping.com”

based on Folksonomy

Lterature review of folksonomy

Related works of folksonomy

Application of folksonomy

Using of folksonomy

Bibsonomy

Semantically identical queries Related queries

Group 4

Research on Establishment and

Development of Hackerspaces in

Chinese Libraries

American famous hackerspaces

Hackerspaces abroad

Famous hackerspaces abroad

Hackerspaces

Techshop

Film information retrieval

Seek film information

Movie recommendation

Movie searching

Movie retrieval

Group 1

Film retrieval

Design of Movie Recommendation

Based on Duban User’s Tags and

Comments Data Mining

Movie Search

Movie search engines

Movie search behavior

Needs of movie search

Douban and film

Douban search engine

Douban Guess

Douban

Figure 3.The relationshipbetween searchqueries

Mean score of each groupOption no. Group 1 Group 2 Group 3 Group 4 Group 5 Mean score SD SE ANOVA value of p

1 4.33 3.50 3.75 4.00 2.00 3.40 1.142 0.255 0.008**2 3.67 2.25 1.75 2.00 2.00 2.25 1.070 0.239 0.1453 1.33 2.25 1.50 2.00 2.40 1.95 1.146 0.256 0.6834 4.33 3.00 3.75 2.75 2.80 3.25 1.070 0.239 0.2025 5.00 5.00 5.00 5.00 4.20 4.80 0.696 0.156 0.3066 4.67 4.50 5.00 4.75 4.40 4.65 0.587 0.131 0.6537 4.33 4.50 4.75 4.75 3.80 4.40 0.995 0.222 0.626Notes: Option 1: Chinese-language database; Option 2: English-language database; Option 3: Professionaltools and websites; Option 4: Academic search engine; Option 5: Search engine, Option 6: Document-sharingplatform; Option 7: Q&A community. **p⩽ 0.01

Table III.Usage habits inrelation to learningresources at thebeginning of the task

10

AJIM70,1

From Table III, we observe differences between the groups’ usage habits in relation tolearning resources. Overall, search engines, document-sharing platforms, and Q&Acommunity are commonly used in each group, with these having the higher mean scores.Professional website and English-language database utilization rates are lower, with themean value only around 2. This indicates that all groups are more inclined to use learningresources from the internet than professional academic resources provided by the library.However, there is significant difference between the groups in the usage of Chinese-language databases (Option 1, p¼ 0.008). The difference between Group 5 and other groupswas most significant, especially with Group 1. Members of Group 5 are all freshmen, andhave not yet mastered the use of professional academic resources. Meanwhile, the membersof Group 1 are juniors, and their understanding and use of these resources are better afterthree years of study and practice. In addition, Group 1’s mean scores on other learningresources except for Option 3 are higher, reflecting a stronger sense of information literacy.

We also investigated the participants’ assessments of the importance of different learningresources though structured diaries in the first stage (see Appendix 1.3). The results areshown in Table IV; a score of 1 means least important and 7 means most important.

All groups considered the use of academic search engines (Option 4), professional toolsand websites (Option 3), Chinese-language database (Option 1), and search engine(Option 5) to be very important. From previous studies (Rowlands et al., 2008), 89 percentof college students start information retrieval from a search engine, while only 2 percentstart from academic resources in the library. However, the information from the generalnetwork lacks the specialization and authority of the content for academic research. Thus,the participants in this study also considered that professional academic learningresources are more important.

In the second stage, we used Coagmento’s web statistics function to analyze the use ofvarious types of resources on the web, as shown in Table V. The results showed thatmembers used the search engines most, but there were few differences between searchengines and library and database. The most popular academic resources wereChinese-language databases, such as CNKI and Wanfang Data. According to the screenrecords, most participants were still accustomed to visiting the database from the homepage of the digital library.

Google Scholar was the second most popular search engine in the experiment.This indicated that members like to use specialized academic resources on the web whilesearching for academic information. This is because users can easily find more informationon research topics through academic resources in order to understand those topics.In contrast, the contents of general search engine result pages were broader and lessprofessional. Academic resources are characterized by authority and reliability, and canpromote the development of research work effectively.

Mean score of each groupOption no. Group 1 Group 2 Group 3 Group 4 Group 5 Mean score SD SE ANOVA value of p

1 4.67 4.25 2.50 5.00 4.40 4.15 2.059 0.460 0.5142 4.67 4.25 3.00 2.50 2.60 3.30 1.593 0.356 0.2193 5.00 5.75 3.50 2.75 4.20 4.20 1.704 0.381 0.0904 3.67 4.75 5.00 4.25 4.60 4.50 1.906 0.426 0.9305 3.33 3.00 5.50 5.75 3.20 4.15 2.368 0.530 0.2746 3.67 3.25 3.75 4.75 4.60 4.05 1.791 0.400 0.7557 3.00 2.75 4.75 3.00 4.40 3.65 2.519 0.563 0.745

Table IV.Participants’

assessments on theimportance of

different learningresources

11

Collaborativeinformationsearching

Although the rapid development of search engines has changed methods of findinginformation, users are increasingly dependent on search engines such as Google and Bing.However, these still cannot replace the position of the library, as the library has a largenumber and various types of well-integrated academic resources. Therefore, academicresources still play an important and irreplaceable role in research work. It is important tostrengthen information resources and improve information services in order to meet thegrowing needs of users for academic information.

Labor division. Division of labor is an important issue in collaborative search behavior.The appropriate division of labor can have a significant effect on the completion of the taskand help group member learn more effectively.

Depending on personal influence and ability, each member had a particular role in eachgroup. Analysis of chat histories in three-hour experiment by ATLAS.ti 7 (see Table VI)indicates that there was a leading role in each group during the division of labor. The groupleader was generally the member who understood the research topic deeply, or whounderstood it best within the group.

For example, from the chat history of QQ, we found that in most groups, the differentaspects of the division of labor were listed by one member, who assigned particular tasks toother members. Accordingly, when the division of labor was mentioned in the interview(see Appendix 3.1), a member of Group 1 said: “We would choose one group member tounderstand the work from a macro point of view, and then decided the various aspects ofthe division of labor.”

Web type Total

Search engineGoogle 90 203Baidu 44Google Scholar 69

Library and databaseChinese-language database 151 189English-language database 7Library 31

PDFDatabase 23 32Webpage 9

WebGeneral website 127 176Academic professional webpages 9Document community 40

Table V.Usage of differenttypes of learningresources in theexperiment

Group roles Vocabulary Word freq.

Leading roles Division of searching 2Guidance and help in subtasks 2Statement of labor division 1Confirm the division structure 2Remind others 1Responsible for macro tasks 1

Other roles Recommend others to divide labor 1Accept the subtasks 3

Table VI.Coding results on thegroup roles fromchat history

12

AJIM70,1

In summary, when determining the research topics and coordinating the division of labor,there was always a group leader controlling the overall situation. There were severalreasons for this. First, the academic work tasks were usually complex, with variousdimensions, such as research topics and frameworks. Thus, it is necessary to have amember to sort out the ideas from all the group members. It could also help the group tofind the direction of the task faster. Second, a number of individuals were involved in theacademic task, and this would generate various opinions, ideas, and problems.A coordinator or leader was needed to conduct the division of labor and maintain thestability of the group. In addition, the task of writing a proposal for research projectsgenerally takes a long time, from a few months to several years. In this process, someadjustments could occur in the different stages, and this also needs a leader to grasp themacro research process.

We also transcribed and analyzed the records of the interviews that followed thethree-hour experiment (see Appendix 3.1) using ATLAS.ti 7 for qualitative study. Theresults are shown in Table VII.

The content of the research project application was very complicated and involveddifferent aspects, so the division of labor (Group 1) by the contents is a reasonable approach,and one that could make up for any gaps in individual capabilities. Because the researchproject proposal had to follow a particular format, some groups divided labor according tothis structure. Group 3 divided the labor entirely by the structure of the proposal. Group 2began by dividing the labor by the structure of the proposal, and then divided the labor bysubject subdivision and followed the labor division instructions that were written before theexperiment. As a result, their experiment process went relatively smoothly. Initially,Group 4 also divided the labor by the structure of the proposal. However, a lack ofpreparation resulted in a chaotic division of labor, and this affected the progress of theexperiment. After repeated discussions among members, Group 4 changed their approachand divided the labor by contents. Group 5 took a different approach to the division of labor.They initially searched together without a specific division of labor and shared theinformation among the group members. As the task progressed, members gradually formedtheir own focus according to the different direction of searching. According to the chathistory from QQ, there were two stages to Group 5’s labor division, first by roles, and thenby contents.

Learning results in the collaborative search processIn this section, we first used the final results of competition to evaluate each group’s learningresults under the academic task. The sentiment changes, cognition on communication andinformation sharing, as well as the learning gains, were used to evaluate the members’learning in collaboration.

Labor division mode Vocabulary Word freq. Correlation coefficient

By contents Division by contents 3 0.75Division by structure of proposal 2 0.50Subject subdivision 1 0.25First macro, then refine 1 0.25Division by reference 1 0.25

By roles No specific labor division 1 1.00Work synchronously 1 1.00Form the key aspects automatically 2 2.00Search together 1 1.00

Table VII.Coding results on themode of labor division

13

Collaborativeinformationsearching

Results of academic task accomplishment. Our work was based on an actual academic taskrelating to an application for research projects. Because it was difficult to finish and improvethe proposals during the experiment, we did not set an evaluation index to evaluate theeffects of participants’ collaboration. The competition has a committee to evaluate whetheror not a group wins, based on their proposals. All five groups in this study handed over theirproposals to the school, for consideration alongside many other proposals. The committee inthe school, made up of five experts from different majors, evaluated whether or not a groupwould gain approval. The top 10 proposals would then be recommended to a university-level committee to decide whether or not a research project could be awarded nationalfunding, as well as whether or not a review was necessary. The winner could gain nationalfunding, or be awarded university-level funding.

This procedure guarantees the objectivity of the evaluation, so we also chose the finalapplication results as the evaluation of each group’s work. In Table VIII, the numbers 1(high) to 5 (low) indicate the evaluation rating of each research project. There weresignificant differences in the final application results.

Sentiment changes. As the work tasks lasted for a long period, the sentiments of groupmembers could have changed. These changes could reflect the group members’ learningeffects. Sentiment factors have an effect on the users’ search behavior and the final effects(Bilal, 2000; Wang et al., 2000; Chu et al., 2017), while there were fewer discussionsfocusing on the academic context.

The participants were asked to record their sentiments at the beginning of the work task.We divided sentiments into seven types (Rodden, 1991), namely three positive feelings(clarity, optimism, and relief ), one neutral feeling (uncertainty), and three negative feelings(doubt, disappointment, and intension). Participants were asked to mark with a numberfrom 1 (low) to 5 (high) to show the degree of these feelings.

We first investigated the sentiment states of three groups in the first stage of study usinga structured diary (see Appendix 1.4), and the results are shown in Table IX.The questionnaire results for Groups 1 and 2 were invalid for the obvious contradictions intheir structured diaries, so these are not shown.

Groups 3 and 4 experienced more positive feelings in the first stage, especially clarityand optimism, while the mean values for neutral and negative feelings are low. For Group 5,

Groupno. Final application results

Ratings oftasks

1 Failed application 52 Successful application for university-level funding 33 Successful application for national funding 14 Successful application for university-level funding 35 Participated in the review of the university-level committee, while receiving only

university-level funding2

Table VIII.Final applicationresults and rating oftheir applications

Feelings (mean)Group no. Clarity Optimism Relief Uncertainty Doubt Disappointment Anxiety

3 4.25 4.56 3.64 1.86 1.75 1.39 1.644 4.00 4.25 2.75 2.00 1.50 1.25 1.755 2.50 3.25 3.00 2.25 3.00 2.75 3.00

Table IX.Sentiment states ofthree groups at thebeginning of thework task

14

AJIM70,1

all mean values are relatively low, without significant differences, indicating that membersof this group showed sentiment stability in the preparation and exploratory stage.

After the writing process in the three-hour experiment, we studied the sentiment states ofthe five groups (Appendix 2.1). All participants were asked to choose a number to trulyrepresent their feelings, with 1 meaning totally disagree and 5 totally agree. The results forall the groups are shown in Table X.

This shows that at this stage, none of the groups had high levels for any of the types offeelings. However, the mean values for the positive feelings were higher than those forthe negative feelings, and the participants tended to be optimistic after the experiment.In the follow-up interviews (see Appendix 3.2), four groups expressed positive feelingstoward the task, with comments such as “OK,” “pretty good,” and “had fun during theprocess.” Only Group 4 thought their process “moved slowly and was a mess” and gave anegative summary of their work.

However, the groups did exhibit differences in sentiment cognition. Feelings such asuncertainty and disappointment showed highly significant differences, and feelings such asdoubt showed significant differences. The values for positive feelings of Groups 3 and 5were much higher than for their negative feelings. In contrast, the average values of allfeelings showed little differences between Groups 1 and 2. This indicates thatcommunication, preparation, and exploration in the early stage may have an impact oncollaborative work and sentiment.

Communication and information sharing. Communication is an important aspect of thecollaborative searching as learning process, which is also different from individuallearning. The collision of ideas during communication provided the opportunity formembers to learn from each other. There were five main methods of communication:e-mail, IM, call or SMS, video online, and face-to-face. We used a structured diary toinvestigate the importance of these five methods of communication in the first stage(see Appendix 1.5) using a five-point Likert scale ranging from “most important” (5) to“least important” (1). The results are shown in Table XI. IM (e.g. QQ) and face-to-facecommunication are both popular methods, as well as the most important methods, whilevideo online is the least important method.

We also included seven options in the questionnaire to study the effect and influence ofcommunication and information sharing in the process of collaborative search(see Appendix 2.2). The analytical results of the questionnaire are shown in Table XII.

As shown in Table XII, the mean scores for communication and information sharing foreach group were higher than 4.00, and there were significant differences with regard toOption 1 ( p¼ 0.023 o0.05) and Option 3 ( p¼ 0.047 o0.05). Some group memberscommunicated with each other often (Option 1), such as those in Groups 4 and 5, whileothers did so slightly less often, such as those in Groups 1 and 3.

Mean of each groupFeelings Group 1 Group 2 Group 3 Group 4 Group 5 Mean SD ANOVA value of p

Clarity 3.00 3.50 3.50 2.75 3.60 3.30 0.923 0.665Optimism 3.00 3.25 4.25 3.00 3.80 3.50 1.051 0.401Relief 2.67 3.25 4.00 2.25 3.40 3.15 1.089 0.191Uncertainty 4.00 3.50 2.50 3.00 1.80 2.85 0.988 0.003**Doubt 3.33 2.75 1.75 2.75 2.00 2.45 0.826 0.041*Disappointment 2.67 3.50 1.50 1.75 1.80 2.20 1.005 0.009**Anxiety 2.00 3.00 1.25 1.75 1.60 1.90 0.968 0.088Notes: *0.01opo0.05; **p⩽ 0.01

Table X.Sentiment states forthe five groups after

the three-hourexperiment

15

Collaborativeinformationsearching

The QQ chat history of Group 1 consisted mainly of discussion about the division of laborand the confirmation of research topics, with no communication about respective searchresults. The members focused on their own subtasks. As a result of Group 3’s thoroughpreparation before the experiment, its members’ focus was browsing information andwriting following the division of labor.

The highest mean score among all the groups was for Option 3 (4.55), indicating thatparticipants were willing to share information with others. The next highest mean scoreswere for Option 5 (4.40), Option 6 (4.45), and Option 7 (4.35), suggesting that all groups had apositive approach to sharing information in the collaborative search process. In particular,the mean score for Option 6 (4.45) indicated the positive improvement on knowledge fromthe information sharing. Group 5 had the highest mean scores for each individual option.From the system log analysis, we found that members of this group always shared usefulinformation via QQ during the experiment.

Through the above analysis, we suggest that the frequency of communication couldpromote stronger awareness of information sharing. Stronger awareness of informationsharing leads to higher evaluation of sharing. Thus, participants would be more willing tocommunicate with others and develop the habit of sharing, which is a virtuous circle.

Learning gains. Although the groups’ positive sentiments dominate in the collaborativelearning process, some problems led to the emergence of negative sentiments. We studiedthe problems members faced in the experiment and the outcomes after the experiment toinvestigate their learning gains. Through the interview (see Appendix 3.3), we investigatedthese problems and the interview results were coded using ATLAS.ti 7 (see Table XIII).

We divide the problems in the participants’ learning into three categories: lack ofresearch skills, problems in task planning, and problems in communication. As theparticipants are undergraduates, they could encounter difficulties in reading literature inEnglish and in finding the information in the process of learning during collaborative

Mean score of each groupOption no. Group 1 Group 2 Group 3 Group 4 Group 5 Mean score SD SE ANOVA Value of p

1 3.33 4.50 3.75 4.75 4.80 4.30 0.801 0.179 0.023*2 3.67 4.00 4.00 3.75 4.40 4.00 0.649 0.145 0.5533 4.00 4.75 4.25 5.00 4.60 4.55 0.510 0.114 0.047*4 3.67 4.50 3.50 4.75 4.60 4.25 0.786 0.176 0.0585 3.67 4.50 4.25 4.50 4.80 4.40 0.598 0.134 0.1076 4.00 4.50 4.25 4.50 4.80 4.45 0.510 0.114 0.2587 4.00 4.50 4.00 4.25 4.80 4.35 0.671 0.150 0.373Note: *0.01opo0.05

Table XII.Cognition ofcommunication andinformation sharing

Mean score of each group

Methods Group 1 Group 2 Group 3 Group 4 Group 5Meanscore SD SE ANOVA Value of p

E-mail 3.33 3.00 3.00 2.75 2.80 2.95 1.371 0.294 0.986IM 3.33 3.00 3.50 4.75 4.00 3.75 1.020 0.228 0.117Call/message 2.67 3.00 2.75 3.00 2.00 2.65 1.040 0.233 0.620Video online 1.33 2.00 2.00 1.00 2.40 1.80 1.473 0.329 0.697Face-to-face 4.33 4.00 3.75 3.50 3.80 3.85 1.226 0.274 0.943

Table XI.Importance ofdifferentcommunicationmethods at thebeginning of the task

16

AJIM70,1

information searches. In addition, their own research skills are limited, so some ideas aboutthe proposal could not be fulfilled.

With regard to the task planning, the main problem was that the direction was not clear,and there were also problems with the selection of topics and the lack of preparation.The unclear direction included the formation of the pre-structure and the ideas for writing.Problems in communication occurred among group members, as well as between the groupmembers and the teachers.

In the third stage, the activities of all five groups were few and simple. All groups spentmost of the time on finishing the proposal, indicating that the final stage is a process forcompletion, synthesis, and reflection. Thus, we investigated what participants learned in thecollaborative information search process, by transcribing and analyzing the interview records(see Appendix 3.4) following the experiment using ATLAS.ti 7, as show in Table XIV.

In the interview, the participants indicated that they gained various skills, such asresearch and teamwork skills. Members of Group 2 stated: “It was the first time I hadconducted a research project. I didn’t know how it would progress and how to prepare it atfirst. But, through this experience, I gained better knowledge of the topic and was clearabout how to conduct future studies.” Thus, participants developed research skills such asunderstanding the research process, data-collection methodologies, and writing.Participants also developed a deeper understanding of some research areas. Researchskills were cultivated during the collaborative search, as were research interests.Nevertheless, in the process of collaboration, participants found it was important to findsuitable group members. Members of Group 2 state: “It’s the most interesting part of theresearch to find like-minded friends and work together through thick and thin.” Formembers of Group 3, “during this collaboration, we formed collaborative work groups andthis collaboration continued in daily studies.” Teamwork skills, including communication

Problems Vocabulary Word freq. Correlation coefficient

Lack of research skills Lack of innovation ability 1 0.25Difficulties in reading English 1 0.25Difficulties in finding literature 4 1.00Failure to achieve expected results 1 0.25

Problems in task planning Unclear direction 3 0.60Selection of topics 1 0.20Lack of preparation 1 0.20

Problems in communication Inconsistent opinions 1 0.50Communication with teachers 1 0.50

Table XIII.Coding results

on problems duringthe whole task

Gains Vocabulary Word freq. Correlation coefficient

Research skills Understand the research area more 2 0.17Understand the research process 3 0.50Read English studies 1 0.17Data collection and writing 1 0.17

Teamwork skills Find a suitable colleague 6 0.86Focus on communication 1 0.14

Information literacy Improve information literacy 2 1.00Others Topic is important 1 1.00

Instructor is important 1 1.00

Table XIV.Coding results on

gains following thework task

17

Collaborativeinformationsearching

skills, were also enhanced. Members of Group 1 specifically mentioned that they gainedmore skills in information seeking, and that their information literacy was improved.

The learning gains in the collaborative search process were mainly research andteamwork skills. Compared with Table XIII, participants gain different skills after the task,and these skills could help them to address the difficulties they have had. For the academictask, many of the participants involved in the application for the research project graduallybegan to understand the process and learned the skills of data collection and writing. Theyalso improved their research skills and, in the process, became interested in academicresearch. In particular, they recognized the importance of collaboration with suitablecolleagues, something that could also strength their friendships.

Evaluation of and suggestions for the collaborative search systemEvaluation of the collaborative search system. Search systems play an important role insearching as learning (Chi et al., 2016). We therefore analyzed the performance of thecollaborative search system to study its impact on learning. Coagmento has bookmark,recommend, annotate, snip, and editor functions to support collaborative work. However, inour study, only the bookmark, snip and editor functions were used by the five groups.There were 15 bookmarks and three snippets, contributed by seven members and twomembers, respectively.

We collected groups’ evaluation of the Coagmento functions using a five-point Likertscale, with 1 indicating “not at all useful” and 5 indicating “very useful” (see Appendix 2.3).The functions used during the experiment were highly regarded by participants, especiallythe editor function (see Table XV). Participants thought that the collaborative searchsystem was helpful for collaborative search tasks.

However, the feedback in the interviews (see in Appendix 3.5) revealed a low rate ofutilization of Coagmento: only Group 5 continued to adopt it in their collaboration tocomplete the proposal after the three-hour experiment. This situation was partly caused byusers’ unfamiliarity with the collaborative tool and by their search habits. Nearly all groupsmentioned in the interview that the collaborative search system did not run smoothly duringthe experiment, and this led them to abandon it in their later study. They further identifiedthat IM software (QQ) was used as the replacement. Group 4 even stated: “The collaborativesearch system had nothing special for collaboration compared to the instant messagesoftware, and it could be totally replaced by QQ.”

Suggestions for the collaborative search system. In the interviews following theexperiment (see in Appendix 3.5), we inquired about the problems encountered duringthe experiment and asked for suggestions for a collaborative search system, using ATLAS.

Function Mean SD SE ANOVA value of p

Bookmark 3.80 1.322 0.296 0.030*Recommend 3.35 1.089 0.244 0.548Annotate 2.85 1.182 0.264 0.522Snip 3.45 1.276 0.285 0.365Editor 4.50 0.607 0.136 0.321Chat 3.50 1.638 0.366 0.130Notification 2.85 1.309 0.293 0.981History 3.55 1.432 0.320 0.089Notepad 3.40 1.273 0.285 0.944Note: *0.01opo0.05

Table XV.Evaluation ofCoagmento’s functions

18

AJIM70,1

ti 7 to code the records of the interviews (see Table XVI). We did not conduct associationanalysis owing to the dispersion of problems and suggestions from different groups.

It can be seen from these results that the main problem is that the collaborative searchsystem often crashed and dropped the line, which could delay the process of groups tasksand affect the efficiency of learning. There is a very noteworthy phenomenon, namely therelationship between the collaborative search system and IM software. Of the participants,12 stated that, “when I used instant messaging software, I forget to use the collaborativesearch systems” (or words to that effect). This relates to participants’ daily behavior.IM software is the most common communication tool used by users, and participants aremore familiar with it than they are with the collaborative search system. In addition, thecollaborative search system lacks support for cross-language usage. More importantly, IMsoftware and the collaborative search system have some duplication in terms of theirfunctions, e.g., sharing links or URLs. In this study, participants could use QQ to send linksor screenshots and share documents in the discussion group. Hence, they thought there wasnothing special about Coagmento, and that this system could be replaced by QQ.

The participants also gave some suggestions for improving the collaborative searchsystem in academic group work. Group 1 preferred functions that could share informationmore efficiently about others’work: “Shared information should be highlighted immediately.However, [in the current system] you need to click first if you want to check what othershave recommended. It’s not convenient.” They added: “The system should record andhighlight the important links that all members have clicked.” Group 5 said that they hopedthe collaborative search system could provide some information related to their project.

DiscussionLearning behavior in the process of collaborative searchPrior studies have shown that users would continue to increase their level of knowledge inthe collaborative information search process (Chi et al., 2016). Our research also confirmedthat collaborative search had positive effects on learning. Collaboration in writing researchproposals can be regarded as a way to stabilize knowledge structures. Analysis of ourthree-month results on search behaviors and sentiments shows that the participants’stabilization of knowledge structures is similar to that described in the work of Vakkari(2016), including the three stages: restructuring, tuning, and assimilation. Individuals first

Vocabulary Word freq.

ProblemsSystem crashes, drop line 5QQ can be replaced 4Prevent the use of other software 2Lack of reminder function 1Some functions are repeated 3Low rate of function utilization 2

SuggestionsAdd a backup function 1Mark the progress of personal tasks 1Add a reminder function 2Support copying and insertion of pictures 1Add a voice function 1Automatically provide information 1Parent child window 1Add a help function 1

Table XVI.Coding results about

problems andsuggestions oncollaborative

search system

19

Collaborativeinformationsearching

search a wide range of information to clarify their information needs and to make sense ofthe research topic together, so the exploratory search occurred most frequently (Table II).New knowledge is acquired during searching.

Most participants’ sentiments were positive at the beginning of the task, although theydid not feel relaxed (Table IX), as the work task seemed heavy. Based on a personal initialunderstanding of the topic, they shared the information they found and restructured theirqueries. This can be regarded as a way to restructure existing knowledge, and personalknowledge structures can also be influenced. Users often discuss and communicate witheach other to reach agreement on a general concept of the topic together. The scope of therelated concepts will be clearer in this tuning process. Preparation and exploration in theearly stage will contribute to subsequent collaboration and will make it more likely thatparticipants’ sentiments are positive. Finally, when writing a research proposal, participantsintegrated all the knowledge they have obtained. The final proposal was, in fact, producedby reflecting and assimilating the knowledge of all group members.

In summary, learning through collaborative information searching is a process ofrestructuring knowledge, including obtaining new knowledge and supplementing existingknowledge structures. As the task progresses, knowledge structures evolve throughrestructuring, tuning, and assimilation. Sentiment also changes from vague to clear, andshows a generally positive state (Table X).

The collaborative information search process could also enrich the gains of users. All theparticipants in this study had gained more skills by the end of the task (Table XIV). Ourresults indicate that communication is important in the collaborative search process, incontrast to an individual search (Table XI). Despite some communication problems andobstacles in the completion of work tasks, teamwork abilities will be strengthened andimproved through communication (Table XII). Communication and teamwork abilities arevital benefits of collaboration. The experiences of collaborative search also bring otherbenefits for learning, especially research skills. Students can learn how to conduct researchand, through collaboration, can compensate for any inadequacies in their individualabilities. Collaborative search asks students to explore and think for themselves, and will bean important supplement to traditional classroom education. Participants could directlydevelop more skills through collaborative information searching, which is part ofinformation literacy (Table XIV). Such searching helps users to develop more relevant skillsand achieve self-improvement. In our research, we studied learning in a collaborative searchprocess within a real academic setting, and found that collaboration is essential in writing aresearch proposal. This does not simply mean the addition of personal search results only.It offers a great opportunity to participate actively, and is something from which everyonecan benefit. This is not only because complex tasks can be completed more easily andeffectively when collaborating. It is also because collaboration provides a chance to learnfrom each other and progress together.

Ways to support learning through collaborative searchAs previously mentioned, there were differences between the results of the applications.We summarized the ways to improve the learning results, from the perspective of learningand collaboration to enhance the positive role of collaborative search in learning.

It is important to choose professional and appropriate resources to lay a solid foundation forlearning. Our participants preferred the Chinese-language database (Table V), ensuring theauthority and reliability of resources. Academic search engines such as Google Scholar alsoplayed an important role in enhancing the richness and ensuring the greater integration ofresources. With regard to understanding simple fact-based information, public search enginessuch as Google might be more convenient. As well, exploration is central in learning andunderstanding (White, 2016), so the exploratory searches are essential in collaborative learning.

20

AJIM70,1

Collaboration effects also have significant impacts on the learning process and the results.At the beginning of the tasks, a reasonable division of labor was necessary. An effectivedivision of labor at this stage could make up for any lack of personal knowledge through theprogress of the whole group. Each group would choose a leader to determine the division oflabor based on personal knowledge and skills (Table VI). A clear division of labor helpsmembers not only to understand the information better, but also to avoid confusion andimprove the efficiency of their work. In addition, positive communication and sharing areessential for saving time and supporting groups’ control of the progress of the work.

ImplicationsThrough our study, collaborative search systems can provide positive support for learningin the collaborative search process, and collaboration is important for information retrievalsystem (Twidale et al., 1997).

During this experiment, the collaborative search system had a high acceptance rate but alow level of usage. According to the participants’ suggestions, the collaborative searchsystem should provide instant communication to support the need for group discussion(Table XVI). The social aspects of information seeking should get more attention (Nicholsand Twidale, 2011), and the importance of communication and information sharing has beenshown, so communicating and sharing functions are necessary. This can promoteknowledge tuning and assimilation. The system should also pay more attention to datamining and providing academic information. In order to improve learning efficiency, it isimportant for a collaborative search system to help group members grasp the task’sprogress. Furthermore, concept maps should be applied to support group learning. Eachgroup member can mark out the relevant knowledge node and its relation to other nodes ona common screen, forming a knowledge network diagram. Such a concept map can alsomake the knowledge structure clearer, and the different parts of the map can also provide abasis for the division of labor.

This study can also help the library to expand the knowledge of information literacyeducation. Information literacy is an important quality in the information age, and it is alsoan important aspect of the library carrying out educational work. In general, course ofliterature retrieval is main method of information literacy education in library, whichemphasizes the skills of seeking information seeking and ability of evaluating information.The information-seeking process improves not only these two abilities, but also thecomprehensive ability to solve the problem, which is accumulation of research ability andmethod. Therefore, the information literacy education of the library can be more diversifiedand pay more attention to the cultivation of comprehensive abilities.

ConclusionsThis research explored undergraduates’ learning through a collaborative informationsearch process for an academic group task. We conducted a three-month study usingseveral methods to analyze participants’ learning in writing an actual research proposal.A three-hour experiment on a collaborative search system was designed to identify its rolein supporting learning. In analyzing the data, we found that collaborative search couldpromote learning experiences and have a positive effect on collaborative learning.New knowledge and skills, such as communication and research skills, were developedthrough the collaborative search session process and updating of existing knowledgestructures. In order to achieve a better learning effect, all the groups chose more professionalacademic information sources and also had effective collaboration as a support. We proposethat collaborative search systems should be required to pay more attention to thecharacteristics and information needs of education, in order to provide an environment inwhich individuals can experience more effective learning.

21

Collaborativeinformationsearching

The number of groups and participants in our research is relatively small and allparticipants are from one school. And there is a lack of variations in team sizes and specificnature of the task. Furthermore, besides the committee’s evaluation, we did not take properinstruments to measure learning. Thus, there is a need for future work to investigate moreusers in other knowledge domains using a larger sample. Learning metrics also need to beadded in the future. In addition, the different factors in group learning results, such aslearning experience, personal abilities, etc., should be addressed.

References

Aalst, J.V. (2009), “Distinguishing knowledge-sharing, knowledge-construction, and knowledge-creation discourses”, International Journal of Computer-Supported Collaborative Learning, Vol. 4No. 3, pp. 259-287.

Bah, A. and Carterette, B. (2014), “Using ‘model’ Pseudo-Documents to improve searching-as-learningand search over sessions”, Searching as Learning Workshop IIiX 2014, Regensburg, August,available at: www.diigubc.ca/IIIXSAL/Papers/RabiouCarterette.pdf

Bilal, D. (2000), “Children’s use of the Yahooligans! Web search engine: I. cognitive, physical, andaffective behaviors on fact-based search tasks”, Journal of the American Society for InformationScience & Technology, Vol. 51 No. 7, pp. 646-665.

Blake, C. and Pratt, W. (2006), “Collaborative information synthesis II: recommendations forinformation systems to support synthesis activities”, Journal of the American Society forInformation Science and Technology, Vol. 57 No. 14, pp. 1888-1895.

Boyatzis, R.E. (1998), Transforming Qualitative Information: Thematic Analysis and Code Development,Sage, Thousand Oaks, CA.

Chi, Y., Han, S., He, D. and Meng, R. (2016), “Exploring knowledge learning in collaborative informationseeking process”, SIGIR 2016 Workshop: Searching As Learning, Pisa.

Chu, S., Sonnenwald, D.H., Wu, M. and Chang, S.J.L. (2017), “Theory Development within theinformation sciences”, Bulletin of the ASIS&T, Washington, DC, available at: www.asist.org/files/bulletin/feb-17/Chu_EtAl_2016Meeting.pdf

Dillenbourg, P. (1999), “What do you mean by ‘collaborative learning’?”, in Dillenbourg, P. (Ed.),Collaborative-Learning: Cognitive and Computational Approaches, Elsevier, Oxford, pp. 1-19.

Egusa, Y., Takaku, M. and Saito, H. (2014), “How to evaluate searching as learning”, Searching asLearning Workshop IIiX 2014, Regensburg, August, available at: www.diigubc.ca/IIIXSAL/Papers/Egusaetal.pdf

Foster, J. (2006), “Collaborative information seeking and retrieval”, Annual Review of InformationScience and Technology, Vol. 40 No. 1, pp. 329-356.

Freund, L., Kopak, R. and O’Brien, H. (2016), “The effects of textual environment on readingcomprehension: implications for searching as learning”, Journal of Information Science, Vol. 42No. 1, pp. 79-93.

Hansen, P. and Rieh, S.Y. (2016), “Editorial: recent advances on searching as learning: an introductionto the special issue”, Journal of Information Science, Vol. 42 No. 1, pp. 3-6.

Hwang, G.J., Tsai, P.S., Tsai, C.C. and Tseng, J.C.R. (2008), “A novel approach for assisting teachers inanalyzing student web-searching behaviors”, Computers & Education, Vol. 51 No. 2, pp. 926-938.

Hyldegård, J. (2006), “Collaborative information behaviour – exploring Kuhlthau’s information searchprocess model in a group-based educational setting”, Information Processing & Management,Vol. 42 No. 1, pp. 276-298.

Hyldegård, J. (2009), “Beyond the search process – exploring group members’ information behavior incontext”, Information Processing & Management, Vol. 45 No. 1, pp. 142-158.

Jansen, B.J., Booth, D. and Smith, B. (2009), “Using the taxonomy of cognitive learning to model onlinesearching”, Information Processing & Management, Vol. 45 No. 6, pp. 643-663.

22

AJIM70,1

Karanam, S., van Oostendorp, H. and Fu, W.T. (2016), “Performance of computational cognitive modelsof web-navigation on real websites”, Journal of Information Science, Vol. 42 No. 1, pp. 94-113.

Kim, J. and Lee, J. (2014), “Knowledge construction and information seeking in collaborative learning”,Canadian Journal of Information and Library Science, Vol. 38 No. 1, pp. 1-21.

Leeder, C. and Shah, C. (2016), “Collaborative information seeking in student group projects”, AslibJournal of Information Management, Vol. 68 No. 5, pp. 526-544.

Morris, M.R. and Teevan, J. (2010), “Collaborative web search: Who, what, where, when, and why”, inMarchionini, G. (Ed.), Synthesis Lectures on Information Concepts, Retrieval, and Services,Morgan & Claypool Publishers, San Rafael, pp. 1-85.

Nichols, D.M. and Twidale, M.B. (2011), “Recommendation, collaboration and social search”,in Ruthven, I. and Kelly, D. (Eds), Interactive Information Retrieval, Facet Publishing, London,pp. 205-220, available at: https://researchcommons.waikato.ac.nz/bitstream/handle/10289/5659/recommendation-collaboration-social-search.pdf?sequence=1&isAllowed=y

Oostendorp, H.V. and Aggarwal, S. (2014), “Improving information seeking performance”, Searching asLearning Workshop IIiX 2014, Regensburg, August, available at: www.diigubc.ca/IIIXSAL/Papers/OostendorpAggarwal.pdf

Pian, W. and Khoo, C.S.G. (2014), “Health information learning within the context of health informationseeking on a discussion forum”, Searching as Learning Workshop IIiX 2014, Regensburg,August, available at: www.diigubc.ca/IIIXSAL/Papers/PianKhooAbst.pdf

Reddy, M.C. and Jansen, B.J. (2008), “A model for understanding collaborative information behavior incontext: a study of two healthcare groups”, Information Processing & Management, Vol. 41No. 1, pp. 256-273.

Renugadevi, S., Geetha, T.V., Gayathiri, R.L., Prathyusha, S. and Kaviya, T. (2014), “Collaborativesearch using an implicitly formed academic network”, Aslib Journal of InformationManagement, Vol. 66 No. 5, pp. 537-552.

Reynolds, R.B. (2016), “Relationships among tasks, collaborative inquiry processes, inquiry resolutions,and knowledge outcomes in adolescents during guided discovery-based game design in school”,Journal of Information Science, Vol. 42 No. 1, pp. 35-58.

Rodden, T. (1991), “A survey of CSCW systems”, Interacting with Computers, Vol. 3 No. 3, pp. 319-353.

Rowlands, I., Nicholas, D., Williams, P., Huntington, P., Fieldhouse, M., Gunter, B., Withey, R., Jamali, H.R.,Dobrowolski, T. and Tenopir, C. (2008), “The Google generation: the information behaviour of theresearcher of the future”, Aslib Proceedings, Vol. 60 No. 4, pp. 290-310.

Saito, H. and Miwa, K. (2007), “Construction of a learning environment supporting learners’ reflection: acase of information seeking on the Web”, Computers & Education, Vol. 49 No. 2, pp. 214-229.

Schacter, D.L., Gilbert, D.T. and Wegner, D.M. (2009, 2011), Psychology, 2nd ed., Worth Publishers,New York, NY.

Shah, C. (2010), “Coagmento – a collaborative information seeking, synthesis and sense-makingframework”, Proceeding of the 2010 ACM Conference on Computer Supported CooperativeWork, pp. 527-527.

Shah, C. (2012), Collaborative Information Seeking: The Art and Science of Making the Whole GreaterThan the Sum of All, Springer, Berlin, pp. 3-8.

Shah, C. (2014), “Collaborative information seeking”, Journal of the Association for Information Scienceand Technology, Vol. 65 No. 2, pp. 215-236.

Sormunen, E., Tanni, M. and Heinström, J. (2013), “Students’ engagement in collaborative knowledgeconstruction in group assignments for information literacy”, Information Research:An International Electronic Journal, Vol. 18 No. 3, pp. 1-16.

Stahl, G., Koschmann, T. and Suthers, D. (2006), “Computer-supported collaborative learning: anhistorical perspective”, in Sawyer, R.K. (Ed.), Cambridge Handbook of the Learning Sciences,Cambridge University Press, Cambridge, pp. 409-426.

23

Collaborativeinformationsearching

Stange, D., Heyn, H. and Nürnberger, A. (2014), “Capturing information gain in informationseeking with concept Maps”, Searching as Learning Workshop IIiX 2014, Regensburg, August,available at: www.diigubc.ca/IIIXSAL/Papers/StangetalAbst.pdf

Twidale, M.B., Nichols, D.M. and Paice, C.D. (1997), “Browsing is a collaborative process”, InformationProcessing & Management, Vol. 33 No. 6, pp. 761-783.

Vakkari, P. (2001), “A theory of the task-based information retrieval process: a summary andgeneralisation of a longitudinal study”, Journal of Documentation, Vol. 57 No. 1, pp. 44-60.

Vakkari, P. (2016), “Searching as learning: a systematization based on literature”, Journal ofInformation Science, Vol. 42 No. 1, pp. 7-18.

Wang, P., Hawk, W.B. and Tenopir, C. (2000), “Users’ interaction with World Wide Web resources: anexploratory study using a holistic approach”, Information Processing & Management, Vol. 36No. 2, pp. 229-251.

White, R.W. (2016), Interactions With Search Systems, Cambridge University Press, Cambridge,pp. 201-230.

Wildemuth, B.M. (2004), “The effects of domain knowledge on search tactic formulation”, Journal of theAmerican Society for Information Science and Technology, Vol. 55 No. 3, pp. 246-258.

Wildemuth, B.M. (2009), Applications of Social Research Methods to Questions in Information andLibrary Science, Library Unlimited, Westport, CT, pp. 73-82.

Xu, J.P. and Zhang, H.C. (2005), “Testing intercoder reliability by multi-approaches in qualitativeresearch”, Psychological Science, Vol. 28 No. 6, pp. 1430-1432.

Yin, C., Sung, H.-Y., Hwang, G.-J., Hirokawa, S., Chu, H.-C., Flanagan, B. and Tabata, Y. (2013),“Learning by searching: a learning environment that provides searching and analysis facilitiesfor supporting trend analysis activities”, Educational Technology & Society, Vol. 16 No. 3,pp. 286-300.

Zhang, P. and Soergel, D. (2016), “Process patterns and conceptual changes in knowledgerepresentations during information seeking and sensemaking: a qualitative user study”, Journalof Information Science, Vol. 42 No. 1, pp. 59-78.

24

AJIM70,1

Appendix 1. Structured diaries at the beginning of the work task ( first stage)

25

Collaborativeinformationsearching

Appendix 2. Questionnaires on the three-hour experiment (second stage)

26

AJIM70,1

Appendix 3. Interviews after the three-hour experiment (third stage)3.1 Labor division modeIn the experiment process, what method of labor division did you choose?

3.2 Participants’ sentiment feelings after the three-hour experimentHow do you feel after this academic group work?

3.3 Participants’ problems during the whole taskWhat are the main problems that you faced during the whole task?

3.4 Participants’ gains after the whole taskWhat have you gained from this group academic task?

3.5 Participants’ evaluation of and suggestions for the collaborative search systemWhat are the main problems you faced when you used Coagmento?Do you have any suggestions for a collaborative search system?

Corresponding authorDan Wu can be contacted at: woodan@whu.edu.cn

For instructions on how to order reprints of this article, please visit our website:www.emeraldgrouppublishing.com/licensing/reprints.htmOr contact us for further details: permissions@emeraldinsight.com

27

Collaborativeinformationsearching