Theory Development in the Information Sciences. Edited by Diane H. Sonnenwald. Austin, TX: University of

Texas Press, 2016. 343 pp. $90.00 (hardcover). (ISBN 978-1-4773-0824-0)

Introduction

This book is an anthology containing 17 chapters, the first and last written by the editor and the 15

remaining chapters written by 16 different authors, mainly outstanding researchers from information

science1. As indicated by the title, it is about theory development in information sciences in the plural.

However, nowhere does the editor provide a listing of which sciences are included in her conception of

information sciences. On page one is stated: “Emerging in the first half of the twentieth century as a

discipline, the information sciences contribute …”. This seems for to me to be a contradiction in terms. I

consider the book to be about information science in the singular. There are, for example, no contributors

from museology, information management, terminology studies etc. Almost all authors seems to be

institutionalized within information science (but many of the authors use the plural “information sciences”,

perhaps in order to reflect the title of the book).

Theory cannot be overestimated for the progress of our field. Recently there have been voices about crisis

in information science (e.g., Bawden, 2015; Madsen, 2016; Nolin and Åstrøm, 2010). Although these articles

try to deny the weakness (as does Sonnenwald in this book), or to turn weakness to a strength, the point

here is about how we cope with our challenges and the role of theory for this purpose. One problem seems

to be that other fields are doing better and seem to absorb information science. David Bawden, for

example, wrote:

“It is also true that where LIS does develop genuinely new and interesting ideas, other

disciplines absorb them as their own. Information retrieval is the classic example: 30 years ago,

it was clearly part of LIS, and very few computer scientists took it seriously; 15 years ago it was

1 The table of content is available at: http://utpress.utexas.edu/index.php/books/sonnenwald-theory-development-information-sciences

1

spread across the boundary lines of the disciplines; now, the party line is that it is an integral

part of computer science, and always has been [Bawden provided other examples as well] ”

(Bawden, 2015).

I consider it a fact that computer science today dominates IR. Statistical IR as represented in computer

science is weakly represented in our field. The question is if IR still is also a subfield of information science.

Do we have other approaches that are able to contribute to and may justify our existence as a scholarly

discipline? If the final answer is that the statistical approach simply won, then there seems no future for IR

as a subdiscipline in LIS (and the same may be the case with other subfields). The question of content areas

is closely connected to the question about theories and approaches: There are not God-given or logic based

content areas (but disciplines should perhaps rather be understood as battlefields or territories). Therefore,

the discourse about theory seems to be extremely important for the future of information science. It is

from this perspective that the book under review is examined.

The organization of theories (and of the book)

In chapter one, Sonnenwald uses a theory taxonomy from Gregor (2006): (1) theory for analyzing, (2)

theory for explaining, (3) theory for predicting, (4) theory for explaining and predicting, and (5) theory for

design and action. In the same chapter, she also classifies other chapters according to this classification

(except chapter 7, 8, 9, 13, which are not classified). The organization of chapters in the book, however,

crisscross this classification: Part 1: [Theories about] Behavior of individuals and groups, Part 2: [Theories

about] Evaluation, Part 3: [Theories about] Design, and Part 4: [Theories about] Cultural and scientific

heritage. This organization is rather loose. For example, the chapters by Bawden and Meadows are placed

in Part 4 but are not about cultural and scientific heritage.

2

There are many ways to classify theories. Case (2012, p. 167) suggested a classification by paradigm, grand

theory, formal theory, substantive theory and observations. Sonnenwald does not consider alternative

ways of classifying theories and although Gregor’s taxonomy seems to work well (as also demonstrated by

Thelwall in his chapter on theory development in webometrics) I have reservations about it. It seems

superficial, ahistorical, and not fit to distinguish the major theoretical perspectives in information science.

Take the understanding of “information” as an example:

(1) in information theory “information” is defined:

“As a mathematically defined quantity divorced from any concept of news or meaning […];

spec. one which represents the degree of choice exercised in the selection or formation of

one particular symbol, message, etc., out of a number of possible ones, and which is

defined logarithmically in terms of the statistical probabilities of occurrence of the symbol

or the elements of the message.” (Oxford English Dictionary, Information, sense 2c).

(2) In the cognitive view:

a. “On the one hand information being something which is the result of a transformation of a

generator’s knowledge structures (by intentionality, model of recipients’ state of

knowledge, and in the form of signs) and

b. On the other hand being something which, when perceived, affects and transforms the

recipient’s state of knowledge.” (Ingwersen and Järvelin, 2005, p. 33; italics in original).

(3) In documentation theory:

“The term "information" is also used attributively for objects, such as data and documents,

that are referred to as "information" because they are regarded as being informative, as

"having the quality of imparting knowledge or communicating information; instructive."

(Oxford English Dictionary, 1989, vol. 7, p. 946)." (Buckland, 1991b, p. 351). Consider this

3

implication: "We conclude that we are unable to say confidently of anything that it could

not be information" (Buckland, 1991a, p. 50. Underlining in original).

(4) In a socially informed theory:

"An item of information is an interpretation of a configuration of signs for which members

of some social group are accountable." (Goguen, 1997).

Each of these different theoretical perspectives (information theory, cognitive theory, documentation

theory and social theory) are both about analysis, explanation, prediction and action in information science

and the example above nicely demonstrates the principle that different theories tend to understand basic

terms differently. As we shall discuss in the next session, both the classification theories as well as the view

of theory development seems to be too narrow. Concerning the different functions of theories, I strongly

agree with Stephen Robertson:

“I consider myself a theorist. That is, my inclination is to theoretical argument, to achieving

theoretical understanding, in information retrieval as in other realms. To me, understanding is

what theory is about; those other attributes of theory, prediction and application, are side-

effects only, secondary to the main purpose” (Robertson, 2000, p. 1).

Theory development

The goal of the book is most clearly described in the last short chapter “Supporting future theory

development” by the editor. The goal is to improve theory development by individual researchers, for

example during their doctoral training. Many of the chapters in the book are autobiographical reports

about how the authors constructed theories in their respective specialties (intended as learning exemplars).

The overall perspective seems extremely individualistic, however. As Bonnie Nardi writes in her chapter

“Appropriating theory”:

4

“Many of the chapters in this volume concern the development of new theory. I want to take a

slightly different tack and focus on the scholar’s appropriation of existing theory. I believe that

such appropriation is a critical step along the way in developing new theory and that many

existing theories are underexploited” (p. 204).

Yes indeed! The overall situation in information science today is a chaos of theoretical contributions, each

paying no or much too little interest in the existing ones, what Åström (2006, p. 20) after Whitley (1984)

called a 'fragmented adhocracy', a field with a low level of coordination around a diffuse set of goals. If this

is the case, PhD students should not just be encouraged to develop new theories, but primarily to consider

major theoretical positions, which are often implicit and therefore have to be uncovered by historical-

theoretical analyses of information science.

About the single contributions

2. Marcia J. Bates: Many paths to theory: The creative process in the information sciences

This chapter both contains advice such as “be open”, “draw on a variety of research traditions” and “read

deeply” and then goes into some specific areas of information science. I agree very much with a point

made, that human-computer interaction (HCI) looks different from an information-seeking point of view

compared to the more general field of HCI. (In my view, information science is more about humans

communicating with the accumulated recorded knowledge with or without the use of computers, and is

not primarily involved in HCI). It is thus important always to consider to research from the perspectives of

your own field, a point also made by other authors, for example, Andrew Dillon in chapter 12.

3. ShanJu Lin Chang: Reflections on theory construction in human information behavior: A theory of

browsing.

5

This a description of the author’s research about browsing and theory construction, but it ends up with the

conclusion that the developed theory is not the final word about browsing. In comparing behavioral with

cognitive perspectives, Chang finds the cognitive perspectives most useful. What the chapter does not do,

however, is to relate more systematically browsing theory to metatheory in information science. It seems

obvious to me, that some domains have more strict criteria of relevance (corresponding to Buckland’s

figure p. 107 that some domains are more compact while other are more diffuse). Is it not probable that

browsing activity is shaped by cultural factors and scholarly metatheories? That, taking Buckland’s figure as

the point of departure, should physicists be more constrained in their professional browsing compared to

zoologists? Also, that a paradigm shift in a domain tends to change the focus of all forms of information

behavior? The chapter provides a fine overview of Chang’s research process but used as a lesson for PhD

students it may say much about browsing but less about theory development in general.

4. Carol Collier Kuhlthau: Reflections of the development of a theoretical perspective.

Kuhlthau is one of the leading researchers associated with “the cognitive view” in information science and

this chapter is a fine presentation of her theory-developing process. She refers to John Dewey and Lev

Vygotsky among the basic theoretical inspirations–the same authors that I also often include in my own list

of basic inspirations. Kuhlthau refers to “constructivism”, a term with different interpretations. I see myself

closer to social constructivism than to psychological, individual constructivism. Kuhlthau’s empirical and

theoretical work describes stages in the development of students information seeking (termed the ISP

model, “information search process model”). Well, I do not believe it provides a realistic picture of

information seeking and use. For me, Jerome McGann’s autobiographical chapter in this book offers a much

more realistic picture. Every time a serious student works with a problem, she discovers that she does not

know what is needed to answer the question, and that new information is needed, which unfortunately

opens up huge new literatures with too little time available. (Of course, students mostly learn to satisfice

but is that a good thing?). The situation in which students and researchers find themselves are social

6

conditions, including paradigmatic conflicts and uncertainties. Whether we are pessimistic or optimistic is

not just a matter of our own psychological construction, but also the success of the field in which we are

working and the opportunities that seems open to us. Therefore, I believe that Kuhlthau’s view —and

generally, the cognitive view in information science— is based on what has been termed “the psychological

fallacy”:

"In 1910 Dewey [p. 250] wrote a statement that expresses a central tenet of cultural

psychology. He said that the processes that animate and form consciousness lie outside it in

social life. Therefore, the objective for psychologists is to use mental phenomena (e.g.,

perception, emotions) as clues for comprehending the life processes that they represent. [...]

´The supposition that these states [of consciousness] are somehow existent by themselves and

in this existence provide the psychologist with ready-made material is just the supreme case of

the "psychological fallacy"´ " Ratner at the same place also referred to Vygotsky (1997, pp.

272-273, 327) "for a remarkably similar statement" to Dewey´s. (Ratner, 2002 , p. 3).

If PhD students do not learn about such criticism, where does that bring them in their own theory

developing process?

5. Gary M. Olson and Judith S Olson: Converging on theory from four sides.

This chapter reports on the author’s theory building in relation to distance collaborations, including large-

scale scientific collaborations. The theory building was done by four main classes of resources: (1) one’s

own empirical research, (2) mining the literature, (3) compilation of data from more than seven hundred

collaboratories, and (4) an online assessment tool build by the authors. For me, the chapter does not seem

to suggest a real theory, just sets of factors affecting such collaboratories. It is my suggestion that a deeper

foundation in the sociology of science could strengthen the theory development. Whitley (1984) provided a

7

taxonomy of the sciences based on two main factors: (1) The degree of mutual dependence between

scientists and the organization of scientific fields and (2) The degree of task uncertainty and the

organization of scientific fields.

Again, students may learn a lot about scientific collaborations and many of the things reported in the

chapter are important. However, a deeper level of theory-discussion may provide better possibilities for

independent theory construction.

6. Michael K Buckland: Drawing graphs for theory development in bibliometrics and retrieval.

Buckland is well known as a major theoretical contributor to information science, arguing for, in particular,

document theory and cultural theory. This chapter seems at first to be different. It consists of two case

studies about the relations between (1) obsolesce and scattering and (2) recall and precision. It argues

about the fruitfulness of drawing graphs, but it could also – at a deeper level –- be considered about the

fruitfulness of thought experiments. For example, in information retrieval the fruitfulness of comparing

three extremes: (a) the perfect search, (b) the random search, and (c) perverse retrieval. The chapter is

important and so seems the conclusions, including this: “Over the years bibliometrics has become a hugely

sophisticated specialty in which dazzling quantitative analysis of surface phenomena have not yet been

accompanied by any know progress in explaining the deeper causal issues at the core of the information

sciences”. (It should be mentioned, however, that Buckland’s claim about the necessary inverse relation

between recall and precision has been refuted by Fugmann, 1994 and elsewhere). At the deeper level, this

chapter may be related to document theory and cultural theory by considering bibliometrics and retrieval

in the light of different scientific conditions.

7 Kalervo Järvelin: Two views on theory development for interactive information retrieval.

This chapter presents an advanced argument for theory development in a core area of information science

by one of the most prestigious researchers in the field. My most important comment is that the chapter is

8

an important contribution to the book. It represents the experimental approach- or rather an extension of

the well-known laboratory approach founded by the Cranfield experiments and continued by the TREC

experiments. The extension consists mainly in the inclusion of the human side of the search process by

using simulated search tasks. It considers theory development closely connected to or derived from

experimental findings.

That said, I believe that many serious objections can and should be raised, and the approach is different

from what I consider the way forward. Just a single example, Järvelin wrote (p. 126): “Even success in IIR

[interactive information retrieval] is debatable; how should success be measured?” My suggestion is that

the answer is epistemological, that, for example, evidence-based medicine has explicit (although

debatable) criteria of what should be found. From such criteria, the effectiveness of search systems must

be evaluated (see further in, for example, Hjørland, 2016a).

My main point is that theories are not just derived from experimental findings, but are primarily in the head

of the researcher prior to the experiments. Experiments may be important, but are always limited by the

implicit assumptions in the head of the experimenter. There is not space in this review for further

discussion, but I hope to be able to continue my analysis in other contexts.

8. Tefko Saracevic: Relevance: In search of a theoretical foundation

I agree with the author that relevance is a core concept in information science. However, in 1975 Saracevic

declared “the subject knowledge view” to be the most fundamental perspective of relevance, but this view

was since forgotten or repressed without argument, which I consider problematic (see Hjørland, 2010). My

suggestion is that conceptions of relevance depend on basic theories of information science just as the

concept of information in the example above and that improvement first and foremost is connected with

realizing the limits in the cognitive and user-oriented conception of relevance. Compare Buckland’s

comment in this book (p.113): “The notion of relevance itself is problematic and unscientific”.

9

9. Mike Thelwall: The story of a colony: Theory development in webometric research

The author does not consider himself as engaged in theory development, but reports two exceptions in

which he developed theory: (1) The development of information centered research (ICR) and (2) The

theoretical framework for link analysis (TFLA). The first seems strange for me. As I read it in this chapter (I

have not consulted the primary papers) I understand it as suggesting an atheoretical description of new

web-phenomena for the purpose of pointing out research problems for which they are relevant. The

problem I see with this is that what counts as a relevant source for a given research project depends

strongly on the questions asked and the theory behind the research questions. I have addressed this

problem in the context of classifying and indexing documents in many publications. The second theory

(TFLA) seems also for me to be just a special case of the theory of citation motives and citation behavior in

informetrics. I expect webometric theories to be just special cases of bibliometric theories, and I have

outlined some thoughts in Hjørland (2016b).

The final words in the chapter are the acknowledgement of the importance of theory and a tribute to

Merton’s sociology of science. Yes, webometrics should certainly be considered from the perspectives of

the sociology and philosophy of science, among other metasciences.

10. John M. Carroll: Theorizing the unprecedented

I will limit my review of this chapter to declare my full support for the conclusion: “A programmatic

implication of this chapter is that information science and technology should take scenarios more seriously

as theoretical objects” (p. 200). I could add that to raise the question, as did Hahn (2003), “What has

information science contributed to the world?” seems equally important: Our visions of the future should

be based in our identity, of what we have done in the past, and the lessons we have learned.

11. Bonnie Nardi: Appropriating theory

10

As already mentioned, this chapter is not about the author’s development of her own theory, but by her

discovering and appropriating an existing theory, which is “activity theory” or “cultural-historical activity”,

CHAT. Like the author, I too consider this theory important. However, its most fundamental claim that

human psychology and actions are mediated by language and other culturally developed tools, is a view

shared by many other theories (such as semiotics, hermeneutics, pragmatism, paradigm theory and social

epistemology). I certainly find this family of theories not just fruitful, but strongly necessary for the further

development of the information sciences (but badly represented in the present book). Although Nardi

argues for CHAT in human computer interaction and thus for the design of IT-products, she also confesses

that much good design is made without theory. Her most important argument (p. 210) is perhaps that

unless you compare theories, you are unable to see the relative strengths and weaknesses of the theories

you use, explicitly or implicitly).

12. Andrew Dillon: Theory for design: The case of reading.

This chapter is about human interaction with textual information in both print and electronic media. It

argues that the sentences such as “reading from paper is faster than reading from screens” are

problematic. In order to be answered meaningfully it should be specified what kind of text, for what

purpose the text is read, etc. The author has developed a theoretical model for such analyses. He writes a

lot about the psychology of reading but argues that “the information sciences must find a level of

theoretical insight that serves its purpose, and this will inevitably require theories that are not just derived

from existing [=other] disciplines. To do this requires those of us in the field to take the steps of theory

production and to share those steps with others [in the field]” (p. 237).

13. Jerome McGann: The poverty of theory; or, the education of Jerome McGann.

This chapter is written by a leading scholar in digital humanities and editorial philology/text studies. It is

written in the third person and is extremely self-critical and ironic. An example of a main point is that the

11

author as expert in book studies turned to computers to discover that he knew nothing of books! The field

he is writing about is certainly relevant for information studies (I know only one person from information

studies working in this field, Mats Dahlström). Some of the important conclusions are: (1) McGann “found

he no longer knew how to separate his scholarly from his political life” (p. 243), (2) That the convention

developed out of nineteenth-century British models of the series of Oxford English Text editions turned out

to be troublesome (p. 245), and (3) “That the scholarly edition —any effort at scholarly editing, for that

matter—was itself a theoretical move of a special and crucial kind: an act of literary interpretation carried

out performatively, as an act of textual representation” (p. 250-251).

14. Hilary S. Crew: Illuminating Daughter-mother narratives in young adult fiction.

This chapter has in my opinion nothing to do with the information sciences. Because librarianship and

information studies are about mediating all domains, this does not imply that any writing in any discipline is

part of information studies. This misunderstanding is one of the reasons our field is in a crisis. By including

this chapter, the editor implicitly says that it is about theory development in the information sciences and

that it could be used as inspiration for students and researchers in our field.

15. David Bawden: The noblest pleasure: Theories of understanding in the information sciences.

Bawden is interested in domain theory (chemistry, in particular) and considers information science to be a

metascience. He is also interested in Shannon’s information theory, in Bertie Brookes’ theory based on Karl

Popper, among others. These are general theories of information science, but Bawden also writes:

“[I]t follows that it is unreasonable to expect there to be “a theory” of information science

specifically, or of the information sciences more generally. Rather, there will be a range of

theories, dealing with different aspects of the subject,”(p. 287). ”

12

Bawden is one of the few researchers who demonstrates interests in the field as a whole and at the same

time contributes to specific issues. I must admit that I would have liked more discussion about the relative

strengths and weaknesses of the different theories. Bawden has contributed, for example, to Aitchison,

Gilchrist and Bawden (2000), but I fail to see how information theory can advance this field (compare

Hjørland, 2016a) or any other field of information science. However, Bawden’s papers are well researched

and important, and always marked by his deep and sympatric wish to understand the things he writes

about.

16. Jack Meadows: Apologia pro theoria sua.

This chapter describes the career of the author from astronomy research, to researcher in the history and

philosophy of science, to information and library studies research. It comments about navigating these

different theoretical traditions. There is no overall conclusion about theory development in information

science, and for the reviewer it is disappointing that no stronger conclusions from the background in

astronomy and the philosophy of science are drawn. One of the things mentioned en passant could, in my

opinion, be generalized as the most fundamental principle for the field: “[I]t became evident that research

groups could sometimes be at loggerheads, and that this might affect their refereeing of each other’s work

for publication.” Yes, indeed, every level of the information chain, every actor in knowledge production,

information dissemination and use is of course based on views of the knowledge/information

communicated. Therefore, different phenomena at the deepest level can be explained by explicit or implicit

theories of the actors.

Conclusion

This book is important because theory development is important and because it presents views and

experiences by leading scholars related to information science.

13

However, one of the things I missed in the book was information about the editor’s positions and overall

view. It would have been a good thing in the introductory chapter to include a brief overview of related

books and a statement about how she considers the present book fitting into the existing literature (for

example, lbekwe-SanJuan & Dousa, 2013 and Leckie, Given & Buschman, 2010).

The selection of authors, views and subfields subtopics can, of course, be discussed. There are no papers in

knowledge organization, information literacy, or social media such as Wikipedia. There is very little

discussion of the different identities of the information sciences (and the relations between information

studies and other fields such as computer studies, cognitive sciences, sociology and philosophy of science

etc.).

A number of chapters described empirical research and theory developed from these. There is, however,

not much about theoretical studies as such, about theoretical, historical and philosophical methods for

studying theories (although mentioned in chapter one). In the chapters about empirical research (e.g.

chapter 3 on browsing and chapter 5 on distance collaboration) the impression is that we learned much

about the specific subfield, but the question remains how useful this is for theory development in other

subfields. There are very few named theories in the book (and the index sometimes fails to refer to

theoretical positions, even if they are mentioned, e.g. postmodernism, p. 316).

Returning to the starting point about the current state of information science, the book confirms the

impression of a fragmented field by presenting 16 different voices rather than tendencies towards

synthesis. I see this as partly a consequence of the individualistic focus of the book.

14

Nonetheless, the book is important by providing a snapshot of information science and a platform for

further considerations. Hopefully, it will be followed by more books and discussions about theoretical

issues in information science(s). This is urgent.

Birger Hjørland

Royal School of Library and Information Science, University of Copenhagen,

6 Birketinget, DK-2300 Copenhagen S, Denmark

Email birger.hjorland@hum.ku.dk

References:

Aitchison, J., Gilchrist, A., & Bawden, D. (2000). Thesaurus construction and use. A practical manual. 4.

edition. Chicago, Ill. : Fitzroy Dearborn Publishers.

Åström, F. (2006). The social and intellectual development of library and information science. Umeå: Dept.

of Sociology. Diss. http://www.diva-portal.org/smash/get/diva2:145144/FULLTEXT01

Bawden, D. (2015). On not being a weak discipline. Blog: The Occasional Informationist irregular thoughts

on the information sciences. July 6, 2015 http://theoccasionalinformationist.com/blog/

Buckland, M. (1991a). Information and information systems. New York: Greenwood Press.

Buckland, M. (1991b). Information as thing. Journal of the American Society of Information Science, 42(5),

351-360.

15

Case, D.O. (2012). Looking for information. A survey of research on information seeking, needs, and

behavior. Third edition. Bingley: Emerald.

Dewey, J. (1910). The influence of Darwin on philosophy and other essays on contemporary thought. New

York: Holt.

Fugmann, R. (1994). Galileo and the inverse precision-recall relationship. Knowledge Organization, 21(3),

153–154.

Goguen, J. A. (1997). Towards a Social, Ethical Theory of Information. In: G. Bowker et al. (Eds.). Social

science research, technical systems and cooperative work. Beyond the great divide (pp. 27-56). Erlbaum

Gregor, S. (2006). The nature of theory in information systems. MIS Quarterly, 30(3), 611-642.

Hahn, T. B. (2003). What has information science contributed to the world? Bulletin of the American Society

for Information Science & Technology, 29(4): 2-3.

Hjørland, B. (2010). The foundation of the concept of relevance. Journal of the American Society for

Information Science and Technology, 61(2), 217-237.

Hjørland, B. (2016a). Does the traditional thesaurus have a place in modern information retrieval?

Knowledge Organization, in press.

Hjørland, B. (2016b). Informetrics needs a foundation in the theory of science. In C. Sugimoto (Ed.).

Theories of Informetrics and Scholarly Communication (pp.20-46). Berlin: Walter de Gruyter.

16

lbekwe-SanJuan, F., & Dousa, T. (red.). (2013). Fundamental notions of information communication and

knowledge. Dordrecht: Springer.

Ingwersen, P. & Järvelin, K. (2005). The turn: Integration of information seeking and retrieval in context.

Dordrecht: Springer.

Leckie, G. J.; Given, L. M. & Buschman, J. E. (2010). Critical theory for library and information science.

Exploring the social from across the disciplines. Westport, Conn.: Libraries Unlimited.

Madsen, D. (2016). Liberating interdisciplinarity from myth. An exploration of the discursive construction of

identities in information studies. Journal of the Association for Information Science and Technology, Article

first published online: 23 DEC 2015. DOI: 10.1002/asi.23622

Nolin, J., & Åstrøm, F. (2010). Turning weakness into strength: Strategies for future LIS. Journal of

Documentation, 66(1), 7–27.

Oxford English Dictionary, Information, sense 2c. Retrieved from http://www.oed.com/

Ratner, C. (2002). Cultural psychology: Theory and method. Dordrecht: Kluwer Academic.

Robertson, S. (2000). Salton award lecture: On theoretical arguments in information retrieval. Talk given at

SIGIR 2000, Athens, July 2000. ACM SIGIR Forum, 34(1), 1–10. Retrieved from

http://www.soi.city.ac.uk/~ser/papers/salton_lecture_web.pdf

Vygotsky, L. S. (1997). Educational psychology. Boca Raton, Fl.: St Lucie Press (originally written 1921).

17

Whitley, R. R. (1984). The Intellectual and Social Organization of the Sciences. Oxford: Oxford University

Press.

18