trends in human-computer interaction in information seeking

White Papers Trends in Human-Computer Interaction for Information Seeking

Trends in Human-Computer Interaction for Information Seeking: What’s Next?Rich Miller, Research Scientist, LexisNexis

AbstractThis paper describes a conceptual foundation for integrating new technologies into the next generation of user interfaces for information seeking. The conceptual foundation consists of three primary components that may be used to identify opportunities for using new technology to enhance Human-Computer Interaction: 1) a framework for thinking about information seeking behavior, 2) a vision of information access and organization, and 3) a set of the most significant trends and technologies expected to shape future user interfaces. Taken together, this information can be used to create effective next-generation user interfaces.

IntroductionThe world of human-computer interfaces is on the verge of significant change. Many consider current user interfaces obsolete, and a swarm of new technologies and computing trends have matured to become significant factors in Human-Computer Interaction (HCI). The challenge for product designers and developers is to integrate the most useful technologies in a way that allows users to take advantage of their power while maintaining a high level of usability. This paper focuses on HCI as it relates to online information seeking.

When the now-predominant WIMP (windows, icon, mouse, pointer) interface was introduced in the early 1980s, it was revolutionary. Today’s user interfaces as a whole are not significantly different from those in the beginning of the WIMP era, but there is a growing need for new, more effective interfaces. The main reason cited for this is the tremendous increase in the volume of the information and tools that users are required to manage. For example, the first Macintosh (which represented the WIMP interface entering the mainstream) had no hard drive and was not likely to be networked. Now it is common for a new computer to come with a 60G hard drive and be networked to a world full of an endless supply of data. In addition, there is a plethora of emerging technologies that can be applied to user interface solutions.

A Framework For User BehaviorBefore considering how a given new technology or trend can enhance information seeking, it is useful to have a framework containing the critical components of user behavior that may be used to map user behavior and needs to potential solutions.

Process-oriented model: A great deal of research has addressed information seeking. The process-oriented model based on an electronic information seeking process defined by Marchionini (1995) and shown in Figure 1 will be used as the foundation for the information seeking model. Marchionini’s model has been adapted here to include the concept of organizing information, expected to be important in the future of increasingly more available information.

2002 Interactive Media Forum1

Trends in Human-Computer Interaction for Information Seeking White Papers

Figure 1—Process-oriented model of information seeking.

Activity-based model: The LexisNexis Human Factors group (1997) defined six separate information seeking activities that address user goals. The appendix lists the six activities and related user goals (Bayer, 2000). Figure 2 shows the relationship between these activities and the process model introduced earlier. While some activities involve the entire process, others deal with a subset of the steps.

Figure 2—Process+Activity-oriented models of information seeking.

Previous research in information seeking behavior can be mapped to this framework. The information seeking strategies and tactics shown in Figure 3 were collected from studies on information seeking behavior (Marchionini, 1995; Kuhlthau, 1988, 1991; Ellis, 1989; Bates, 1990; Chun Wei Choo et al., 2000). This integrated view of user behavior can be used to identify opportunities for enhancing information seeking products. The framework can help determine how to enhance HCI by considering to what extent a given technology affects various steps and activities. For example, speech recognition may be more appropriate for behaviors that involve simple query formulation (Step 3), but not for activities associated with Step 6 (Examine Results) that tend to be more visual in nature. This model may also be used to identify what is still unknown about user behavior.



Figure 3—Mapping strategies and tactics to the process- and activity-based models.

Visions of the HCI FutureAlso useful in planning for and creating the next generation of user interfaces is a review of visions of the HCI future from various experts in the computing field. Together, the visions provide boundaries on what is considered reasonably likely. Table 1 summarizes a sample of visions.

Name Vision summaryVannevar Bush In remarkably prescient 1945 paper, proposed a “memex” device that shares

attributes with the world wide web and personal digital assistants.David Gelernter As part of a manifesto on computing, called for the overhaul of the WIMP interface

and proposed a primarily time-based solution, reasoning that humans organize around life events.

Jaron Lanier, Freeman Dyson, Lee Smolin, Nathan Mhyrvold

Part of a group asked to respond to Gelernter’s manifesto. Lanier said Gelernter was too much of an idealist and pointed out “software can be counted on to drag the whole thing down,” calling it a reverse Moore’s Law for increasingly less efficient software. Dyson felt that tools should be the focus of an evolving cyberspace. Smolin and Mhyrvold pointed out that the future of computing could be very much like the present, and use the unchanging general interface for the automobile as an example.

Michael Dertouzos In The Unfinished Revolution, argued that real progress will only be achieved by a human-centered computing approach, aided by the confluence of five emerging technologies: natural interaction, automation, individual information access, collaboration, and customization.

Joe Clabby In Visualize This, perhaps too optimistically predicted a revolution in computing within the next 3 or so years (i.e., by circa 2004). The result is the “Sensory Virtual Internet” driven by the evolution and confluence of five types of technological improvement: computer-to-human sensory output, human-to-computer communication (e.g. speech recognition), network and computer infrastructures,



applications development, and collaborative applications.George Colony Proposed the “X Internet” in which X represents executable and extended, a new

software category that allows for two-way conversation delivered (or extended) via the Internet to many more devices, such as handhelds and cell phones.

Jakob Nielsen A recognized “guru” of HCI, calls simply for easier to use applications/sites, achieved through simple changes such as much larger (and ultimately portable/foldable) displays.

Table 1—Visions of future human-computer interaction.

The Personal Knowledge Base VisionOne way to understand how user-technology integration should evolve is to construct a vision of the ideal information-seeking system within a reasonable reality to strive for. The following vision employs a data-centric perspective, given this paper’s focus on information access and organization.

A Personal Knowledge Base (PKB) is a unique, highly organized and connected repository of information for and about a user. The purpose of the PKB is to support all the information needs of its owner by organizing collected “information objects,” relating information to personal goals, and connecting personal data to the outside world. When such a system exists in a complete and reliable form depends on the rate of continued integration among the current set of disparate data sources that users rely on and interface with today, such as email folders, local and enterprise network folders, workgroup/organization/enterprise portals, Internet browsers, third-party data repositories, handhelds, cell phones, and hardcopy documents, folders, etc. The current, evolving system is tolerated by most users but leaves much room for improvement, considering all the time and effort users spend accessing and organizing information.

Although a system like the PKB may not be perfected within the next few years, it is useful to consider its characteristics and implications, and then plan how best to make information seeking products and services fit with such a system. A conceptual picture of the PKB is depicted in Figure 4. The human accesses data from the world of information through his PKB, which is connected to a larger workgroup KB, which in turn is connected to an even larger organization KB. The inner layer in the figure surrounding the core data represents metadata (e.g., author, publication, and date of an article) used to organize and present data through the outer layer, which represents a virtually unlimited amount of flexible user interfaces or “views.” The PKB is centrally located, secure, and accessible from anywhere using any device. The PKB is constantly evolving and self-optimizing, and may use “agents” designed to look for useful information objects and fit into the PKB it in a meaningful way.



Figure 4—The Personal Knowledge Base.

Figure 5 shows an example of what type of data may comprise a PKB. This view of a PKB is based on the various roles that the user plays, such as homeowner, family member, community member, employee, etc. Figure 6 shows an exploded view of the employee role information, divided into type or purpose of information, such as projects, financial and benefits, and communications. In addition to this type of information, a complete PKB might also have information about a user’s identification for authentication, a “digital wallet” with secure information for automated financial transactions, custom settings for tools and applications, user goals and preferences, and some form of personal “agents.” Benefits of a PKB include less time spent finding and organizing information, more work done for a user through automation, more relevant information presented to a user as it is available, and increased collaboration effectiveness due to links between KBs based on the nature of the relationship between KB owners. The better organized the PKB, the easier it is to create interfaces or views that serve the user for a particular task in a particular context. Thinking about the various ways in which data should be organized for a given user can shed light on the best tools or applications to create.



Figure 5—The composition of a Personal Knowledge Base.

Figure 6—The composition of the “Employee” section of a Personal Knowledge Base.

Trends and TechnologiesListed below are ten changes expected to occur in the user experience as a result of the introduction of new technology. Many of these changes are well underway.



1) Access to increasing amounts of more diverse data.

2) Data available to the user will be much more organized.

3) Personal data stores will be more integrated with external data sources.

4) More will be done automatically for the user, with an increasing reliance on agents.

5) Users will interact using a greater breadth of their perceptual/motor apparatus.

6) Computing will occur at more places.

7) Users will interact with many more devices.

8) Users are more likely to collaborate as collaboration tools are increasingly integrated into applications.

9) Users will interact more with pictures, and user interfaces will be more visual in general.

10) The human attention resource will become more and more precious as the workday becomes more fragmented.

Tables 3 and 4 list the trends and technologies most likely to affect future user interfaces. Table 3 lists those directly affecting HCI (divided into those related to input vs. output) and Table 4 lists more general trends and technologies that are expected to have more indirect effects.

Input

Speech recognition Digitization and interpretation of human speech for dictation or “command and control” input as part of a “natural dialog” between human and computer.

Handwriting recognition Transferring written input from a user into a form that matches keyboard input. Closely tied to pen computing.

Input–further out

Gesture/gaze Sense of touch, now primarily for games controls and virtual reality.Haptics Recognizes hand gestures or eye movements as commands.Biometrics Bio-based authentication–fingerprint ID, retinal scan, etc.

Applications should appear within the next two years and enter the mainstream later.

Thought/direct-brain Interpretation of distinctive brain patterns generated voluntarily by the user as commands.Typically limited to less than five distinct commands.

Prosthetics In the future, prosthetics will move from filling a void to augmentation (e.g., a third arm).

Output

Display evolution Continued evolution toward larger, smaller, and higher resolution displays. Large displays for co-located collaboration, small displays for mobile

computing.Visualization Graphical representation of information designed to optimize review and

analysis, and uncover hidden patterns. Visualization spans a broad range of complexity from simple bar, pie charts to 3D representations of thousands of data points.

Electronic ink “Smart” ink used to create “electronic paper” or digital displays as thin and flexible as paper, yet cheap enough to be sold in volume or integrated together as a notebook.

Virtual reality 3-D immersive environment responding to user actions in a natural way.Often associated with avatars.

Table 3—Technologies directly related to HCI.



General UI evolution Evolution of graphical interfaces for desktops, browsers, etc. Includes factors such as software development tools and user acceptance.

Data explosion The creation of increasingly more data available for consumption. The diversification of data beyond text into rich media, such as images, audio,

and video.Data organization The proliferation and use of metadata to make data collections more intelligent

and organized. Improve users’ ability to access information in a more efficient and context-

laden manner. Includes concepts such as metadata, XML, portals, topic maps, knowledge

management, and the Semantic Web. Pervasive computing The confluence of many forces (e.g. wireless access, mobile workforce and

devices) resulting in the spreading of computing away from the desktop and into the general environment.

Perhaps the most significant current HCI trend.Collaboration Evolution of tools enabling collaboration and communication between co-

workers, customers, etc. Includes e-learning, instant messaging, and workplace evolution

Agents Dynamic software entities that are self-contained and perform tasks proactively on behalf of a user or user-initiated process. Includes synthetic characters.

Customization Customizing the content of interfaces for user types or personalizing for specific users.

Globalization The worldwide production and consumption of online information. Subsumes internationalization (common) and localization (specialized).

Other forces A collection of forces that include copyright/legal issues, digital rights management, security and privacy, standards battles, the increase in the computing/cost ratio (Moore’s law), profit-making forces, basic human behavior (e.g., laziness, self-interest), and geopolitical forces.

Table 4—General trends and technologies



The Hype CurveThe hype curve concept developed by the Gartner Group is a very useful tool for tracking the maturation of a new technology as it approaches adoption. As Figure 7 depicts, the curve begins when a “Trigger” (event or cause) gives birth to a technology, leading to a buildup of hype as the technology approaches the “Peak of Inflated Expectations” before falling downward toward the “Trough of Disillusionment” as the hype yields to concerns about the viability of the technology. As issues are worked out and problems are solved, the technology ascends the “Slope of Enlightenment,” ultimately reaching the “Plateau of Productivity” when enough problems are solved that the technology has entered the mainstream. Certainly not all technologies make it to the “Plateau” and the speed with which a technology passes through the curve can vary greatly.

Figure 7—Hype Curve for HCI-related technologies.

The hype curve in Figure 7 includes technologies that are expected to either directly or indirectly have effects on HCI and user interfaces. Many of these technologies are subsumed under technologies or trends listed in the previous section. For example, Pervasive Computing includes PDA phones and Wireless Web, while Data Organization includes Topic Maps, Agents, DRM (Digital Rights Management), Digital Wallets, Portals, and XML. As the legend shows, the color of a circle representing a technology indicates how many years to expect it to fully mature. The position and color of technologies are based on a review of the literature including predictions from analysts and the author’s analysis of how the technologies may affect user interfaces for information access and organization.



Discussion Now that all the components of the conceptual foundation for designing information seeking user interfaces have been described, we can consider strategies for integrating appropriate technologies into user interfaces, as well as the implications for the evolution of information seeking products. An effective strategy for new technology integration should include developing and maintaining an understanding of both 1) the user behavior and associated needs within a given domain, and 2) a realistic expectation of how various technologies map to user needs and how rapidly those technologies will reach maturation and general adoption. Large gaps in knowledge of these areas can result in wasting time and money on integrating immature or inappropriate technologies. Leveraging the right technology for the right user activities can result in the next “killer” application.

The contents of this paper have many implications for educating and training the next generation of interface designers. Academic institutions should anticipate changes in computing technology and include in programs a consideration of HCI environments at least 2 to 4 years out (when students will be entering HCI-related jobs). Universities should also seek opportunities for collaboration between industry and academia so that there is a clear path between research and implementation. The foundation of any HCI evolution effort must be a deep understanding of human behavior and needs, the most valuable knowledge in determining the usefulness of a given new technology

The concept of applying new technology to the next generation of user interfaces raises many questions to ponder. Listed below is a sample to feed discussion:

What will be state of user interfaces 2, 5, 10 years from now?

What are the effects of the interaction of various trends and technologies? How are the effects of various new technologies dependent on one another?

Independent of the technology available, how will users really want to interact with computers? Related to this question, what are the social implications of interacting with computers using keyboard/mouse vs. speech vs. handwriting input?

To what extent is the Personal Knowledge Base concept realized in today’s computing world? What are the biggest hurdles in realizing the PKB?

What factors affect the speed with which a given technology travels through the hype curve?

What rate of change is healthy and acceptable for users and their organizations? What forces throttle the rate at which new technology is integrated into products?

How should the education and training of HCI–related professionals change over time?



AppendixSix primary information seeking activities (Bayer, 2000)

Activity Goal Example TasksThorough/Exhaustive research

Gather all documents and information

Investigate company for acquisitionResearch a story for publication

Quick & dirty research Find “the” answer for a business problem

Find simple facts that answer a business question Find overview information that covers an issue

Get a specific document Get the single document needed for a work product

Get article from a publicationGet a company’s annual reportFind a country reportFind a bill or floor vote

Validate/Verify specific information

Confirm accuracy or relevance of known information

Check company address/locationCheck a person’s name/titleReview government statistics

Continuously track specific topic/issue

Have the most recent information available for a critical business need

Track a company or industryFollow a competitor in the news

Keep current on general topic of interest

Follow a trend or topic related to my job

Review today's headlinesFollow the latest technology trends



ReferencesBates, M.J. “Where Should the Person Stop and the Information Search Interface Start?” Information Processing and Management. 26 (3). 575-591. 1990.

Bayer. Target tasks for nexis.com users. LexisNexis memo. 2000.

Bush, Vannevar. “As We May Think.” The Atlantic Monthly. 176(1):101-8. July, 1945.

Chun Wei Choo, C., Detlor, B. and Turnbull, D. “Information Seeking on the Web: An Integrated Model of Browsing and Searching,” First Monday. 5(2). February, 2000.

Clabby, J. Visualize This. New Jersey: Prentice Hall. 2002.

Dertouzos, M. The Unfinished Revolution. New York: Harper-Collins. 2001.

Ellis, D. “A Behavioural Approach to Information Retrieval System Design.” The Journal of Documentation. 45(3):171-212. 1989.

Fenn, J. and Deighton. “Emerging Technologies for Human-Computer Interaction.” Gartner Group Report T-14-0298. October 1, 2001.

Gelernter, D. “The Second Coming—A Manifesto.” The Edge. 70. June 15-19, 2000. http://www.edge.org/documents/archive/edge70.html

Gilmour, K. “The Future's Bright!” Internet Magazine. August 1, 2001.

Kuhlthau, C.C. “Developing a Model of the Library Search Process: Cognitive and Affective Aspects.” Reference Quarterl.. 28(2): 232-42. 1988.

Marchionini, G. Information Seeking in Electronic Environments. Cambridge: Cambridge University Press. 1995.


http://www.edge.org/documents/archive/edge70.html

trends in human-computer interaction in information seeking

Technology

chun wei choo

personal knowledge

information

information

user behavior

oriented model

search system

dirty research