1

A Collaborative Property-Based Note Management System

Jonathan Huang Computer Science Division

Electrical Engineering and Computer Sciences Department University of California

Berkeley, CA 94720-1776

Abstract NotePals is an ink-based, collaborative note sharing application built on top of a property-based document management system. NotePals users write notes in their own handwriting on a per-sonal digital assistant (PDA) or CrossPad. These notes are then synchronized with a central re-pository and can be accessed and shared via an online web application. The system supports both the traditional folder-based hierarchy for document organization, as well as creating and using customized properties for managing, searching, and operating on notes. The latter method allows users more organizational flexibility without losing the benefits of sharing notes in a collabora-tive environment. This report describes our second design and implementation of the NotePals system, which adds several new features, including the property-based organization.

1 Introduction NotePals is a web-based application that allows its users to share handwritten notes taken on portable pen-based devices, such as PDAs and CrossPads. It was first developed to facilitate the communication of ideas and experiences among members of a workgroup. Group members often exchange knowledge and information while working together in a meeting environment. Tradi-tionally, as information is discussed, the members can either take their own notes to attempt to capture the discussed information, or assign a scribe to take notes for the entire group. In the first scenario, some details may be lost depending on how well each person can remember what was discussed. In addition, whenever a group member is actively speaking and not taking notes, he or she is losing the opportunity to write down some piece of meeting information that may be forgotten later. Furthermore, those participants who could not attend the meeting would have to rely on the notes of others to attempt to understand what was discussed at the meeting. In the second scenario, where a scribe is recording information according to what he or she thinks is relevant and important, she may be leaving out information that could be considered as relevant to one or more of the meeting participants. Furthermore, if the scribe is someone who does not have the same background knowledge as the other members of the group, she may not understand all of what is being discussed and may leave out even more information. Even if the scribe is a member of the group who has the same background knowledge of the discussion topic as the other group members, she will have to take a less active role in the meeting since she will be focused on note taking.

2

To address these problems, NotePals was developed to allow meeting participants more direct access to the thoughts of their colleagues through the use of shared meeting notes. Coupled with the advent of cheaper, smaller, pen-based portable devices such as PDAs and CrossPads, users can now record their notes electronically without sacrificing the ease of traditional pen-and-paper interface. Furthermore, since the World Wide Web is a universal medium for information shar-ing and dissemination, NotePals takes advantage of this medium by providing its users with web-based access to the shared notes. In addition, NotePals users can organize notes using a folder-based hierarchy, and search for notes through a variety of contexts, such as author, date, type, and keywords. The original NotePals system [1, 2, 3, 4, 5] was developed and used actively by members of the Group for User Interface Research at the University of California at Berkeley. Through the use of this application, members of our group were able to more easily gain access to each other’s meeting notes, along with notes from conferences, classes, etc. However, a few problems with the NotePals web interface became apparent over time (see Figure 1). First, the system provided a limited number of search methods, and was not extensible to finding notes by other contexts. Users could find notes by specifying their project name, author, creation date, type, or keywords, but would not be able to find it by other means, such as the location at which the notes were taken. In addition, the dependency on a single shared hierarchical organization meant that users could not create their own organization schemes and still share notes with other users – users would need to agree on the same organization hierarchy. Furthermore, note sharing was not in-tuitive and available to everyone – note sharing was done by restricting access to the projects they belonged to, and only NotePals administrators had the privilege to change these settings. And finally, there was not any support for sharing notes among specific groups – all NotePals users were part of the same group, and once a note was made public, it was available to everyone in the system. To address these problems, we have developed a new version of NotePals. The new system al-lows more flexibility in managing, searching, and operating on shared notes, while providing additional support for groups and community-based interactions. This new version of NotePals still uses the same pen-based devices for capturing notes, as well as a web-based interface for note access and management. However, the entire application has been redesigned to take ad-vantage of the latest web, server, and database technologies, such as HTML styles sheets, client-side JavaScript, Java Servlets, and Microsoft SQL Server. Moreover, the entire system backend has been redeveloped into a property-based document management system, while at the same time providing backwards compatibility for traditional folder-based hierarchical organizations. Additional support has been added for creating dynamic group relationships, as well as a more interactive group environment through the use of community-based message boards. And by leveraging a more powerful, commercial database backend, the new system now provides in-creased query performance, as well as more methods for finding notes. The rest of the report is organized as follows. First, we discuss work related to NotePals and its note sharing characteristics. Next, we describe the fundamental characteristics of a property-based document organization system. We then justify our design for the new NotePals applica-tion with a discussion of our design discoveries through multiple low-fi prototypes and user test-

3

ing sessions. Following these results we then focus on the new NotePals features and discuss its internal architecture. Finally, we describe our plans for future work and conclusions.

Figure 1. The old NotePals web interface

4

2 Related Work The concepts and design of the new NotePals system is based on the original NotePals applica-tion. Here we will compare and contrast NotePals with similar research in two main areas: com-puterized meetings rooms and personal ink-based note-taking systems.

2.1 Computerized Meetings Rooms Some meeting room systems seek to improve specific kinds of meetings by structuring meeting activities. The Electronic Meeting System, for example, leads a group through brainstorming, idea organization, voting, and comment phases [6]. These tools can improve the quality and number of ideas generated by a group, but they are inappropriate for other styles of meetings. Other meeting room systems make no attempt to structure meetings, but instead give participants new means to communicate and record meeting activities. These systems often give participants access to traditional applications that have been group-enabled [7, 8, 9]. Some of these systems have been shown to help groups create documents that better reflect a group’s ideas and deci-sions. These meeting room systems share some problems. They may shift a meeting’s focus to docu-ment creation, redirect some of the group’s attention to complex computer interfaces, and they often require participants to type during meetings, which can be disruptive. Another class of meeting room systems tries to enhance natural interaction styles or record keep-ing methods, without shifting the meeting focus or process. WeMet [10], for example, provides access to a shared drawing space running on multiple workstations. Tivoli [11] allows users to manipulate handwritten text in structured ways on a LiveBoard [12]. Tivoli notes and meeting audio can be captured together, allowing participants to access the audio from the notes after the meeting [13]. Similarly, the Classroom 2000 project [14] records classroom audio, presentation slides, and the professor’s LiveBoard notes, and provides ways to browse through them after class. An early prototype of this system merged handwritten notes taken on Apple MessagePad PDAs with lecture slides. A later system allowed students to write directly on electronic versions of the presentation slides using a display tablet. Unlike Classroom 2000, we do not assume class-room settings, and we focus on the sharing of notes between meeting participants. These more natural systems have been influential in our work with NotePals. We have imple-mented many of these ideas in cheaper, more portable systems than the traditional computerized meeting room. This is important for supporting informal meetings or conferences. Unlike these systems, we also focus on the sharing of personal notes and information between group members. Dolphin allows co-located and remote groups to link personal notes and documents in shared spaces [15]. NotePals also links notes and documents, but we make these links automatically when possible.

2.2 Personal Freehand Note-taking Systems Since typing can interfere with many note-taking situations, we have also taken inspiration from research in informal, personal note-taking systems. Freestyle allows personal, handwritten notes

5

and annotated documents to be shared using electronic mail [16]. NotePals also takes advantage of the simplicity of informal, personal note taking, but has more automated sharing and supports note taking away from the desktop. The PARCTab [17] was an early handheld CSCW system that supported a simple shared draw-ing application. This showed that small devices could be used collaboratively. There has also been research in portable, handwritten note taking and audio recording systems, such as Filochat [18] and Dynomite [19]. Handwritten notes written with these systems can be used to access au-dio that was recorded when they were created. The simplicity of these note-taking interfaces and the automatic recording of audio context makes these systems very similar, in spirit, to NotePals. NotePals, however, uses even cheaper hardware and allows personal notes to be shared. Inter-views of Dynomite users showed that free-form ink can be lighter weight and more expressive than text entry.

3 Property-based Document Organization To address the problems in organizing and sharing notes in the original NotePals application, we decided employ the idea of a property-based document management system. We were inspired by the Placeless Documents project developed by researchers at the Xerox Palo Alto Research Center (PARC) [20, 21, 22, 23, 24]. Placeless Documents seemed like an ideal solution to our problems because they address both the problems of a rigid singular hierarchical organizational scheme, as well as the complications from sharing documents in a collaborative environment. In a Placeless Document system, instead of referring to documents by their location in a con-tainment hierarchy, users query the system for the desired documents by referring to their properties. A document’s properties usually consists of property/value pairs, such as author = john, topic = placeless, type = paper, status = draft, and so on. The property values themselves can be arbitrary, and a user can create any number of properties and assign them to documents. To find one or more documents, the user would then specify the desired document properties to find the results. Unlike traditional document organization systems, where documents in different locations repre-sents disjoint document sets, documents in a Placeless Document system can exist in multiple locations at the same time. To provide a compatible metaphor to the desktop interface, research-ers at PARC have developed the idea of collections as entities for grouping documents and act-ing on them collectively. Documents in Placeless can therefore simultaneously exist in multiple collections, and are referred to as the same document despite their differing “locations”. In addi-tion, document membership in collections can be organized dynamically according to document properties. Since collection membership is dependent on queries, documents can automatically appear or disappear from collections depending on whether they satisfy a particular collection’s query criteria. Although these types of collections, known as dynamic collections, are extremely powerful, they present problems in interactive systems due to the fact that documents can appear and disappear from collections without a user’s direct intervention. To address this problem, the researchers at PARC added inclusion and exclusion lists that take precedence over a collection’s query criteria. Documents in a collection’s inclusion list always appear in the collection, even though they may not satisfy the collection’s query criteria, and likewise, documents in the exclu-sion list are always removed even when they satisfy the constraint. Researchers at PARC have

6

incorporated the manipulation of these lists into direct manipulation interfaces (by dragging documents in and out of collections, for example) to make the interaction more familiar. There are several advantages to using a property-based document management system. First, document properties are embedded directly in the document themselves, as opposed to encoding it in document storage locations (e.g., c:\my documents\conference\talk\placeless\). This allows documents to be moved from one collection to another without losing their characteristics. Sec-ond, the fact that a document can have any number of unordered properties associated with it implies that it can be organized without the limitations of hierarchies. For example, in the path example at the beginning of this paragraph, if I decided to reorder my documents first by “talk” and then by “conference,” then I would need to create two new directories (talk\conferences), and move my documents from the old directory to the new. And if those documents were shared with another colleague, then he or she would no longer be able to find them in the old location. However, in the property-based scenario, since documents are arbitrary and unordered, I would not need to do any additional work to find these documents. I would still refer to them by refer-ring to their properties (e.g., owner = me, location = conference, type = talk, topic = placeless). The third advantage to using properties to organize documents is that document properties can be encapsulated at the user level so that different user can apply different properties to the same set of documents. This is ideal for a collaborative environment where documents are often shared, but users may have different ways of organizing the same set of documents. Users can optionally choose to share document properties with other users to expose additional information about the document. The new NotePals applies the property-based document management idea to managing notes in the following ways: 1) The system assigns to each note a base set of properties that is available to every user. These properties are defined as the base or “core” properties. 2) In additional to the core properties, each user can define customized properties with text-based name and values. The property values are optional, and users can query for notes according to whether they con-tain a given property or not (e.g., for a property named “important,” a user can find notes that either contain this property or not). 3) NotePals implements collections with static memberships (i.e., users have to manually move notes into and out of collections), but a note can appear in multiple collections at the same time. 4) Collections can be shared between different users, and notes in shared collections are shared accordingly.

4 Design Discoveries We constructed several versions of both low and medium-fidelity prototypes of the new Note-Pals user interface accompanied with user testing before arriving at the new web-based interface. We will now discuss two of the designs, their basic concepts, and the lessons we learned from the user studies.

4.1 Initial Design Our first prototype featured multi-level dynamic categories and a member-based note sharing system (see Figure 2). Dynamic categories are query-based “folders” whose content changes automatically depending on the documents and their associated properties within the system.

Upon entering the system, users could create or join note groups (e.g., Statistics class, HCI re-search group, etc.) to share notes with other users of the system. Note groups can contain one or more dynamic categories, and members of the same group have access to public group categories. Users are by default members of a Personal group, made up of private categories Recent Up-loads and My Notes. Users could also create ad-hoc categories whose contents are static and be-have the same way as in folder-based organization systems. Furthermore, users could create mul-tiple levels of categories to reflect current hierarchical organization style.

To perform doscreen (e.g., anotes matchinThe system alscreate propertithese propertiequery: “show a

4.2 User We tested thismany differenthe study the p

Figure 2. Initial paper prototype for the new NotePals web-based user interface

7

cument queries, users set property values that appear as icons on the top of the uthor, date/time, type, keyword, location, etc.). Clicking the search button returns g the property values, with the option of saving the current query as a category. o allows users to create new properties and assign them to documents. Users can es with simply a name and a list of possible string values. The user can then assign s to notes and use them to organize notes by creating dynamic categories (e.g., ll notes with author jhuang”).

Study initial property-based design with five UC Berkeley undergraduate students from t disciplines, ranging from engineering to biological and social sciences. During articipants were asked to perform seven tasks that required retrieval of notes using

properties, organization of notes using categories and note groups, and manipulation of notes us-ing editing tools. The retrieval tasks asked the participants to find specific notes by specifying different note properties. The organization tasks prompted the user to create and modify catego-ries and group for storing and sharing notes. And the note manipulation tasks required the user to modify a note’s attribute by using the tools provided, such as adding an attachment, changing its date, etc. The study indicated that although users were comfortable with retrieving notes based on their properties and with manipulating categories, they were not aware of the dynamic nature of cate-gories (i.e., the contents of categories may change suddenly). Furthermore, despite the increased effort to bring properties to the attention of the users, we found that most users favored drag-and-drop and single-click actions over setting properties to browse and perform other functions. These results led us to adopt more one step actions, along with moving the role of properties to the background, in our next design.

4.3 Revised Design We took several issues into account when redesigning our low-fi prototype: 1) users do not ac-tively manipulate properties for organizational purposes; 2) users favor one step actions for browsing and other functions; and 3) users dislike system behavior to change dynamically unless explicitly specified. Figure 3 is one of the views in our revised prototype – one that combines the organization, searching, browsing, and manipulation of notes. On the far left of the diagram is a vertical list of notebooks. These notebooks are created by users and can be made sharable be-

Figure 3. Revised paper prototype

8

9

tween two or more individuals by specifying user names as a notebook property. A notebook is created whenever a user performs a search on notes using properties (e.g., all notes taken on Mondays from 3:30-5:00pm at 405 Soda Hall). Notebooks are not dynamic – they contain only what was returned by the query at the time of its execution. However, users have the option to update these notebooks with the click of a button. To the right of the notebooks is a list of notes returned by the query in thumbnail format, and to its top a history of notebooks representing the most recent selections. These thumbnails allow users to browse through notes quickly, while the notebooks provide quick reference to past queries. Finally, the right side of the screen displays a large screenshot of the actual note, with tools and operations to manipulate and edit the note. To find notes, a user first clicks on a notebook, which then opens up and reveals a list of note properties that the user can use to search on. These property values can be changed. Clicking on the search button performs the query and returns both a new notebook and a list of notes match-ing that query in thumbnail view. The user can then browse through this list of thumbnails to find the desired note, or continue searching using different property values. This prototype served as the basis for the new NotePals web-based interface. Although many in-teractions have been changed due to the technological constraints of using a web-based platform, the majority of the fundamental note sharing and usage ideas remained the same. In the next sec-tion, we examine the new NotePals application in detail.

5 NotePals User Interface The new NotePals application contains a variety of features designed to facilitate the manage-ment, search, and other operations on shared notes through the web-based client. These features are divided into the following sections: Browse, Search, Management, Upload, Community, and Session Tracking. Each of these major functions are accessed via the main navigation bar at the top of the window (see Figure 4).

10

5.1 Browse The browse section of the NotePals application allows users to find notes by specifying the de-sired note properties and values. It is divided into five different interface areas: property list, property values, criteria list, note search results, and note action controls. The list of available note properties is shown at the top left portion of the screen. As the user selects a property, the associated view for specifying that property’s values is shown in the bottom left window (e.g., the Type values in Figure 4). The list of currently selected properties and values is displayed as a criteria list in the top right portion of the window, with each criterion in the form of a hyperlink that serves as a shortcut back to the value selection view associated with that property. The set of notes matching the current criteria, or filtered notes, is shown in the right center portion of the window, and is updated automatically with every change in property or value selection. The user can navigate through multiple pages of the note set by using the previous and next page controls. Clicking on the thumbnail of a note brings up a full-sized note image. At the bottom of the screen is a list of available actions that can be taken on notes, such as adding notes to a collection, modifying note properties, emailing notes to other users, etc.

Figure 4. NotePals Browse Screen

Figure 4. NotePals Browse screen

The rest of this section is divided as follows. We first discuss property selection and the base set of properties available for browsing notes, accompanied by an examination of the different views available for modifying property values. We then briefly discuss how the criteria list works, de-scribe the functionality associated with looking through filtered notes, and conclude with the dif-ferent actions available for manipulating notes and collections.

5.1.1 Properties in the NotePals System The property pane displays the list of properties available to the user for finding specific notes (see Figure 5). By default, the NotePals system provides a set of predefined properties that is available for every note and whose values are automatically determined when the note is up-loaded into the system. These are the core properties. The user can also create customized prop-erties for organizing and searching notes (discussed in section 5.3.4). The set of core properties available are: Collection, Author, Group, Type, Creation Date, Upload Date, Recognized Text, and Keywords.

5.1.1.1 Collections Collections are named containevidual users of groups of userscompatibility for users who preside folders and hierarchies. Thebe placed in a hierarchy. Howevand collections. For example, evlections themselves do not posseven when Collection A is the plection A. This is a result of hcussed in more detail along witspite this fact, however, the clie(i.e., selecting Collection A willtions) so that the behavior is conchical organization systems.

Figure 5. NotePals core property list

11

rs for holding references to notes. They can be shared with indi-. Collections also serve as the means of providing backwards

fer the traditional organization scheme of placing documents in-y are similar to folders in that they are named containers and can er, there are also many fundamental differences between folders en though collections can be organized into a hierarchy, the col-ess the containment relationship with each other. For example, arent of Collection B, Collection B is not an item inside of Col-

ow collections are implemented in the system, and will be dis-h the description of the NotePals Architecture in Section 6. De-nt interface has been designed to give the illusion of containment show all the notes inside it and also those inside its child collec-sistent with what the user is accustomed to in traditional hierar-

One of the key consequences of this non-containment relationship between collections is that notes within the NotePals system can easily exist in multiple collections at the same time, not as copies of each other, but as references to a single note. Collections serve only as pointers to notes, and removing a note from its containing collection simply removes this relationship, while the note continues to exist within the system and may be referenced through other collections. This behavior allows users to always refer to the same note even notes that appear in multiple collec-tions, and allows changes in notes to automatically propagate to other collections in a shared en-vironment. Another important difference between collections and traditional folders is that notes do not have to be placed inside collections at all for them to be accessible to the user. Unlike traditional folder-based systems where folders are the only way for users to refer to documents, in a prop-erty-based organization scheme all documents exist in a document space, where users retrieve desired documents by referring to their associated properties and values. Given this fact, collec-tions in the NotePals system only exist as an optional note property. By default, users in the NotePals system have access to only their own notes. To make note shar-ing easier, the system provides shared collections and private/public collection spaces as ways to share notes between individuals and groups within the system. 5.1.1.1.1 Shared Collections and Independent Collection Views

Note sharing in the NotePals system is done through shared collections. When a user first creates a collection, that collection and its contents are private to the user by default (unless the collec-tion was created in the public collection space, to be discussed later). To share a collection, users can go to the management section, find the collection they wish to share, and add one or more users to the collection’s share list (see Figure 6). Adding users to this list does not make this shared collection automatically appear in the other users’ collection hierarchy. Instead, users on the shared list must first find the shared collections by specifying the original author’s NotePals ID or a collection name. Once the desired collection is found, the user can then save an alias to the shared collection in his or her own collection hierarchy. Making the user go through the process of finding and saving shared collection has two advantages: 1) users of shared collec-tions have control over which collection they want to have access to (as opposed to having the collection appearing automatically in their hierarchy, polluting their name space) and 2) allowing users to save an alias to shared collections in their own collection hierarchy allows them to have independent collection views while maintaining the sharing property.

For users to share the same set ofaccess to the same set of folderdrawback since users often have

Figure 6. Collections share list

12

documents in traditional folder-based systems, they must have s with the same hierarchy and folder names. This is a major their own document organization scheme. For example, one

13

user may place his conference notes for a talk on Monday on a particular subject with a particu-lar speaker in the format [conference name] > [date of talk] > [speaker] > [subject]1. However, another user who attended the same conference and talks and wishes to share her notes may or-ganize them in the format [conference name] > [speaker] > [date of talk] > [subject]. Using tradi-tional systems, they will not be able to share their notes unless they agree beforehand on a par-ticular folder hierarchy format and put their notes in the right places. This is not a problem in NotePals. Following the same scenario, one user (User A) can share their collection with the other (User B), and vice-versa. Once this is done, each user can then find the shared collection, and save an alias to it in their respective hierarchies (possibly naming it something other than the original collection). They will then automatically have access to each other’s notes despite their different locations in the hierarchy. What’s more, any changes in the contents of these shared collections are automatically reflected in both users’ views. These col-lection aliases are similar to folder aliases in Microsoft Windows and symbolic links in Unix op-erating systems. Despite the advantages of shared collections through independent collection views, to support cases where a fixed shared hierarchy is desirable, the system also supports shared collection hi-erarchies in the form of public and private collection spaces. 5.1.1.1.2 Public and Private Collection Spaces

When a user first becomes a NotePals member, he or she automati-cally receives a private collection space for managing his or her own collections. Any collections created in this space are private to the user by default, and can only be shared through the method described above. This area is meant to store collections that the user either does not wish to share with others, or only wishes to share with a select few (by sharing one collection at a time). In contrast, all collections created in the public collection space are accessible by other members of the system, with the visibility re-stricted by group membership. Users of NotePals have the ability to create and become members of groups, and with each group there is an associated collection space that is available only to its members. Users can use this space to share notes with other group members without going through the steps of sharing collections individually. Since collections in the public space are shared on a group level, shar-ing occurs automatically without user intervention as users become members of a group. Likewise, when a user is no longer a member of a group, his or her access to the group-shared notes will be removed as well, without any action from the user. To find notes that reside in collections, the user first selects either My Collections or Public Collections in the Properties list. Selecting My

1 “>” indicates a parent-child relationship Figure 7. Finding notes

through collections

14

Collections will display the user’s private collection hierarchy, while selecting Public Collec-tions will display the user’s public collection space, indexed by group names at the top level (see Figure 7). Clicking on a closed collection (indicated by the right arrow) expands the collection to expose any sub-collection in the hierarchy. Likewise, clicking on an open collection (indicated by a down arrow) will collapse that piece of collection hierarchy. Selecting the checkbox next to a collection name will add the corresponding collection and all of its child collections (if any) to the criteria list. The notes within these collections will then be used as the base note set for ap-plying other property criteria. The set of notes that belong to these collections and satisfy any additional criteria will then appear in the search result panel.

5.1.1.2 Authors and Author Lists Every NotePals note is automatically tagged with its author when it is uploaded into the system. This author identification is a unique user ID that is used to determine both note ownership and the list of fea-tures available to the user. Users can filter desired notes by author through the use of author lists. Author lists are collections of authors grouped under a named entity (see Figure 8). One sample author list might be a list of conference paper authors that can be used to restrict the particular author a NotePals user is interested in when browsing through notes. Author lists have a containment relationship with its authors in that selecting an author list will automatically include all authors in the list in the current search criteria. Users can also select individual authors under an author list. Users can create and manage author lists through the management section of NotePals.

5.1.1.3 Groups Groups are similar to author lists in that they are collections of user names under a named entity. However, these lists are generated automatically and are determined by group membership. By default, all NotePals users are members of the All NotePals Members group. Users can use this relationship to create collections that are globally available to all NotePals members, and also to find notes that are available to anyone in the system. Users can filter notes using the group prop-erty by either selecting one or more group names and viewing notes available to its members, or by picking individual group members whose notes the user is interested in.

5.1.1.4 Types Currently, the NotePals system supports the following note types: action item, new meeting, next meeting, note, CrossPad, CrossPad XP, and slide. Each note type has varying attributes such as the size of the note and any associated values (e.g., time of next meeting). The note type is deter-mined during upload, and is used to control how the note is processed. Users can select one or more note types to restrict the type of note desired in the results. Many of these types are artifacts of the original Palm pilot NotePals.

Figure 8. An Author List

15

5.1.1.5 Creation and Upload Dates Every NotePals note is associated with two date values by default: the creation date and the upload date. The Creation date is determined when the note was first created, and its exact time may vary depending on the client the note was taken on. For example, the CrossPad client sets the creation date to when the note was first created, while the Palm client uses the time when the note was actually written on. Regardless of the method, this value is determined during upload and is stored as a core note property. In addition, the system automatically timestamps each note with the time that the note was uploaded into the system. The upload time allows users to interleave notes from different users in the note search results by time. The system supports browsing notes by date using both specific dates and date ranges (see Figure 9). Users can control the date selection using the From and To check-boxes for range selection, and the month, day, and year dropdowns to set specific dates. Each date has an associated Today button that automatically sets the field to correspond to today’s date. Unlike the other property value controls, users must manually update the date criteria by clicking on the Update Criteria button. This is to prevent pre-mature querying, as most date selections usually include setting more than one field.

5.1.1.6 Recognized Text and Keywords Finally, the system attempts to extract text and keywords from the digital ink for every note that is uploaded to the system. The system sends the strokes from the note’s digital ink into a hand-writing recognizer, which then returns a list of recognized text and keywords for that note. This information is then associated with the uploaded note as a property, and is used in retrieving notes using the system’s keyword search functionality.

5.1.2 Criteria List The criteria list displays the currently selected properties and values that are used in retrieving the desired note set (see Figure 10). Each property in the list is shown as a hyperlink that, when clicked on, will automatically re-select that property and display the associated view for modify-ing its value. This list has the purpose of both providing additional feedback to the user on the current criteria, and also as a quick shortcut for jumping back to a particular property. In the fu-ture version of NotePals, the system will display a property’s values when the user places the mouse over that property.

Figure 9. Finding Notes using Dates

5.1.3 LThe sys(see Figshowinguser cannew winat one tuser to aous pagnotes byall the fselected

5.1.4 NThe Nottions, prnote(s) fnote(s) ption by f

ooking through Fitem returns the set of noteure 11). The thumbnail im all the details, and are a s view a note and its detailsdow. Since the number of

ime, the system divides thdjust this number using ae controls for navigating t clicking on the corresponiltered notes. Once selecte notes.

ote Actions ePals system provides a lioperties, and other attriburom a collection, moving roperties, emailing selectirst selecting the desired n

Figure 10. Sample Criteria List

ltered Notes s that satisfy the search criteria as a list of thumbnail images ages provide the user with a quick view of each note without

pace-efficient solution for displaying a large amount of data. A by clicking on the thumbnail, with the full note appearing in a notes matching the specified criteria may be too many to view e result set into groups of six, thirty, or sixty, and allows the

drop down list. The system also provides next page and previ-hrough the note groups. Finally, users can select one or more ding checkbox, or clicking on the Select All checkbox to select d, the user can then perform one or more note actions on the

s

Figure 11. Navigating through filtered note

16

st of functions for managing notes and their associated collec-tes. These actions are: adding note(s) to a collection, removing note(s) to a collection, deleting one or more note(s), modifying ed note(s), and annotating note(s). The user performs each ac-ote set and then pressing the associated function button.

17

5.1.4.1 Adding Note(s) to a Collection Pressing the Add to collection button allows the user to add the selected notes to a new or exist-ing collection. When the user clicks on this button, a new window appears with controls for navigating the public and private collection hierarchy, as well as one for creating new collections. When the user selects the desired target collection and clicks OK, the system will “copy” the se-lected notes to the selected collection. As noted earlier, each collection contains references to the actual note, and not the note itself. Therefore, during this process only references to the selected notes are created, and no new notes are created in the system. The system then refreshes both the collection hierarchy and note results screen, so that any notes that now satisfy the browse criteria as a result of this action will appear in the new result set.

5.1.4.2 Removing Note(s) from a Collection Pressing the Remove from collection button removes the selected notes from their respective col-lections, without deleting them from the system. As stated earlier, since collections only contain references to collections, removing notes from collections simply removes the references be-tween them and the collections they reside in. The notes themselves continue to exist in the sys-tem, and may be referenced by looking in other collections or found using other criteria. The sys-tem determines each note’s respective collection automatically as part of the query process, and it shows the user a confirmation screen before continuing with this action. Once the action has been performed, the system will refresh the result set to reflect the changes.

5.1.4.3 Moving Note(s) to a Collection Pressing the Move to collection button first adds the selected notes to the specified collection, and then removes them from their original collections. This action is convenient for moving notes between different collections without having to copy and remove. The selection interface for performing this action is identical to that of adding notes to a collection, with the additional effect of moving the selected notes as well in the process.

5.1.4.4 Deleting Note(s) Pressing the Delete button permanently deletes the selected notes from the system, and removes all references to these notes in any collection(s) they may reside in. This action is used for com-pletely removing a note from the system so that it cannot be found using any criteria. Since per-forming this function has serious consequences, each NotePals user can only delete their own notes, even for shared notes that they have access to. The system will prompt the user for con-firmation before performing this action.

5.1.4.5 Modifying Note(s) Properties Pressing the Properties button allows the user to modify the properties of the note(s) and their associated values. For a given set of notes, the system will determine the subset of properties that is present in all the selected notes. Aside from the core properties, the system will only display the set of custom properties that exists in all the notes. Furthermore, setting a property value will affect all the selected notes. In addition, the user can add additional properties and values to the selected notes through this function. The interface is a pop up window (see Figure 12) with the list of common properties on the left, the associated property values controls on the right, and buttons for adding more properties and closing the window on the bottom. Each note is updated

with the associated property and value as soon as a change is made, and the user can finish this action by clicking on the Cancel button to return to the browse screen.

5.1.4.6 Emailing selected NThis function allows the user towho may or may not be NotePaa dialog box appears promptingters the email address and clickspecialized note viewer. Whennote viewer, which includes a larea for the full note image on these notes through the standarweeks, upon which time the usea NotePals member. The recipiemail. This auto-expire feature minimize any risk that may be time.

Figure 12. Modifying Note Properties

18

ote(s) temporarily share selected notes with one or more individuals,

ls users. When the user clicks on the Email selected notes button, the user to enter one or more email addresses. After the user en-s Send, the system sends an email to each address with a link to a a recipient clicks on that link, he or she will be brought to the ist of shared notes as thumbnails images on the left, and a large the right (see Figure 13). Since these recipients are not accessing d NotePals client, their access will automatically expire in two r will either have resend these notes or the recipient can become ent is informed of this expiration date along with the link in the is to ensure that access to these notes is only temporary, and to incurred by accidentally sharing notes for an extended period of

5.1.4.7 Annotating Note(sFinally, pressing the Annotateand annotation (see Figure 14and are available to anyone posted, and are associated by a

5.2 Search The NotePals search functionone or more keywords or phrbrings up the search screen (ssearch input and options on

Figure 13. NotePals Shared Notes Viewer

19

) button allows users to annotate the selected note with a subject

). Annotations are automatically associated with the selected notes with access to these notes. Annotations cannot be removed once uthor and automatically time-stamped during creation.

ality allows users to quickly find notes or collections that match ases. Clicking on the Keyword Search in the main navigation bar ee Figure 15). The search screen is divided into four regions: the

top, the collections that match the search criteria on the left, the

matching note set on the middle right, and the same note action controls as discussed in the browse section on the bottom right portion. We discuss each section in more detail below.

Figure 14. Annotating notes in NotePals

n

Figure 15. NotePals Search Scree

20

21

5.2.1 Search Input and Options The top portion of the search window allows users to enter one or more keywords as search crite-ria, and also specify one or more additional search options. To find a particular collection or note, the user first enters one or more keywords into the textbox, followed by choosing the desired op-tion in the left dropdown list box (Collections or Notes). The system uses sub-string matching on all the specified keywords. The user can also control how the system interprets the list of key-words in the search: using all the words, any of the words, or an exact phrase match. Depending on whether the user selected Collections or Notes, the system with either return a list of collec-tions whose names either match or partially match the specified criteria, or a list of notes that contain one or more of the specified keywords in the criteria. The collection results is shown in the bottom left portion inside the Collection Results window.

5.2.2 Collection Results Each collection that matches the specified criteria is shown as a hyperlink with the text format collection name (in collection path). The collection path details the order of collections from the root collection to the specified collection, and provides additional context for helping the user remember the collection contents, and also to resolve any ambiguities that may arise from collec-tions with the same name. When the user clicks on a collection hyperlink, the notes inside the specified collection will appear in the search results window.

Figure 16. Sample collection results from a search for “test”

5.2.3 Search Results The right hand portion of the search window displays the sets of notes that are either contained inside a collection matching the search criteria or they themselves contain the keywords that the user is searching for. If the user has specified to search for notes, then the collections the result-ing notes appear in will be shown above each note in the result set. The format of the notes and the associated action controls are identical to that in the Browse section, and will not be dis-cussed here in more detail. Users can act on the notes matching the search criteria in exactly same manner as those found by browsing. Having discussed how users can find and manipulate notes and their associated properties and attributes, we will now discuss how users can manage the many entities that exist inside the NotePals system.

5.3 Management The management section of the NotePals client application allows users to modify collection structures, author lists, user groups, note properties, and account information. Clicking on Man-age in the main navigation bar brings up the management screen (see Figure 17). Users use a dropdown list to choose the object they want to manage. We will now discuss the functionality for each object in detail.

5.3.1 ColleThe collectioncollections. Tlated to one or

ctions s management vie

hese actions can be more collections i

Figure 17. Saving Shared Collections

22

w allows users to perform a variety of actions associated with divided into two types: general collection actions and those re-n the collection hierarchy.

5.3.1.1 General Collection Action The only general collection action allows users to find collections shared by other users and save aliases to them. An aliased collection will contain the same notes as the original shared collection, but it can have a different name and can be placed anywhere in the user’s collection space. This is the concept of shared collections through Independent Collection Views discussed earlier in the Collections section. Aliased collections allow users to share notes without regards to how the aliased collection is organized in each user’s hierarchy. To perform this action, the user first clicks on the Find Shared Collection link, which brings up a corresponding view for finding shared collections. This view contains two text boxes, one for entering the NotePals ID of the user who is sharing the collections, and another for entering a collection name. Giving one or both of these criteria, the system will return the collections that are shared with the current user that match those criteria. The shared collections are shown in a table format, listed by the NotePals ID of the shared collection owner, the name of the collection, and the complete collection path (see Figure 17). To save an alias to the shared collection, the user can click on the Add hyperlink next to the desired collection, which then brings up a dialog box containing the user’s private and public collection spaces. The user can use this dialog box to create new collections and navigate the collection hierarchies. When the user has found the collection under which to place the alias to the shared collection, he or she can enter the name of the alias in the Alias textbox and click OK. Doing so will create a new collection that is an alias to the shared collection.

5.3.1.2 Collection-specific Actions The other actions associated with managing collections all require the user to select a collection to act on as part of the management process. For example, the Add Collection functionality al-lows users to add new collections to the collection hierarchy. The new collection will be added under the last collection that was selected in the collection hierarchy. When the user clicks on the Add Collection hyperlink, the system displays a dialog box prompting the user for the name of the collection to create (see Figure 18). If the user enters a collection name that does not already exist under the selected collection, the system will create a new collection with the specified name under the selected collection. Once a collection has been created, the user can perform more collection-centric actions such as Add to Share List, Move Collection, Rename Collection, and Delete Collection.

Figure X. Saving Shared Collections

s

Figure 18. Adding a Collection and Collection-specific Action

23

To perform the actions specified above, the user must first select a collection from the collection hierarchy. Once he or she has done so, the right side of the screen will display information about the collection, such as collection name, collection actions, and a list of NotePals members with whom this collection is being shared with. To add additional members to the share list, the user first clicks on the Add to Share List hyperlink, which brings her to an add NotePals Members view. This view contains a box for the list of NotePals members to add to the share list, controls for modifying this list, and search functionality for finding NotePals users. When the user has finished modifying this list, she can click on the Update share list button, which will add the listed members to the collection share list. To remove members from this list, the user simply clicks on the delete hyperlink next to each member entry in the share list, and that user will no longer have access to the shared collection (see Figure 19).

Another collection-rMove Collection hyptrols for creating newspace (see Figure 20nal selected collectiotwo varying actions moving to. When a user moves apublic), or from the pall those beneath it inated notes in each coers of these collectioall shared relationshiusers would expect fr However, when a usethe following actionsit will be made at thated by the user willcreated by the user wthe user, and not to public group space inotes within them; 2erarchy, such that thmain accessible to th“move,” the user wo

Figure 18. Adding a Collection and Collection-specific Actions

elateerlin co

). Thn unmay

colriva thellectns wps wom

r mo wille new be rill r

the gs sh) whey are otuld e

Figure 19. Adding Users to a Collection’s Share List

24

d action is the Move Collection action. When the user clicks on the k, a dialog box appears with the collection hierarchy, as well as con-llections and moving between the user’s private and public collection is dialog box prompts the user to choose a collection to move the origi-der, and when the user has selected the target collection and clicks OK, occur, depending on which collection space the selected collection is

lection to the same collection space (i.e., private to private, or public to te to the public space, the following actions occur: 1) the collection and hierarchy will be moved to the new collection, along with their associ-ion; 2) if the move goes from a private to a public collection, the own-ill change accordingly (i.e., becoming public in the public space); and 3) ill be maintained after this move. These effects are the same as what

traditional folder-based file system.

ves a collection from the public group space into her own private space, take place: 1) a copy of the selected collection and all those underneath location, as well as copies of the notes within them; 2) all notes cre-

emoved from the original location in the public space, while notes not emain where they are; 3) the new collections will only be accessible to roup. The reason for these actions are as follows: 1) collections in the

ared by all group members, and each group member has access to the en a user decides to move public collections into her own collection hi-e no longer shared with the group, the collections and notes should re-her group members; 3) since the specified action being carried out is a xpect that collections and notes will be transferred to the new location.

25

Therefore, notes that are owned by the user will be moved to the collections in the private collec-tion space, and removed from the public area. Doing all of the above will ensure that other group members will continue to have access to public notes even after a move, and that no one will have the ability to remove access to shared collections from other users.

5.3.2 Author Lists As described earlier when discussing NotePals properties, author lists are named entities of groups of authors that are used to specify the desired author or authors when browsing NotePals notes. To create an author list, the user first selects manage Author List, and then clicks on the Add List hyperlink, after which a dialog box appears asking for the list name. If the user enters a name that does not already exist, a new author list with the specified name will be created and shown in the author list area. When the user select a particular author list, the system displays information about the list, such as the list name, the detailed user information of each author within this list, and hyperlinks to add author to the list, rename the list, and delete the list (see Figure 21). The interface and steps

Figure 20. Moving a Collection

for adding NotePals members to the author list are identical with that of adding members to a collection’s share list. Clicking on the Rename List hyperlink displays a prompt for a new list name, and clicking on the Delete List hyperlink will delete the author list from the system after confirmation from the user. Author lists are private to each NotePals member and are used to simplify the process of filtering notes by author.

5.3.3 Groups NotePals groups represent groupsapplication. One such resource is tmessages that are visible to other NotePals group will have access t

Figure 21. Sample Author List

of NotePals users that have access to shared resources in the he Community bulletin board, where group members can post

members of the same group. In addition, members of the same o a shared collection hierarchy. Furthermore, similar to author

Figure 22. Creating a NotePals User Group

26

27

lists, users can use group to restrict the note authors they are interested in while searching for notes. Any NotePals can create a group, and a user can join a group via an email invitation from its creator. Let’s outline the group creation process below. To manage groups, a NotePals user first selects Manage Groups from the dropdown menu under the Manage screen. NotePals members can then create user groups by clicking on the Create Group hyperlink. Doing so will bring up a view for entering group name, an option to choose whether any member can invite others to join this group, and controls for creating a list of invitee email addresses (see Figure 22). The steps for modifying this list are the same as that described earlier. When the user clicks the Create Group button, the following actions will occur: 1) the system creates a group with the specified name, setting the user as its administrator; 2) a new public collection space will be created with the given group name as its top-level collection. This collection hierarchy will be available only to members of the group; 3) the system sends an email invitation to each address specified by the user, with a hyperlink the recipient can follow to join this group.

Figure 23. Joining a NotePals User Group

When the recipient clicks on this link, he or she is brought to a page prompting him for his Note-Pals login and password (see Figure 23). If he is not already a NotePals member, he can follow a separate link that brings him to the new account page, after which he can come back, log in, and join this group. If this user is already a NotePals member, he can simply enter his login informa-tion, after which he will become a member of this group. By default, all NotePals users are members of the All NotePals Members group. Users can use this relationship to post messages to all members in the Community area (discussed later), or share collections in the public group col-lection space.

5.3.4 Properties The user can create custom properties for organizing and managing notes by first selecting Man-age Properties and then clicking on the New Property hyperlink. To create a new property, the user needs to specify the property name, select the type of property (tag or multi-valued), and possibly enter values if it is the latter (see Figure 24). Tag properties are those without values and are used to label notes. For example, the user could create an important property to associate with notes that are considered important. Multi-valued properties are also properties used to label notes, but whose values reflect a particular aspect of the note it is attached to. For example, a student can create a multi-valued property named class with different class names as its values. He or she can then attach this property to her class notes, labeling each with the appropriate value. Having done so, he can then find the desired class notes later by specifying the class prop-erty with the appropriate class values. Using properties to organize notes allows more flexibility compared to traditional folder-based fixed methods, and makes sharing notes easier, since a note’s organization scheme is not fixed to its location.

Figure 24. Creating a Location Property

28

29

5.3.5 Account Every NotePals member has a user account with information related to the user, such as name, email, password, and so on. Users can change this information by selecting the Manage Account option from the Manage screen. The system will then display a HTML form that allows users to update and submit the updated information. NotePals uses a user’s account information for emailing notifications, announcements, etc.

5.4 Upload NotePals supports uploading notes from two platforms: Palm Pilot and IBM CrossPad. Palm Pi-lot users can upload notes to the NotePals server by first installing the NotePals Palm application and then synchronizing the application with the desktop. The notes will then be placed in collec-tions in the same hierarchical layout as the Palm application. IBM CrossPad users use the client upload manager to upload notes to the NotePals server. It is a Java-based application installed locally on the user’s computer, and allows users to either upload notes directly into the system, or decide where to place the notes in collections before uploading them. Once the uploading cli-ent has been installed, it is integrated with IBM’s Ink Manager to automatically start whenever notes are uploaded to the user’s computer. Let’s outline the steps involved in uploading notes to the system.

5.4.1 Authentication The first screen the user sees is a user authentication screen (see Figure 25). The username and password for the uploading client is the same as that of the web-based notes browser. During au-thentication the user has the option of launching the upload manager. If he or she chooses not to do so, the system will place all uploaded notes in the user’s private My Notes collection by de-fault, and the user can then organize her notes online via the web application. The user can des-ignate the type of CrossPad he or she is using before uploading notes to the server. The NotePals server then uses this information to render the note images in the correct size.

However, if the user chooses to launch the upload manager, the system will start the entire client application to let the user organize her notes before uploading.

Figure 25. NotePals Upload Authentication

5.4.2 Upload Manager The client upload manager allows users to designate which collection to place one or more of the uploaded notes into before they are sent to the NotePals server (see Figure 26). When the user chooses this option, the system downloads the user’s latest collection hierarchy from the Note-Pals server as soon as the user has been authenticated. The users can manipulate the collection hierarchy in a tree view on the left side of the application, and the changes will not be made until the user uploads the notes into the system. On the right side of the application users can preview and scroll through the set of notes and selectively upload one or more notes into the desired col-lections. The NotePals server places the uploaded notes into a queue and processes each note sequentially. The system allocates a separate job process to handle each upload, and in the event that the up-load process fails while processing that note (e.g., an error occurred during handwriting recogni-tion), that specific upload process is aborted and the system will continue processing the next upload job in queue. The user can monitor the upload process through the Upload Status section of the NotePals web application.

Figure 26. NotePals Upload Manager

30

31

5.5 Community The community bulletin board allows users to post text-based messages online that are available to other group members. This feature is designed to enhance the peer-to-peer sharing experience by allowing users to exchange messages in addition to sharing notes. Users can only post mes-sages to groups that they are a member off. To post a message to group, the user first selects Community from the navigation bar, and then clicks on the desired group to post the message to. Once a group is selected, the system displays a list of subjects of the existing messages as hyperlinks, starting with the latest message. To read the contents of a message, the user clicks on the subject hyperlink and is shown its entire content (see Figure 27). To post a message, the user first clicks on the Post hyperlink, then enters a subject heading and message content, clicks Post Message and the message will be posted to the specified group. Messages cannot be deleted and are meant as public announcements to a par-ticular group.

5.6 Session Tracking and Security NotePals uses session tracking to identify the user, customize the online content that is relevant to the user, and to enforce restrictions to shared content. A session is defined as a server-side en-tity that stores any user-specific information between the time when the user first logs into the system, until the time when he or she either leaves the system by manually logging out, or when the session expires after a pre-determined time. Session tracking is implemented as part of the Java Servlet interface (discussed in more detail in Architecture and Implementation) and exists only in the server. When a user first logs into NotePals, the system creates and associates with the user a unique session that is used to store the user’s identification information and any related user-interface state variables. NotePals uses the user’s identification to restrict access to publicly shared content, such as notes, collections, groups, and community messages. Since a user’s identification is maintained on the server-side, as opposed to encoding it in the Universal Resource Locator (URL) or HTML code, users are prevented from spoofing the system and gaining access to re-stricted content (e.g., by changes the user id in the URL).

Figure 27. Sample Community Message

32

As the user navigates and explores the different features in the NotePals application, all user-state information is stored in the user’s session to provide persistent content throughout the user’s visit to the web site. This state information includes: 1) any hierarchy-based entities from which the user can select or de-select (collections, author lists, groups, properties, etc.), and 2) selected criteria and preferences while browsing/searching notes (e.g., desired creation date and number of note per result page). NotePals would not have been possible without the use of ses-sion tracking, since Hypertext Markup Language (HTML) pages are stateless, and passing state information between client page requests would not be a feasible solution. To see how sessions and other NotePals features work in general, we will now discuss the Note-Pals architecture, followed by several examples on how each component is involved in deliver-ing the NotePals features.

6 NotePals Architecture The NotePals system has a three-tiered architecture made up of following components: the cli-ent-side browser that displays the content to the user, the intermediate Java Servlets that provide application logic and deliver the user content to the browser, and a database backend that main-tains all user, system, and note information. Before looking at each of these components in detail, we will first discuss the decisions made when choosing these components, as well as some of the different tradeoffs involved. Following this discussion, we will then examine the structure and functionality of each component, and then explore their roles and interactions in three of Note-Pals’ key features: navigating hierarchies, querying for notes, and general system management through the web interface.

6.1 Architectural Design and Tradeoffs The main factors we considered when choosing how to implement the system were: accessibility, performance, and ease of deployment. One of the main goals of NotePals is to simplify the note sharing process and to make shared notes easily assessable from any location. With this in mind, the web seemed liked the ideal platform to develop this application on top of with its pervasive accessibility. However, developing for the web has many tradeoffs compared to developing for the standard desktop environment. One of the biggest limitations of web applications is that their functionality is limited to what is possible using the standard HTML components and JavaScript scripting. Although it is possible to port applications directly to the web by using Sun’s Java Applets, this too, has many limitations. First of all, current browsers only support up to Java language version 1.1 by default, so any application written using constructs above this version will not run prop-erly. Though users can work around this problem by downloading a separate plug-in to enable the latest Java Virtual Machine, this will limit accessibility since it requires the user to have some knowledge of downloading and installing software, and it also requires that the user have the permission to install the plug-in on the machine. Furthermore, once the user moves to a dif-ferent machine that does not have the plug-in installed, he or she must install the software once again to be able to use the application. In addition, Java Applets have several performance con-siderations. For example, users must first download the application before they can use it. Users that access the application over a slow network connection will experience a significant lag time

33

whenever the client needs to download an application component. Furthermore, there is signifi-cant performance consideration for large application, especially for those that manipulate a lot of graphics objects. Therefore, to reduce the amount of work a user has to perform before she can use the application, we decided not to user Java Applets to develop our application. Furthermore, as stated earlier, web application features are limited to what is available using standard HTML objects for the UI components, and JavaScript for client-side interaction. Due to this fact, web applications tend to have a simpler design compared to their desktop counterparts. Even though HTML is very limiting, a developer can use client-side JavaScript to simulate many of the interactions a user would expect from a traditional desktop application. JavaScript is a cli-ent-side scripting language that lets you write and execute code in the client browser. In addition to supporting many standard constructs found in programming languages, such as variables, loops, and data structures, another useful aspect of JavaScript is its ability to manipulate HTML components. We will later see how JavaScript plays an essential role in making many of the fea-tures in NotePals possible. A traditional client-server desktop application tends to have much lower delay compared to its web-based counterpart. This is due to the fact that web-based applications must go through the network to determine how to respond to user input, whereas in desktop applications all of the decisions regarding the UI response can made locally on the client. For example, a typical client interaction in the NotePals application goes through the following steps: 1) the user clicks on an object that initiates a page request to the NotePals server; 2) the server gathers the input from the browser and decides the appropriate response; 3) the server contacts the database to retrieve any relevant information or perform specified actions; 4) the server processes the database informa-tion, and outputs the results to the browser in HTML format; 5) the client parses the incoming HTML code, possibly doing additional processing by executing JavaScript instructions. Com-pared this to a client-server application -- when a user performs an action that requires a change in the interface, the decision on how to update the interface is made locally on the client, possi-bly contacting a networked database to retrieve any relevant information. However, despite the obvious performance bottlenecks with a web-based application, we believed that we could over-come these problems by simplifying our web application and using client-side scripting to mini-mize network and server delays. For ease of deployment, the web development model has many advantages over traditional desk-top clients. For example, since web users always access the web application by connecting to a particular network address, the developer can easily deploy or upgrade an application by direct-ing the user to a new URL or by changing the code on the server. But with desktop clients, users must download and install the application every time there is an update, which requires the user to have some knowledge of the desktop environment, and also the necessary privileges for in-stalling the software. Therefore, the web development model seemed like the right choice when it comes to easy of deployment. To summarize, web applications offer the advantages of pervasive accessibility and ease of de-ployment, while their performance is limited by network speed and their functionality is limited by a reliance on HTML and JavaScript. Desktop applications generally run much faster than web applications and contain much more advanced functionality, but lack widespread accessibility

and requires more user intervention during installation and upgrades. Weighing these factors, we chose to develop for the web due to its high accessibility and ease of deployment. Now that we have discussed the reasons for our high level architectural choices, let’s take a look at each of the components in detail.

6.2 NotePals Components The NotePals architecture contains three main components: the client browser, server-side appli-cation logic, and the backend database (see Figure 28). Each component in turn contains many modules that perform specified tasks.

6.2.1 Client Browser The client browser delivers the apPals users to interact with the sysPals server, download the HTML provides the following user interfframes, images, lists (both dropdovarious combinations of these com In addition to rendering the intertional instructions that may have the form of specialized JavaScripFor example, the server may usehave been rendered. Some examvalue, assigning form attributes beHTML element to another, and paddition, the server can send backfore the form is submitted to the sduces the amount of network trafare written entirely using JavaScrlook at these in more details in lat

2 Currently, NotePals only supports Micdue to the fact that each browser supporported in the application are not compati

Client browser (GUI)

NotePals

Application Server

NotePals Database

network network

F e

igure 28. NotePals Architectur

34

plication content to the user, and is also the only way for Note-tem2. The browser’s basic functionality is to contact the Note-and JavaScript, and render the application accordingly. HTML ace components: buttons, text fields, text areas, forms, tables, wn and scrolling), and windows. The NotePals application uses ponents for its user interface.

face using HTML code, the browser also executes any addi-been downloaded from the server. These instructions come in t code that allows the server to do one of many different tasks. JavaScript code to manipulate HTML components once they ples of these tasks include setting text fields to a pre-defined fore the form is submitted to the server, copying data from one erforming operations between multiple windows or frames. In JavaScript code that verifies the correctness of form fields be-erver. Doing checks such as these on the client side greatly re-fic and delay that the user must face. Some NotePals features ipt (i.e., without using any HTML components), and we will

er sections.

rosoft’s Internet Explorer browser, and not Netscape’s Navigator. This is ts different HTML and JavaScript standards, and some of the features sup-ble in Netscape.

35

6.2.2 NotePals Server The NotePals Server generates the HTML and JavaScript code that controls the application inter-face and underlying interactions. It is built on top of the Tomcat Web Server [25], which sup-ports Java Servlets and the JavaServer Pages Technologies. The server code is divided into mod-ules called Servlets, each responsible for handling a particular type of request from the client. For a given request, a specific Servlet receives input from the client, performs operations based on the application logic, contact the database to retrieve any necessary information, and returns the appropriate HTML and/or JavaScript response to the browser. Each Servlet can use one or both of the following essential Java Servlet API components: session tracking and the database interface. Session tracking is a component built directly into the Java Servlet API and is used heavily by NotePals. Servlets store the user’s identity and any information related to the client’s state. The server uses the user information to restrict both the objects the user can see (e.g., notes, groups, collections, etc.) and also the functions he or she can perform (e.g., removing users from a group, sharing a collection with others, etc.). Client state information includes both visual information that is displayed to the user and any server-side data that should persist across different connec-tions to the server. For instance, the server maintains the collection hierarchy state across multiple requests to the server. Without session tracking, the user would not be able to traverse through the hierarchy, since current web browsers lose all of their display information with every server request. Another important component is the database module that allows Java Servlets access to the backend database. This component handles all transactions to and from the database, and allows a Servlet to execute two types of queries: those that return a result set, and those that act on the data but do not return any results. For queries that return results to the server, the database com-ponent provides methods for traversing through and accessing the resulting data. For methods that only perform operations on data but do not return a result set, the database component re-turns a status code to indicate whether the operation was successful. Queries can be generated dynamically from the server and passed to the database to execute, or stored directly inside the database as stored procedures. For the latter case, the Servlet simply invokes these procedures with the appropriate parameters and the database component then executes these procedures di-rectly on the database, resulting in better performance. Let’s now examine the database itself in more detail.

6.2.3 Database All of the NotePals data and relationship information is stored inside the backend database. Cur-rently, NotePals uses Microsoft’s SQL Server 7.0 for its database implementation. The NotePals server uses the Java database connectivity (JDBC) interface to connect to the database and to ac-cess and manipulate its tables, views, and stored procedures.

6.2.3.1 Tables All of the NotePals data is stored inside tables in the form of rows and columns. There are two types of tables in the NotePals database: those that contain pure data and those that contain ex-plicit relationships between data (i.e., those not already embedded in the data itself). The five major data tables are Users, Collections, Documents, Groups, and Properties.

36

The Users table contains a user’s account information, such as username, password, email ad-dress, name, and so on. It is used to both authenticate the user during login, and also to grant ac-cess privileges to shared content. The passwords are encrypted using Base64 encoding to provide a base-level security. We decided not to use a higher-level security scheme such as MD5 for en-cryption so that the system can still decode the passwords to send to users when they have for-gotten them. The Collections table stores information about individual collections and their relationship with each another. It contains information such as a collection’s unique id, its name, owner, parent, level in the hierarchy, path from the root, and target id (used for aliased collections). In addition, the system can find out information about a collection’s children, or collections at a certain level in the hierarchy, or all collections belonging to a given user through the use of SQL queries. The Documents table contains information about a document’s id, its owner, type, creation date, upload date, keywords, and recognized text. Each document id is a unique number that maps to a note stored on the hard drive. The NotePals server uses the document id to retrieve a note and all of its related information. The Groups table stores a group’s id, name, and its owner. Group membership is stored in a separate table, and can be referenced using a group’s unique id. Finally, the Properties table contains properties created by both the system and user. The core properties are the set of attributes that each document has by default and are set by the system during upload. User-defined properties are created by the user via the NotePals application and can be attached to notes at will. These five tables contain the majority of the static data inside the NotePals database. The other type of NotePals table contains the relationships between different NotePals entities (e.g., Documents, Collections, Users, Groups, etc.). The main relationship tables include Docu-mentsInCollections, PropertiesInDocuments, ShareCollectionGroup, ShareCollectionUser, TempDocAccess, UsersInAuthorGroups, and UsersInGroups. The DocumentsInCollections table maps notes to the collection(s) they belong to. A note can be-long to one or more collections, and all collections containing a given note reference the same note (i.e., they do not reference copies of the note, but rather the note itself). When the user re-moves a note from a collection, it is only this note-in-collection relationship that is removed, but not the note itself. The note continues to exist in other collections it belongs to as before. How-ever, when a user deletes a note, the note itself is removed from the Documents table, and all ref-erences of this note are removed from DocumentsInCollections accordingly. Since deleting a note could potentially remove it from another user’s shared collections, this functionality is only available to a note’s owner. PropertiesInDocuments stores the relationship between properties and the notes they belong to. A given property can belong to several notes, and can have a different value for each property-note pair. Each document contains both core properties and those defined by the user. Core prop-

37

erties are stored along with the note itself in the Documents table, while user-defined properties are stored in the Properties table. When a user performs a query on the set of notes matching a particular user property and value, the system uses this table to first find the set of notes contain-ing that property, and then filters those notes by the specified value. The complete details of the query process are outlined in the sections following the main database structures. ShareCollectionUser describes how a collection is shared among users in the system. Each user-collection pair in ShareCollectionUser specifies that a particular user has access to notes in a particular collection. Users can add or remove this privilege through the NotePals interface. By default, all users have implicit access to their own collections. Similarly, ShareCollectionGroup depicts how a collection is shared amongst members of a group. Each entry in this table contains a collection-group pair, corresponding to a particular collection in the Collections table and a group in the Groups table. The actual group membership is stored inside the UsersInGroups table, and not in ShareCollectionGroup itself. Separating the sharing and group membership information allows users to share collections with entire groups whose exact membership may not be determined beforehand. And once the collection-group relation-ship has been set up, each new group member will automatically gain access to the shared collec-tion without the collection owner explicitly sharing the collection. Likewise, a user can also re-move sharing privileges from a group without removing each user individually. TempDocAccess stores temporary notes access codes for users that are not NotePals members. When a NotePals member emails one or more documents to someone outside of the NotePals system, the system creates a temporary access code granting that person access to the selected notes for two weeks. That code is embedded in a URL that allows anyone to view the selected notes inside a specialized viewer. The receiver will not be able to access these notes once the ac-cess code expires. Finally, UserInAuthorGroups, similar to UserInGroups, outlines the list of users inside a particu-lar NotePals author list. As stated before, author lists are personalized lists of authors that mem-bers can create for filtering out notes by one or more desired authors. This table simply captures this relationship in the form of author-list and user id pairs. Basic information about the author list themselves, such as an author list’s name and owner, are stored in the AuthorGroups table. The above list outlines the major data and relationship tables in the NotePals database. However, in addition to these tables, relationships are also determined dynamically at runtime, possibly involving more than one table. The examples following this section will demonstrate this behav-ior in more detail.

6.2.3.2 Views Views are specialized entities in the SQL Server that represent query-based tables. A view does not contain rows or columns itself, but rather a SQL statement that describes what rows and col-umns it should contain. A view is a non-static entity in that its content is not determined until runtime when the view is actually accessed. In addition, views themselves can be used in queries and are treated as actual tables.

38

The set of Views in the NotePals database are: AllSharedDocuments, AllUserDocuments, Collec-tionList, and DocumentsWithCollection. AllSharedDocuments represents the set of notes that are available to all users in the system, whose rows and columns are derived from the data and rela-tionships in Documents, Collections, DocumentsInCollections, PropertiesInDocuments, ShareCollectionGroup, ShareCollectionUser, and UsersInGroups. AllUserDocuments is a subset of the entries in AllSharedDocuments where the user id is constrained to the current user. Collec-tionList represents the set of collections a particular user has access to, and is based on informa-tion from Collections, ShareCollectionUser, ShareCollectionGroup, and UsersInGroups. Docu-mentsWithCollection contains the union of both note and collection information and is sorted by notes and the collections they belong to. Generally, views are not used heavily in the system since the query they are based on is determined beforehand and they cannot accept incoming pa-rameters. However, stored procedures, as discussed below, circumvent this problem by allowing the server to pass in input parameters to pre-defined queries.

6.2.3.3 Stored Procedures Stored Procedures are executable functions inside the database that accept one or more parame-ters at runtime. Stored procedures can execute multiple SQL statements in sequence, and are ref-erenced by their names and parameter values. There are several advantages to using stored pro-cedures, namely: • Stored procedures are precompiled in the database, so they run faster than dynamic SQL. • Stored procedures execute entirely within the database, so they can perform multiple queries

and updates without network traffic. • Stored procedures provide an additional layer of abstraction between the underlying table

structure and applications using the database, such that any changes to the table layout re-quire changes only in the stored procedures that access them, and do not affect the applica-tion code.

NotePals uses the JDBC API to access stored procedures in the database. Before calling a stored procedure, the NotePals server sets the name of the procedure to call and the corresponding pro-cedure values. The server then executes the procedure, which takes place entirely inside the da-tabase. The database then returns the procedure results, if any, to the calling function. The Note-Pals server uses a large number of stored procedures to access and modify the data and relation-ships inside the NotePals database. We will now examine how the NotePals components interact with each other to carry out some of the common tasks in the application.

6.3 Component Interactions The typical path for displaying information in NotePals involves going through one or more of the components mentioned above. Since browsers themselves do not retain any information be-tween page requests, all of the client’s state information is stored on the NotePals server. And if the user accesses a page whose contents depends on data from the database, the NotePals server will then either use dynamic queries or stored procedures to access and retrieve the desired in-formation. In addition, complex state information is maintained across Servlets through the use of session objects. Let’s now examine in detail how these components interact with each other by looking at three of NotePals most common tasks: browsing a dynamically generated hierarchy, querying the database for desired notes, and uploading notes to the NotePals repository.

39

6.3.1 Browsing Hierarchies Many of the NotePals features involve some form of hierarchical representation, such as collec-tions, author lists, groups, and properties. Although each of these hierarchies presents different information and allows different user interactions, the steps they follow to retrieve and display that information are identical, with the only difference being how the information is retrieved from the database, and how the state information is maintained between page requests. We will now focus on the collections hierarchy, and see how the system displays the initial collection tree, followed by the interactions involved in expanding and collapsing a collection branch.

6.3.1.1 Displaying the Initial Hierarchy Even before the user begins interacting with the collections hierarchy, the system has already gone through many of its components in order to display the initial collections hierarchy. This series of actions begins with the user coming upon a page containing the hierarchy, which sends a page request to a Servlet designated to handle that request. The Servlet then performs a series of tasks before returning the collection hierarchy. First, it validates the user’s identity by check-ing whether there is a unique session associated with this user, which the system creates only af-ter a user has successfully logged in. If such a session does not exist, the Servlet then redirects the user to an HTML page where he or she has the option of logging into NotePals again. If the session does exist, the Servlet then checks whether a collection id has been passed into the Serv-let as part of the page request. This collection id indicates that the user has clicked on a particular collection in the hierarchy, and the Servlet should either expand or collapse the branch under that collection accordingly. Since this particular page request does not involve the user clicking on a collection, this collection id does not exist, and the Servlet continues onto the next step. Following the collection id check the Servlet then retrieves from the session the list of collection ids representing the currently selected collections in the hierarchy. The Servlet adds a collection id to the list every time the user expands a collection, and removes it from the list when the user collapses a collection branch. Using this list, the Servlet then constructs a dynamic SQL state-ment representing the set of collections that the user has access to and that are a child of one of the selected collections in the list. The Servlet then sends this query to the database. When the database receives the query from the Servlet, it executes that query and returns the hi-erarchy in the form of a result set. The following is a sample result set: Collectionid Parentid Userid Name Level Path Alias 20 0 21 Conference 1 Conference 20 21 20 21 UIST 2000 2 Conference:UIST 2000 21 22 21 21 Talks 3 Conference:UIST 2000:Talks 22 23 0 21 Courses 1 Courses 23 24 23 21 Astronomy 2 Courses:Astronomy 46 The collection id is a unique number associated with each collection in NotePals. The parent id is the collection id of a collection’s parent in the hierarchy. The user id is a unique number asso-ciated with the NotePals member who created the collection. Name represents the collection name, and Level is the collection’s depth from the root in the hierarchy. Path is the concatenation

40

of all the collection names from the root down to a particular collection, and Alias is used to rep-resent shared collections (e.g., Astronomy is an alias to collection 46). All collections at the top-most level have a parent id of zero. To find all collections belonging to a particular user at the topmost level, we simply query for collections with parent id of zero and that user’s user id. In addition, to find all collections that the user has selected so far, we simply query for all collec-tions whose parent id is in the selected list. Finally, to organize the collections according to their hierarchical form, we simply order the result set by the associated Path. For example, the above result set represents the following hierarchy: Conference

o UIST 2000 o Talks

Courses o Astronomy

After the Servlet receives the collection results from the database, it then traverses through each entry to output the HTML code that renders the corresponding hierarchy in the browser. Note-Pals uses JavaScript code for rendering this hierarchy. The first part of the code involves con-structing the collections hierarchy and loading it into memory. The second part involves manu-ally opening the specific collection nodes to display the desired hierarchy. For example, the above result set would contain six lines of setup code, one for the root collection, and one for each of the other collections. The Servlet would then add another line of code to expand the root, Conference, UIST2000, and Courses node in the hierarchy. The resulting hierarchy is then what we want to show to the user.

6.3.1.2 Expanding the Collections Hierarchy The user can begin interacting with the collections once the browser displays the initial hierarchy. When the user clicks on a collection, it causes the browser to send another request to the Servlet, along with the collection id of the selected collection. Upon receiving this request, the Servlet then follows the exact same steps as that for displaying the initial hierarchy, with the extra step of adding the selected collection id to the selected collection list in the session. The Servlet then uses that list to construct the corresponding SQL statement and follow the same steps as before.

6.3.1.3 Collapsing the Collections Hierarchy Collapsing the collections hierarchy is similar to that of expanding the hierarchy, with the excep-tion of a few extra steps. When the Servlet receives the request to collapse a particular branch, it first removes the selected collection from its collection list in the session. Since the Servlet does not retain any hierarchical information itself, it cannot remove any child collections under the collapsed branch from this list until it queries the database. This situation arises when the user has traversed several levels down a particular collection path, and decides to collapse the hierar-chy somewhere up the path (i.e., not at the leaf node). When this happens, the desired action is to remove the descendant collections from the selected collection, in addition to the selected collec-tion itself. After the Servlet has removed the selected collection from the session list, it then con-structs the query and executes it on the database as usual. However, when it traverses the result set, it filters out those collections that are part of the collapsed branch, and removes their corre-sponding collection ids from the collection list in the session. Doing so effectively prunes the

41

collapsed branch from the hierarchy, and keeps the selected collection list to a minimum to achieve better query performance. Author Lists, Groups, and Properties use the same exact procedure for rendering their hierar-chies, with the only difference being how the Servlet retrieves the data from the database and how it maintains the state information. Having looked at how dynamic hierarchies work, lets now look at how NotePals retrieves the desired notes that match the user’s specified criteria.

6.3.2 Browsing the Notes Repository NotePals allows its users to find notes by specifying one or more note properties and their asso-ciated values. After the user sets the criteria through the web interface, the corresponding Servlet that handles the note lookups then generates the matching query. The database then executes the query and returns the result to the Servlet, which then iterates through all the entries and gener-ates the HTML that displays the notes in the web browser.

6.3.2.1 Choosing Note Property and Values The NotePals web application allows users to choose from a list of properties associated with notes in the database. Initially, the system provides the user with a set of base properties that all notes must have. The user can also define customized properties with optional values and attach them to any notes he or she can see. To find notes with one or more properties and values, the user first selects the desired property from the property list. Doing so will cause NotePals to con-tact a Servlet that specializes in displaying the appropriate property navigation and value controls. If the selected property uses a hierarchy for displaying its values, the Servlet will display that hierarchy with the associated controls. Likewise, the Servlet will display dropdowns for date-related properties, and checkboxes for properties containing several values. As soon as the user selects a property value, the browser’s JavaScript code will send a page request to the Browse-Criteria Servlet, which updates the criteria list. In addition, the panel containing the property value selection controls will refresh, if necessary, to display any updated information.

6.3.2.2 Updating the Criteria List The BrowseCriteria Servlet maintains the list of currently selected properties and values. Since HTML pages do not retain any information between page requests, the criteria that the user has specified so far would normally be lost when the user moves to a different screen or chooses a different property to look at. When the BrowseCriteria Servlet receives updated criteria informa-tion, it first checks to see if there is a session associated with this particular user. Failure to find this session indicates that the user has either not logged in or that his or her session has expired, and the Servlet will output the HTML that redirects the user to a login page. If the associated session does exist, the Servlet then updates its own property-value entry for the selected property in the criteria list session object. The Servlet then outputs the updated criteria list in HTML for-mat, along with JavaScript code that causes the panel containing the matching notes to refresh.

6.3.2.3 Retrieving the Matching Notes The BrowseContent Servlet is responsible for querying the database and displaying the matching notes to the user. Unlike other standard techniques for submitting queries to the server, it does not use any query parameters encoded in the URL or embedded in the HTTP form submission. Instead, it retrieves all of the selected criteria from the shared session. By iterating through the

42

selected properties and their associated values, this Servlet then translates the criteria into a matching SQL statement. In some cases, the query may reference more than one table, or the Servlet may perform additional queries before arriving at the result. For example, for queries in-volving collections, the Servlet will reference the DocumentsInCollections table to determine which notes belong to which collections. In addition, the query may reference pre-defined views such as AllUserDocuments and AllSharedDocuments to include all the notes the user can see. For queries involving the system-defined properties, the query will reference the main Documents table where these values are stored. For queries containing user-defined properties, the Servlet will retrieve the results in two steps. It will first construct a query that returns the set of notes containing the specified properties and values. This is necessary since the user-defined properties are stored in a different table than the main Documents table, and it is not possible to query the desired notes in one operation. The Servlet then incorporates the returned note set into its stan-dard query involving the system-defined properties and collections, if specified. Once the Servlet has constructed the appropriate query, the database then executes it and returns the result set. The Servlet then iterates through the results and outputs the HTML code to display the notes in the desired format.

6.3.3 Uploading Notes to the NotePals Repository In addition to handling requests from the client browser for display information, the NotePals server also processes uploaded notes from the Palm Pilot or CrossPad clients before placing them into the NotePals repository. The upload process has the following parts: 1) receiving an up-loaded note and placing it on a job queue; 2) taking a job off the queue and creating a new data-base entry; and 3) generating note images files and performing handwriting recognition.

6.3.3.1 Receiving a Uploaded Note The UploadServer Servlet is a specialized process inside the NotePals server that accepts and processes uploaded notes. The UploadServer receives uploaded notes using the standard HTTP POST protocol, and packages each note into a new job entity. A job is a specialized Java object that knows how to process a particular type of note. Jobs are implemented as Java Threads, and contain a header and body. The header stores information such as the time when the note was received and the name of the machine it was uploaded from. The job body contains the raw data that was sent to the server. The UploadServer will create different types of jobs depending on the note type. Currently, only a PalmThread Job type is defined that handles both notes from the Palm Pilot and the CrossPads. The system can be adapted to process different types of notes by adding a new job class. Once the UploadServer has packaged the note into a job, the job is then placed into a job queue. The job queue is a list of jobs waiting to be processed and is maintained by the UploadManager Servlet. Jobs are processed on a first-come first-serve basis, with new jobs inserted at the end of the queue. When a job is removed from the queue, the UploadManager invokes the Run method on the job to begin processing that job’s note. The UploadManager is implemented using a pro-ducer-consumer semaphore algorithm so that the server does waste idle cycles waiting for in-coming jobs, and also jobs are processed as soon as they are placed onto the queue.

43

6.3.3.2 Pre-processing a Note Job When the server begins processing a note job, it first parses the raw data into the following sec-tions: collection name, author, note type, creation date, and ink strokes. For the Palm Pilot plat-form, there may be additional data accompanying the note depending on the note type. The pos-sible note types on the Palm platform are: Action Item, New Meeting, Next Meeting, and Note. An action item contains text describing the associated action, while the new and next meeting types have associated meeting times. The standard Note type contains only the data mentioned above and does not contain any additional data. This data format is the same for the CrossPad platform, with the only difference being the note type as either CrossPad or CrossPad XP, which are of different sizes. After the server has extracted the above information from the raw note data, it uses the database module to create a new note entry in the NotePals database. It inserts the collection name, author, note type, and creation date into a new note entry, and receives a unique note ID from the data-base. The server then uses this ID to generate the other note constituents.

6.3.3.3 Note Image Files and Handwriting Recognition In addition to saving the basic note information, the NotePals server also creates two note images files and performs handwriting recognition on the uploaded note. The server uses the raw ink stroke data to generate a thumbnail and full-sized images of the note. These images are named with either an “i_” or “t_” prefix for image or thumbnail, followed by the unique note ID gener-ated by the database (e.g., t_4324.gif is the thumbnail image for the note with document ID 4324). Furthermore, the server also passes the stroke data to a handwriting recognizer. NotePals uses Calligrapher’s handwriting recognizer to perform its ink processing [26]. The server in-vokes the recognizer with the stroke data as input, and receives recognized text and a list of key-words as output. The server then updates the note entry in the database with the note’s image names, recognized text and keywords. Once the note entry has been updated, this completes the upload processing cycle, and the Up-loadManager begins processing the next job on the queue, if any.

7 Future Work We have only begun exploring the possibilities of a property-based document organization sys-tem using NotePals, and there is much more work to be done. Specifically, we would like to add the ability to create uploading rules, save query-based collections, and add support for sharing properties. In addition, we would like to give users the ability to synchronize notes with other media (e.g., slides, papers, etc.), and saving other forms of attachments to notes.

7.1 Uploading Rules One of the features we would like NotePals to have is the ability to create rules that automati-cally organize notes as they are uploaded into the system. These rules will be sets of condition and actions pairs, such that when an uploaded note matches a rule’s condition, the system will apply the corresponding action to the note. For example, a common rule would be to automati-cally move notes to a particular collection for notes created on a given date and time. Another possible rule could be to automatically add a specific property to notes that match a certain crite-

44

ria, such as those taken at a certain location or by a certain author. The user can then look up these marked notes using that property in the application notes browser. As an additional enhancement, we would like to take the rules concept further and try to auto-matically infer organizational relationships from other sources of information. For example, if we can integrate NotePals with the user’s scheduling program, then we can automatically file the uploaded notes into collections marked by the different time periods in the user’s schedule. For instance, if the user is a student who has an Astronomy class on Mondays from 10-11am and Math on Tuesdays from 12:30-2:00pm, NotePals can automatically place notes taken during the Mondays 10-11am slot into an Astronomy collection, and notes taken during the Tuesday slot into a Math collection. Rules such as these would be rather trivial. The more interesting problem would be to automatically organizing notes according to their content, which is somewhat possi-ble since NotePals is already doing handwriting recognition on each note it receives. Having NotePals automatically organize notes on the user’s behalf would greatly reduce the bur-den on the user. However, much care would still need to be taken to convey to the user what has happened to the automatically organized note so the user would not feel disoriented.

7.2 Query-based Collections We would also like NotePals to give users the ability to create query-based collections. Currently, the contents in NotePals collections are static (i.e., they are manually placed there by the user, and will remain there until the user moves or deletes them). In this sense, these collections be-have in the same manner as folders in traditional desktop operating systems. As an enhancement to the system, the user should be able to create collections whose contents are dynamic, by bas-ing the collection on a query. Consequently, when the user goes to look inside a dynamic collec-tion, the system will automatically perform the query embedded in the collection to retrieve its contents. And as notes are added or removed from the system, the contents of dynamic collec-tions will change depending on how their queries were specified. For instance, consider the example mentioned above in the Uploading Rules section. Another alternative for the student to organize her notes would be to create two dynamic collections, one for Astronomy and another for Math, and base each on the corresponding date and time criteria. Once she has created these collections, she will no longer need to file her Astronomy and Math notes manually. When she uploads her class notes now, they will automatically appear in the right collection since they satisfy the collections’ criteria. However, there are several complications with using dynamic collections. As mentioned earlier, our user study suggests that the idea of dynamically changing collections is not intuitive upon first use. Since NotePals collections look and behave so much like folders in traditional desktop systems, users had the same expectations about how they should behave (i.e., their contents do not move unless acted upon). Therefore, despite the assumed advantages to using dynamic col-lections, much care would still need to be taken to distinguish these collections with those whose contents are static. More user studies are necessary before dynamic collection can be fully inte-grated into the system.

45

Furthermore, the characteristics of dynamic collections need to be further defined before they can be fully implemented. For instance, what is the expected behavior when a user attempts to add or remove notes to/from a dynamic collection? In the strictest sense, such changes should not be possible since those notes should always appear in the selected collection since they sat-isfy the collection’s criteria. But from the user’s standpoint, he or she should be able to modify the contents of dynamic collections in the same way as that of static collections. The system can attempt to solve this problem by creating exceptions to the criteria to deal with notes that have been added or removed from dynamic collections. But this causes extra overhead for each dy-namic collection and subsequent performance problems. Moreover, the syntax for the queries of dynamic collections has yet to be defined, and how to integrate those queries with other property criteria in a standard notes search is still unclear. Therefore, despite the advantages of query-based collections, much more work is still needed, both from usability and technical standpoints, before they can be successfully integrated into the system.

7.3 Sharing Properties Shared properties provide users with a common context for organizing and retrieving notes. Cur-rently, users only have access to their own properties, and cannot find or organize shared notes using another user’s custom property. However, if properties were shared, then a user could ei-ther use the shared property to find notes using another user’s organizational context, or organize shared notes using these properties as a shared label. To illustrate this point further, suppose us-ers A and B both created an important property and used them to label a set of shared notes. In the current system, these users would not be able to find out which notes are important to the other since properties are not sharable. However, if properties were shared, then either user A or B can create the important property and share with the other, and both would use the same prop-erty to label important notes. In this case, both users will be able to find notes that are labeled as important by the other user, since the important property is a shared context between them. To share properties in the NotePals system, an additional relationship table would need to be cre-ated that describes which properties are shared with which users. The system can then use this table to determine which properties to display to a user when displaying a user’s property list.

7.4 Synchronizing With Other Media Another feature we would like to build into the NotePals system is the ability to synchronize NotePals notes with other media, such as PowerPoint slides, conference papers, recorded video, etc. Synchronizing different media provides a user with additional sources of context for recol-lecting information. A common example for the classroom environment is to synchronize Note-Pals notes with the lecture slides and recorded class video. A student should be able to use these synchronized sources of information better than the individual sources alone. We have previously built several prototypes that synchronize different media, but have not inte-grated them into the new version of NotePals. More work still need to be done to refine the web interface, along with the technical aspects for synchronizing these media over the web.

46

7.5 Note Attachments Finally, we would like users to be able to add other types of attachments to notes other than text. These other attachments types could be a web page, a related conference paper or document, an interesting diagram, or multimedia content such as recorded sounds or video. These attachments will provide the user with additional relevant information that the user can use in addition to the note itself. To support these attachments, the NotePals database would need to have two additional tables, one for defining an attachment (its type, file, owner, and other attributes) and another that relates an attachment to a note. In addition, a protocol needs to be defined that allows users to upload attachments and associate them with one or more notes. Once these structures and procedures are in place, the user can then view these attachments when viewing notes in the detailed view.

8 Conclusion The traditional hierarchical structure for organizing documents confines users to the file system’s representation and prevents them from forming their own organizational schemes, especially in a shared environment. By creating a property-based document organization system, NotePals of-fers its users the ability to manage, search, and operate on share notes by referring to their prop-erties. At the same time, it also offers collections as replacements to traditional folders in hierar-chical systems for backwards compatibility. By developing NotePals as a web-based application, users can access their notes more easily without regards to platform-dependent issues. At the same time, the web interface provides a common medium for the exchange of notes, as well as the development of an online sharing community. Through the use of property-based organization, NotePals attempts to make docu-ment organization more powerful and less time consuming, while maintaining the maximum us-ability. The current NotePals implementation has laid down the framework for sharing notes on top of a property-based system. Future work still needs to be done to enhance many of its features to give users the maximum flexibility in arranging their documents in a shared, collaborative environ-ment.

9 References 1. James A. Landay and Richard C. Davis, "Making Sharing Pervasive: Ubiquitous Computing for Shared Note

Taking."IBM Systems Journal, 1999. 38(4): p. 531.

2. Richard C. Davis, James A. Landay, Victor Chen, Jonathan Huang, Rebecca B. Lee, Francis Li, James Lin, Charles B. Morrey III, Ben Schleimer, Morgan N. Price, and Bill N. Schilit, "NotePals: Lightweight Note Shar-ing by the Group, for the Group." In Proceedings of Human Factors in Computer Systems: CHI 99, Pittsburgh, PA, May 15-20, 1999, pp. 388-345.

3. James A. Landay, Richard C. Davis, Victor Chen, Jonathan Huang, Rebecca B. Lee, Francis Li, James Lin, Charles B. Morrey III, and Ben Schleimer, "NotePals: Sharing and Synchronizing Handwritten Notes with Mul-timedia Documents." In Handheld CSCW Workshop: CSCW 98, Seattle, WA, Nov. 14, 1998.

47

4. NotePals: Lightweight Note Taking by the Group, for the Group. Richard C. Davis, Jason A. Brotherton, James A. Landay, Morgan N. Price, and Bill N. Schilit, UC Berkeley Computer Science Division Technical Report UCB//CSD-98-997.

5. J. Huang and J. Michiels, "Exploring Property-Based Document Organization in a Collaborative Note-Sharing System," ACM SIGCHI Conf. On Human Factors in Computng Systems - Extended Abstracts, The Hague, the Netherlands, Apr. 2000, pp. 327-328.

6. Nunamaker, J.F., A.R. Dennis, J.S. Valacich, D.R. Vogel, and J.F. George, Electronic Meeting Systems to Sup-port Group Work, Communications of the ACM, vol. 34(7): pp. 40-61, 1991.

7. Mantei, M.M., Observation of Executives Using a Computer Supported Meeting Environment. International Journal of Decision Support Systems, 1989. 5(June): p. 153-166.

8. Olson, J.S., G.M. Olson, M. Storrosten, and M. Carter, Groupwork Close Up: A Comparison of the Group De-sign Process With and Without a Simple Group Editor. ACM Transactions on Information Systems, 1993. 11(4): p. 321-348.

9. Stefik, M., D.G. Bobrow, G. Foster, S. Lanning, and D. Tatar, WYSIWIS Revised: Early Experiences with Multiuser Interfaces. ACM Transactions on Office Information Systems, 1987. 5(2): p. 147-167.

10. Wolf, C.G., J.R. Rhyne, and L.K. Briggs, Communication and Information Retrieval with a Pen-Based Meeting Support Tool, in Proceedings of ACM CSCW’92 Conference on Computer-Supported Cooperative Work. p. 322-329, 1992.

11. Moran, T.P., P. Chiu, W. van Melle, and G. Kurtenbach, Implicit Structures for Pen-Based Systems within a Freeform Interaction Paradigm, in Proceedings of ACM CHI’95 Conference on Human Factors in Computing Systems. p. 487-494, 1995.

12. Elrod, S., et al. Liveboard: A Large Interactive Display Supporting Group Meetings, Presentations and Remote Collaboration. In Proceedings of Human Factors in Computing Systems. Monterey, CA. pp. 599-607, May 3-7 1992.

13. Moran, T.P., et al. “I’ll Get That Off the Audio”: A Case Study of Salvaging Multimedia Meeting Records. In Proceedings of Human Factors in Computing Systems. Atlanta, GA. Pp. 202-209, March 22-27 1997.

14. Abowd, G.D., et al. Teaching and Learning as Multimedia Authoring: The Classroom 2000 Project. In Pro-ceedings of Multimedia ’96, p. 187-198, November 1996.

15. Streitz, N.A., J. Geissler, J.M. Haake, and J. Hol, DOLPHIN: Integrated Meeting Support Across Local and Remote Desktop Environments and LiveBoards, in Proceedings of ACM CSCW’95 Conference on Computer-Supported Cooperative Work. p. 345-358, 1994.

16. Levine, S.R. and S.F. Ehrlich, The Freestyle System: A Design Perspective. Human-Machine Interactive Sys-tems, ed. A. Klinger. Plenum. pp. 3-21, 1991.

17. Want, R., et al., An overview of the PARCTAB ubiquitous computing experiment. IEEE Personal Communica-tions, 1195. 2(6): p. 28-43.

18. Whittaker, S., P. Hyland, and M. Wiley, Filochat: Handwritten Notes Provide Access to Recorded Conversa-tions, in Proceedings of ACM CHI’94 Conference on Human Factors in Computing Systems. p. 219, 1994.

19. Wilcox, L.D., B.N. Schilit, and N.N. Sawhney. Dynomite: A Dynamically Organized Ink and Audio Notebook. In Proceedings of Human Factors in Computing Systems. Atlanta, GA. pp. 186-193, March 22-27 1997.

20. Paul Dourish, Keith Edwards, Anthony LaMarca and Michael Salisbury. 1999. Presto: An Experimental Archi-tecture for Fluid Interactive Document Spaces. ACM Transactions on Computer-Human Interaction, 6(2), 133-161.

21. Anthony LaMarca, Keith Edwards, Paul Dourish, John Lamping, Ian Smith and Jim Thornton. 1999. Taking the Work out of Workflow: Mechanisms for Document-Centered Collaboration. Proc. European Conf. Computer-Supported Cooperative Work ECSCW'99.

48

22. Paul Dourish, Keith Edwards, Anthony LaMarca and Michael Salisbury. 1999. Uniform Document Interactions Using Document Properties. Proc. ACM Symp. User Interface Software and Technology UIST'99.

23. Paul Dourish, John Lamping and Tom Rodden. 1999. Building Bridges: Customisation and Mutual Intelligibil-ity in Shared Category Management. Proc. ACM Conf. Supporting Group Work GROUP'99.

24. Eyal de Lara, Karin Petersen, Douglas B. Terry, Anthony LaMarca, Jim Thornton, Mike Salisbury, Paul Dour-ish, Keith Edwards and John Lamping. 1999. Caching Documents with Active Properties. Extended abstract for HOTOS-VII (Hot Topics in Operating Systems).

25. Tomcat documentation, Copyright ©1999-2000 The Apache Software Foundation - http://jakarta.apache.org/tomcat/jakarta-tomcat/src/doc/index.html

26. CalliGrapher SDK and API documentation, Copyright© 1997-2000 ParaGraph - http://www.paragraph.com/products/internetink/calligrapher/