strategies llc taxonomy november 2, 2005copyright 2005 taxonomy strategies llc. all rights reserved....

Strategies LLCTaxonomy

November 2, 2005 Copyright 2005 Taxonomy Strategies LLC. All rights reserved.

Information Classification in the Workplace

Joseph A. Busch

2Taxonomy Strategies LLC The business of organized information

Agenda

Why information classification is needed What the research says Agricultural economics case study


Information access issues, problems, and concerns

Enormous volumes of information within organizations Diversity of assets

Content and technology

Complex and IT-oriented standards .NET, SOAP, WSDL, etc.

Limited (if any) integration with applications: Search engines Information management applications Back office transaction-based systems Analytical systems …


Finding information should not be about “Feeling Lucky”


Finding information requires multiple approaches


http://www.tesco.com/winestore

Wine shopping by facets


http://www.towerrecords.com


Powered by

http://www.fortunoff.com


Facets on FirstGov.gov site

OrganizationOrganization

Content TypeContent Type

FrequencyFrequency

AudienceAudience


about 3,890,000 results


2,199 results


More than 1,000 relevant

results


Agenda



Usability research— Taxonomy compared to search results lists

“We found that users preferred a browsing oriented interface for a browsing task, and a direct search interface when they knew precisely what they wanted.”

Marti Hearst (and others)

“The category interface is superior to the list interface in both subjective and objective measures.”

Hao Chen & Susan Dumais


Taxonomy compared to search result lists

0

20

40

60

80

100

120

140

Category List

Me

dia

n S

earc

h T

ime

in

Se

con

ds

In top 20 results

Not in top 20 results

Category is 36% faster

Category is 48% faster

Source: Chen & Dumais


Time saved—Taxonomy compared to search result lists

1 hour per day searching x 36% faster = 22 minutes each day

22 minutes x 250 working days per year = 5500 minutes or 92 hours per year


User interface survey— Which search UI is ‘better’?

Criteria User satisfaction Success completing tasks Confidence in results Fewer dead ends

Methodology Design tasks from specific to

general Time performance Calculate success rates Survey subjective criteria Pay attention to survey

hygiene:– Participant selection– Counterbalancing– T-scores

Source: Yee, Swearingen, Li, & Hearst


User interface survey — Results (1)

Which Interface would you rather use for these tasks?

Google-like Baseline

Faceted Category

Find images of roses 15 16

Find all works from a certain period 2 30

Find pictures by 2 artists in the same media 1 29

…

Overall assessment:Google-like

BaselineFaceted

Category

More useful for your usual tasks 4 28

Easiest to use 8 23

Most flexible 6 24

More likely to result in dead-ends 28 3

Helped you learn more 1 31

Overall preference 2 29

…



User interface survey — Results (2)

6.06.7

4.7 4.6

5.8 5.56.0

4.0

7.26.3

3.5

7.7 7.4 7.8

4.8

7.6

0123456789

Faceted Category

Google-like Baseline



Trusted advisers—Taxonomy avoids costs

“The amount of time wasted in futile searching for vital information is enormous, leading to staggering costs …”

Sue Feldman,

Poor classification costs a 10,000 user organization $10M each year—about $1,000 per employee.

Jakob Nielsen, useit.com


Searching

Creating

Commun-icating

Knowledge workers spend up to 2.5 hours each day looking for information …

… But find what they are looking for only 40% of the time.

Source: Kit Sims Taylor


Creating new

content

Recreating existing content

SearchingCommun-icating

25%8%

Knowledge workers spend more time re-creating existing content than creating new content

Source: Kit Sims Taylor (cited by Sue Feldman in her original article)


Agenda



Task-based test*

15 representative questions were selected Perspective of various organizational units Most frequent website searches Most frequently accessed website content Correct answers to the questions were agreed in advance by team.

15 users were tested Did not work for the organization Represented target audiences

Testers were asked “where would you look for …” “under which facet… Topic, Commodity, or Geography?” Then, “… under which category?” Then, “…under which sub-category?” Tester choices were recorded

Testers were asked to “think aloud” Notes were taken on what they said

Pre- and post questions were asked Tester answers were recorded

* Based on Donna Maurer’s usability

work with the Australian government


Task-based testing: Representative questions

1. How much cotton is imported from China? 2. What are the impacts of “mad cow" disease on U.S. meat production, sales?3. What is the average farm income level in your state?4. How much of our diet comes from fast food?5. How many people receive WIC benefits (Special Supplemental Nutrition

Program for Women, Infants, and Children)?6. How much acreage is planted to genetically engineered corn?7. What is the cost of foodborne illness in the United States?8. What part of food costs go to farmers, retailers?9. Which States produce the most tobacco?10. What percentage of farms in the United States are small farms?11. What are the costs and benefits associated with providing more traceability in

the U.S. food supply?12. How many people in America don’t get enough to eat?13. What is behind the trade balance (surplus or deficit) in agricultural goods?14. What is the extent of conservation compliance? How does that impact

farmer's decisions?15. What are the impacts of foreign trade restrictions on U.S. farmers, U.S. food

prices?


Task-based testing: Closed card sorting

3. What is the average farm income level in

your state?

1. Topics2. Commodities3. Geographic Coverage

1. Topics1.1 Agricultural Economy1.2 Agriculture-Related

Policy1.3 Diet, Health & Safety1.4 Farm Financial

Conditions1.5 Farm Practices &

Management1.6 Food & Agricultural

Industries1.7 Food & Nutrition

Assistance1.8 Natural Resources &

Environment1.9 Rural Economy1.10 Trade & International

Markets

1.4 Farm Financial Conditions

1.4.1 Costs of Production1.4.2 Commodity Outlook1.4.3 Farm Financial

Management & Performance

1.4.4 Farm Income1.4.5 Farm Household

Financial Well-being1.4.6 Lenders & Financial

Markets1.4.7 Taxes


Task based testing: Card sort analysis

Find-it Tasks User 1 User 2 User 3 User 4 User 5

1. Cotton Cotton Cotton Asia Cotton Cotton

2. Mad cow Cattle Food Safety Cattle Cattle Cattle

3. Farm income Farm Income Farm Income US States Farm Income Farm Income

4. Fast foodFood Consumption

Diet Quality & Nutrition

Food Expenditures



5. WIC WIC Program WIC Program WIC Program WIC Program WIC Program

6. GE Corn Corn Corn Corn Corn Corn

7. Foodborne illnessFoodborne Disease

Foodborne Disease

Consumer Food Safety

Foodborne Disease

Foodborne Disease

8. Food costs Food Prices Market Structure Market AnalysisFood Expenditures

Retailing & Wholesaling

9. Tobacco Tobacco Tobacco Tobacco Tobacco Tobacco

10. Small Farms Farm Structure Farm Structure Farm Structure Farm Structure Farm Structure

11. Traceability Food System Labeling PolicyFood Safety Innovations

Food Safety Policy Food Prices

12. Hunger Food Security Food Security Food Security Food Security Food Security

13. Trade balanceCommodity Trade

Trade & Intl Markets

Commodity Trade Market Analysis

Commodity Trade

14. ConservationsCropping Practices

Conservation Policy

Conservation Policy

Conservation Policy

Conservation Policy

15. Trade restrictions Trade PolicyFood Safety & Trade WTO Market Analysis

Commodity Trade


Task based testing: Card sort results

In 80% of the trials users looked for information under the categories that we expected them to look for it.

Breaking-up topics into facets makes it easier to find information, especially information related to commodities.


Task based testing: Card sort results

Test Questions%

Correct%

Agree

1. Cotton 91% 82%

2. Mad cow 73% 64%

3. Farm income 100% 55%

4. Fast food 91% 73%

5. WIC 100% 100%

6. GE corn 100% 100%

7. Foodborne illness 82% 82%

8. Food costs 55% 27%

9. Tobacco 100% 100%

10. Small farms 91% 91%

11. Traceability 36% 18%

12. Hunger 100% 73%

13. Trade balance 36% 64%

14. Conservation 91% 91%

15. Trade restrictions 55% 36%

Possible change required.

Change required.

Possible error in categorization of this question because 64% thought the answer should be “Commodity Trade.”

On these trials, only 50% looked in the right category, & only 27-36% agreed on the category.

Policy of “Traceability” needs to be clarified. Use quasi-synonyms.


Task-based testing: User satisfaction survey

Was it easy, medium or difficult to choose the appropriate Topic?

– Easy – Medium– Difficult

Was it easy, medium or difficult to choose the appropriate Commodity?


Was it easy, medium or difficult to choose the appropriate Geographic Coverage?



User satisfaction survey: Results

-

0.50

1.00

1.50

2.00

Topic Commodity Geography

Facet

Ea

sy

-

->

Dif

fic

ult

EasierMore Difficult


Summary

1. Classifications and classification-like schemes are being used to facilitate information seeking in the workplace.

2. Users take advantage (and prefer) this type of scheme (faceted navigation) when it is made available in the user interface.

3. Hierarchical navigation is guided by the UI.

4. It is best combined with keyword searching. E.g., keyword search followed by faceted navigation of results.

5. What are the challenges they face in navigating classificatory structures?

6. What are the differences between their use of traditional classification schemes such as the Dewey Decimal Classification and navigation of newly developed online directories?

strategies llc taxonomy november 2, 2005copyright 2005 taxonomy strategies llc. all rights reserved....

Documents