Presentation I gave at Enterprise Search Summit 2011. It suggests low-tech and relatively inexpensive ways to bring failing search projects back to life.
TRANSCRIPT
1Guident - 198 Van Buren Street, Suite 120 Herndon, VA 20170 - Tel: 703.326.0888, www.guident.com
Search Systems Redux:Restoring health to failing search
I’ll draw from my experience on various search projects. One started before the web was widely used and before there was a Google, and at the other extreme, a project during the past 12 months. Both projects exhibited some of the same problems and relatively easy fixes.
What most search projects have in common
4
• Organizing and searching content, interface design.
the White Rabbit was still in sight, hurrying... There was not a moment to be lost: away went Alice like the wind, and was just in time to hear it say,…‘Oh my ears and whiskers, how late it’s getting!’
Presenter
Presentation Notes
Speaking of those two search projects, one thing in common with them both of them was how long it took to complete them; the earliest took about a year; the latest over half a year. In both cases I think they should have been done in half that time, but that’s the nature of search projects.
Field of Dreams -- NOT
• “if you build a search system they will come” usually not true.
• Or only be because they must, at least initially.
• When they come, very different search system expectations.
Other search systems Where are they? In CMS systems, on desktop…
Social Systems Tagging Managed or Folksonomy?
Scanners/OCR Poor OCR Search systems can’t find pictures of text.
• Few –within or outside IT– understand all these pieces. •Few –within or outside IT– understand all the systems that are affected by a search project.• Money (or hardware) may not be the problem. Possibly less expensive approaches.
Presenter
Presentation Notes
Formats, rights, volume, language, people… Don’t presume every format can be full-text searched. I’ve found that ZIP files, for example, couldn’t be searched by the OEM edition of FAST in Documentum. I was disappointed, but thought further: what would it mean to search for a ZIP file, full-text searching each file in the package? Just receiving a single hit for the entire ZIP? Rights management might be too blunt an instrument for files within the ZIP too. What about Visio? Culture issue with Rights: Do people expect to be able to view any file they can access? Is sharing just not part of the culture?
“Time required to support and maintain the <search> system 14%.” Response to poll question “What’s the top drawback of Enterprise Search, described in “Go Rogue with Enterprise Search,” InformationWeek, 14 Mar 2011
Presenter
Presentation Notes
IT sees findability projects as “Search System” projects, with a beginning and an end, perhaps a point-and-click installation. May not be aware of business needs or user expectations that led to funding the project. May see the project as “install software and walk away.” Don’t realize the complexity of searching within the firewall, and systems that are involved. Or, integrating Search with enterprise repositories is a point-and-click process. Install it and we’re done. We’ll pick the search system that our favorite vendor sells. Search is like other software, and all search systems are alike. Implementing Search is just like installing any piece of software When we install it, we’re done (except for version upgrades and occasional outages). Assume that they know all about search systems and can configure it without business input. Assume all business users understand search and don’t need training, etc.
What’s wrong? We “selected all” features for the users.
We selected the system based on architectural and functional requirements.
It passed UAT.
“IT generally thinks it doesn’t have a search problem. 56% rank search in the bottom third of project lists.”“Go Rogue with Enterprise Search,” InformationWeek, 14 Mar 2011
What IT Found – Details
• Users griping
• Search systems - lots of moving parts:
– What is indexing anyway? Is it like a database index?
– What are incremental and full indexes? Optimization? What impact do they and other factors have on performance and system resources?
GRIPES: System too slow. Can’t figure out the search screens. Can’t find what they are searching for Overwhelmed by so many features and options. Don’t laugh. Few software developers have ever worked with Search except as a GOOGLE user.
“…straightforward search engine mechanisms are unable toprovide the pattern matching, trend plotting and semantic analysis that may be required.” AIIM Industry Watch, Content Analytics, 2010
Presenter
Presentation Notes
Why do we need training? Nobody ever trained me to use Google. Recipe for
“Search is being brought <into the enterprise by>Business unit leaders, not IT, driving search Purchases.”“Go Rogue with Enterprise Search,” InformationWeek, 14 Mar 2011
What Business Found – Details
• Ease of Use:– Awkward interface
– They need training about how to search inside the firewall. All that metadata!
• Comparisons with Google or Bing:– System is slow and results not relevant.
Nagging belief that search is missing things. “Why can’t I find CAD or zip files?” It takes IT too long to add features: Integrating specialized thesauruses, Adding (or removing) metadata elements Adding new repositories to search Not sure if they are saving any money, but know what they paid initially and ongoing.
How to fix things, quickly and cheaply?
• Look at the full project lifecycle. Anything skipped, or misunderstood? Invite all players to participate
• Probe the competence of IT in managing Search systems.
• Confirm the clients’ and IT’s understanding of Search systems.
Names of the business sponsors What users asked for How well they understood what they asked for How well they understand how to search inside the firewall. Do users understand the difference between “configuring” and “customizing”? Upgrades to user interfaces and search system user interfaces can be done with “Rapid Application Development” and quick turn-around prototyping. Initial selection and installation of Search really needs soup to nuts project management. Documents and pictures have been around a long time, and thus we have strong feelings about them, the meanings of words, and our rights to find and use them. (My Prius vanity plate is “WORD.”) Emotion runs much higher on findability projects than for information systems generally, yet due diligence in findability projects requires we use both the right and left sides of our brains, and the three moving parts such as human/process factors as well as all the steps in the project lifecycle. There are no shortcuts, but you can crawl, walk, then run. 80-20 rule.
Yes, there are lots of details. At least consider the details briefly. And these phases can have loop-backs. You can use “waterfall,” Rapid Applications Development, Agile development, Mike 2.0 with frequent prototypes (I lean towards that), you have to consider all these points. Similar to any project lifecycle, except that the center 4 processes continue. The four blocks to the right continue to cycle, although with reduced resources, after the initial findability project is “finished.” Due diligence requires that you at least consider each of the topics (and more, depending on your processes and culture) in each of the lifecycle phases. AIIM provides a wealth of “8 things eBook series,” and many of them are freely available even to non-members. One in particular, “8 reasons you need a strategy for managing information,” is a personal favorite. You can find the first edition at: http://www.aiim.org/forms/downloads/8things%20ebook--info%20mgmt--ebook--landscape.pdf Although I wouldn’t be surprised if a 2nd edition is in the works.
Initiate
• Unexpected allies? Librarians, taxonomists, records managers, ECM users, Technical Writers, Attorneys (eDiscovery issues), Business Analysts …
• Squeaky wheels?
• Goals and objectives? Business or Technical? Lower costs? Reacting to a lawsuit?
• Green issues can include cost savings.
19
Initiate Analyze Design Build Deliver
“As part of their business case, 37% would find it “extremely” or “very useful” to demonstrate the ‘Green IT’ benefits of ECM <also Search>, particularly with regard to fewer photocopies and file copies. AIIM State of the ECM Industry, 2010.
Goals? You’d be surprised. One sponsor told me candidly that in-house people just hated the system. Nobody came to weekly training sessions. I was sent to talk to the squeaky wheels. This goal was, in part, to put oil on the squeak. What happened is that I learned a great deal from these sometimes difficult interviews.
Initiate
Scope:
– Anticipate trends (blogs, wikis, social tagging…)
– Augment or upgrade what you have today, or will you replace it?
– Training within scope? (It had better be.)
– Cost savings a business requirement? How was it determined? Realistic?
20
Initiate Analyze Design Build Deliver
AIIM Findability Poll response to the question “I believe the ‘wisdom of the crowds’ improves information quality.” Ages 18-30, 57% agreed. Ages 31-45, 49% agreed. Ages > 45, 33% agreed. AIIM Webinar, ECM in 2010.
People are so used to searching with Google that the idea of training seems foreign. Nobody needs training to use Google! Can you set up a wiki for the squeaky wheels and promoters to meet and discuss issues? Or is that kind of give-and-take foreign to the culture? Do the squeaky wheels identify the Project Sponsor/Manager to be the source of the problem, perhaps inflexible? (That’s a difficult issue.) Few people in large organizations understand all the metadata that is available for searching.
Training is a relatively simple fix
• Training– For IT if they don’t understand search systems and all the
moving parts
• Performance
• Thesauruses
• Indexing…
– For users if they think finding information inside the firewall is easy• Search concepts
• Metadata
• Taxonomy
• Compare a rich internal search with metadata with Google search and its metadata.
I was recently on a project where the users I was asked to interview almost universally hated the system because of all the reasons I mentioned. They had quite a few requests for features. I did a deep dive in the vendor literature and found that much of what they wanted was already available. Most of the project simply needed robust training (I did 2 hour sessions).
Analyze
• Interview business users, promoters and detractors. What are they saying?
• Users understand taxonomy (folders, metadata, home page navigation )?
• Usability issues– Difficult to use?
– System slow or not finding what it should?
– Is there a tactical quick win consistent with strategic goals?
Do users really understand how to use the system? Do they understand the difficulty of searching inside the firewall? Tell the Search 1 story about usability, over-featured search, inscrutable HELP files. Is the system really too slow? Do users understand the tradeoffs between rich capabilities, performance, and even ease of use? What do they really hate about it? It could be something as simple as weird error messages or a (perhaps well-founded) belief that the system just doesn’t find all that it should. Was the system interface designed by someone with no usability skills? Was the system designed and built by a series of revolving door consultants? Do users understand the metadata that they can use to search? Incredible usability issues: The button is labeled “Search” in one screen, “Go” in another, and “Query” in yet another. Was there a fundamentally bad design decision, such as “displaying” object metadata but not making it searchable? Is there a useful HELP system? You wouldn’t believe what I’ve seen. “Search Tips” meant for portal developers.
Usability Example: Inscrutable error messages
• One of my personal favorites: Inscrutable error messages
• Yet there is a reason for it behind after you dig deeply enough.
Explain what filters are. Dig deeply to see why (poor OCR in this case) the filter error occurred… first page was image, not text. Which can lead to an indexing failure So users can’t find everything they are searching for.
Example: Indexing Errors / Strategies
• Indexing errors?
• Make sure error logs are being analyzed to fix root problems.
• Full (re-)indexing or index optimization occurring periodically?
Explain to project sponsors. One is very expensive; the other is relatively inexpensive Programmers may want to customize (more challenging), and may not even know what they’re getting in to. Make sure business side understands the difference. For customization versus configuration, I use the simple example: Buying a new car. Green, sound package… Yacht-like wooden wheel, no.
For most content types, our respondent’s ability to “research” is 3-6 times less than their ability to “search”, particularly for rich media files, but also office documents and emails. AIIM Industry Watch, Content Analytics, 2010
Presenter
Presentation Notes
You may be surprised to learn that nobody has heard of governance, or that governance is just those who shout the loudest. Developing a governance framework (we’ll discuss that momentarily) is a delicate balancing act based on company culture, all the content, and your business processes. Tools are available to manage taxonomies, and although they are evolving, you may find them very useful. Specific content management systems usually have partners with such tools that at least have “connectors” to them. Schemalogic, for example, works with SharePoint and Documentum. Yet be aware that “license price” may be a small percentage of full implementation cost, and you will almost certainly need vendor consulting to get you started.
Open Source is being actively promoted at Federal level (Vivek Kundra) Open source is also starting to be implemented at state level (Nebraska – Alfresco) Netflix – Solr Lucene Likely to have up-front customization needs, but you likely need that anyway. Longer term, costs dramatically lower. If culture is inclined towards following IT standards, most open source solutions follow published standards. M&A – Removes the problem of buying a proprietary search system, originating firm gets acquired and product is dropped by the acquiring vendor .
Analysis – questions to consider
• Searchable versus displayable metadata?
• Can you remove some “built-in” properties that are not useful?– Personal favorites: Character Set,
“Too much irrelevant data will be found 23%.” Response to poll question “What’s the top drawback of Enterprise Search, described in “Go Rogue with Enterprise Search,” InformationWeek, 14 Mar 2011
Presenter
Presentation Notes
You will probably need several taxonomies (in the general sense of folder hierarchies, thesauri, etc.). Watch out for your project allies. Even when you think you have consensus on an artifact such as a taxonomy, you’ll find a frequent attempt for team members to revert to their former comfort zones. Records managers may want a taxonomy that reflects their records schedules, which most business users will not be familiar with. Other rules of thumb: Some taxonomy development work is best done without large group interaction. Suggest the following checkpoints: 1) Definition of scope of taxonomy: how will it be used, what content we need to consider, etc. 2) Articulation of the major branches of the taxonomy (with their scope) 3) Determine the second level in each branch 4) Discuss and define all terms in all branches, then ask for sign off.
Design
• Index more metadata?
• Effect on performance (and training)?
• Upgrade taxonomies?– Avoid business organizational (changes, hard to work with cross-organizational content)
– Consider a process approach: What new business processes produce documents to be searched?
• More or different hardware?– Memory? Servers?
– How much?
– Google Search Appliance? • Quick comparison with what you’ve got now.
• Users love Google – 80-20 possibility?
• Integration issues (e.g., target system metadata)
You will probably need several taxonomies (in the general sense of folder hierarchies, thesauri, etc.). Watch out for your project allies. Even when you think you have consensus on an artifact such as a taxonomy, you’ll find a frequent attempt for team members to revert to their former comfort zones. Records managers may want a taxonomy that reflects their records schedules, which most business users will not be familiar with. Other rules of thumb: Some taxonomy development work is best done without large group interaction. Suggest the following checkpoints: 1) Definition of scope of taxonomy: how will it be used, what content we need to consider, etc. 2) Articulation of the major branches of the taxonomy (with their scope) 3) Determine the second level in each branch 4) Discuss and define all terms in all branches, then ask for sign off.
Design
• Index design– Full versus incremental indexing
– When – “on the fly” for everything? End of day or end of week?
• Develop business processes to review search logs and act on them.
• New repositories to search? If so, how to connect with them? CMIS?
• Social Tool Tip: Can users share search queries? Can you set up a WIKI for users to share useful queries?
Who to participate? Those who are familiar with the taxonomy: May not learn as much. They’ve already drunk the Koolaid. Those unfamiliar with the taxonomy: Learn more, need more upfront training and time. Taxonomy means many things to many people. As a hierarchical structure for information it could include a folder structure (for navigation); metadata; thesauri and synonym lists, etc. Testing can take lots of time and resources, although you will never be able to test as much as you’d like. Be sure you have rock-solid definitions of taxonomic terms (and metadata). Otherwise people will think they understand what each node or metadata element means, but they will get vastly different results when they search or look for information. I was surprised how much work was involved with testing an ECM taxonomy in a recent consulting assignment. We decided we had resources only for 4 tests of 2 hours each. We spent at least 10 hours for every hour of testing, to record and analyze the test results. Metadata may have changed over the years. Explain the metadata elements, which repositories they apply to, and time periods where new ones were defined, old ones were dropped.
Deliver – Install and Walk Away?
• No.
• Ongoing Auditing and Governance– Are new metadata elements needed?
– Who decides?
– Are there new repositories to be searched?
– Are there problems that need fixing?
32
Initiate Analyze Design Build Deliver
Information Systems Governance:…a subset discipline of Corporate Governance focused on Information Technology (IT) systems and their performance and risk management. IT governance implies a system in which all stakeholders, including the board, internal customers, and in particular departments such as finance, have the necessary input into the decision making process.Wikipedia, “Information Technology Governance.”
In one project, users wanted the ability to add new metadata or values to existing ones. If they could have done that in isolation, the perceived benefit would have been overshadowed by confusion (and what to do with legacy metadata?). Delivering a well organized intranet site map, a spruced-up home page, and consistent look-and-feel to sub pages, had an unintended consequence: Site traffic increased so much that performance went down. The problem of search performance (for example) merely caused the development life cycle to resume. Still, an ironic measure of success: The intranet orphan went from having nobody claiming it to several organizations saying it was theirs all along. HR finally won the prize. After a year of work proposing an enterprise search solution for a large property-casualty insurance company, I was pretty much losing interest in it when the president of the company smiled and told me he was funding the project. Since enterprise search within the firewall was then a new idea, I worked with our in house multimedia training group to develop a corporate video that would “train” users how to use it. Luckily, the multimedia producer convinced me that would be dead-boring. He suggested a “film noir” story about how Search could mitigate an insurance loss… I helped write and star in this with an enthusiastic business stakeholder (IT alone wouldn’t have worked). The viral acceptance that followed prompted widespread adoption of the search system called “Topic”, a Verity solution. I thought if I had the sponsor convinced, we could implement the solution and walk away, especially after the film noir promotion. Since this project was before Google, I almost lost the support of the business community (who also never had time to be trained). I established an email newsletter with tips and tricks, and success stories. This unexpected need to sell, sell, sell finally won the business over, who –I was surprised to learn– had had the idea all along.
Deliver – Install and Walk Away?
33
Initiate Analyze Design Build Deliver
• Deliver training
• Advertise social feedback tools: Wikis, Blogs, even email• Keep users engaged and involved
• Establish and maintain governance structures
• Have high-level stakeholders publicize their approval
Users are surprisingly passionate about search and you’ll need to track changing requirements, changes versus status quo. The bouncing Easter Egg issue on the intranet made governance a front-and-center issue. Yet oversight threatened to squelch the enthusiastic effort by volunteers. The experience was a little like herding cats. Providing them a quarterly forum to discuss emerging issues and upcoming new features helped keep the cats under control. Governance, however, should be a larger subject than just having volunteers agree. As we recommended to a large Federal agency, there are two sides to governance: Business policy and technical. IT takes care of the technical side, and essentially implements the business policy direction.
In Summary
34
• Andy Grove was right: Only the Paranoid Survive and get to deliver findability results successfully.
• Use both the left (analytical) and right (creative) sides of your brain, and make sure your team has both sufficient technical and political skills, throughout the full lifecycle of your findability projects.
• Don’t boil the ocean. You’ve succeeded if your users are happier and you’ve added search options or repositories to search.
• There are plenty of basic, inexpensive ways to improve the search experience.
• And don’t forget that findability projects never end, they just change their phases.
