© 2005 Hilltop Innovation LLC

Prepared for OM Group by Hilltop Innovation

March 16, 2005

Beyond the Google Gate: Online Research StrategiesBeyond the Google Gate: Online Research Strategies

Beyond Google: Finding Hidden Information on the ‘Net

The “Visible” ‘Net:

Google-accessible information

The Hidden ‘Net:Informationavailable

online- free

- for registration- for pay

Google and similar search engines are still your best gateway, but casual internet surfers stop at “The Google Gate”

and miss a vast untapped pool of information available online. Start with Google; finish by going beyond Google.

Online Research Complements Primary Research

There is no substitute for actually talking to a real person.

However, with the vast resources available online today, there is no excuse for not tapping online information as part of your overall research strategy.

Just as with Opportunity Analysis primary research approaches, upfront planning will increase your research success. Three key planning steps are:

1. Frame your question. What specific information are you looking for?

2. Determine likely information sources. Who probably has the information you need?

3. Develop a search strategy. What tools and strategies will most quickly help you access the sources you seek?

Framing the Question

The classic journalistic “Five Ws” can help you frame your research question about the information you seek.

– WHO Is the research about? Is expert in this topic? Works with/studies/uses information in this topic area? Is likely best source of the data you need? associations,

government agencies, press, research provider, analyst, scholars

– WHAT Information do you need? Statistics, interview sources,

background Information will be most useful? Text articles, presentations,

specific facts, public records Information do you already have? Key words, specialized

vocabulary, key players

Framing the Question--Continued

– WHEN Did the topic or event happen? When is the relevant period you need to research?

– WHERE Might data be found? What kinds of institutions might maintain

or gather the information you seek? Might the topic have been covered? Institutions, populations,

related areas that might mention your topic Might the topic intersect the public sphere and public coverage?

Local newspapers, trade publications, local publications, trade conferences, court records, online discussion groups and blogs, government records and research

– WHY Are you doing the research? To guide a decision, to

analyze a market, to support a plan

Determining Likely Sources

The “Five Ws” can also help brainstorm likely sources of the data you seek.

– WHO Might gather or maintain the information you seek? Is interested or has a stake in the information you seek? Sells this data?

– WHAT Form of information do you seek? Database, public records, text articles, scholarly

information, statistics, broad background Services can you access for one-time or subscription fees? Does each such service offer, and for what price?

– WHEN Are services available, and to whom? Should you pay for information, and when should you seek it for free?

– WHERE Are you most comfortable searching? (services) Do you get the best deals on information? Do you get the easiest/most convenient access to useful information?

– WHY Choose one source or service over another? Fees, ease of use, range of information

available, quality of organization of information

Reaching the Hidden Internet: Approaches and Tools

Search Engines: Google is Great, But Check These Out TooSearch Engines: easy to use, key word-driven, automated indexers of accessible web pages. Cannot reach content protected by passwords and registration or information formatted in ways (databases, certain file formats, non-text media) bots cannot access.

Examples of broad search engines (some of which have evolved into portals or added subject directories to automated search capability):

Google Altavista

MSN Search Hotbot

Lycos Excite

Go.com (formerly Infoseek) Northern Light

LookSmart AskJeeves (natural language)

Teoma (subject relevance)

Searchengines.net is a great place to look for more.

Subject Directories: Never Send a Bot to Do a Human’s Job

Subject Directories: menu-driven database of titles, citations and sites searchable by categories organized and indexed by humans. Useful for background and specialty topics. Selects most relevant, useful sites. Information updates can lag the automated search engines. Due to the human effort involved, subject directories tend to be smaller and more focused than the indexed databases available through the broad search engines.

Examples of subject directories and directories of directories (many of which, like the search engines, have evolved into portals and offer a broad range of information search tools):

Yahoo http://www.yahoo.com/

About.com http://about.com/

Alphasearch http://www.alphasearch.org/

Statistical Resources on the Web http://www.lib.umich.edu/govdocs/stats.html

Infomine http://infomine.ucr.edu/

Dow Jones http://www.dowjones.com/

Direct Search* http://www.freepint.com/gary/direct.htm

The Internet Sleuth http://web.bilkent.edu.tr/Search/Sleuth/

The Internet Public Library http://www.ipl.org/

Webcrawler http://www.webcrawler.com/

*not updated since 2002, but still a great list of links, many of which still work or can be found through Google

Portals: One-Stop Shopping For Online Needs

Portals: a catch-all term for search engines and subject directory services that added other services in an attempt to increase revenue and site visitor “stickiness”—free email, website hosting, scheduling tools, web advertising, chat services, online retail, maps, yellow pages, city search, auctions…

Broad portals: consumer-oriented content

Specialized portals

– facilitate business relationships in a specific industry and offer focused industry-related content

– support other focused communities through services and content useful to that community

Political

Religious

Environmental

Creative

Examples of portals abound, from the grand-daddy of all—Yahoo—to specialized portals for virtually every industry or focused community on the planet. Search Google with “portal” and the industry of interest as a start.

Meta-Tools: Google is Great, But Why Not Search All At Once?!

Meta-Search Tools: search engines that simultaneously search multiple search engines. Suffer from the same limitations of the underlying search engines in their inability to reach the “hidden” web content. Limited to simple key word searches in order to work across multiple search engines, so they sacrifice some of the advanced refinements possible on specific search engines.

Examples of meta-search tools:

Search.com Dogpile.com

Vivisimo Kartoo

Metacrawler.com Inference Find

Cyber411 Highway61

Mamma Profusion

Micro-Tools: Who Ya Gonna Call?

Micro Tools and other specialized search tools: search engines that focus on finding a very specific type of information: phone numbers, court records, quotes, discussion groups, etc. Suffer from out-of-date, error-prone data on the free sites, and the “pay” sites are not much better. For best results, try several sites and seek out specialized sites.

Examples of people-search and “yellow pages” tools:

Anywho Yahoo! People Search

Infospace Switchboard

Examples of “expert” finders:

FACSNET Sources Online http://www.facsnet.org/

Sources and Experts http://www.ibiblio.org/slanews/internet/experts.html

Finding background info through online FAQs:

Search [key word] + FAQ

www.faq.org

Commercial Services

Many commercial information services exist, serving broad to narrow information needs. A few good examples:

Company background information

- OneSource

- Hoovers

- Multexnet

News and lit search capability

- www.dowjones.com

- www.lexis-nexis.com

- www.dialogweb.com

Government Sources

Your tax dollars at work:

Government “portals” to information available:

- www.firstgov.gov

- www.fedworld.gov

- Documents Center www.lib.umich.edu/govdocs

Key government information sites:

- www.census.gov

- www.sec.gov search also for EDGAR and SEC

- www.loc.gov search also for THOMAS and LOC

- www.ntis.gov

- www.bea.doc.gov

- www.fedstats.gov

- CIA World Fact Book

Business Research Sources

For focusing on business research, a few useful sites include:

Company background

- www.thomasnet.com

- Hoovers (pay)

- Dun & Bradstreet (pay)

- OneSource (pay)

Business news

- www.businesswire.com

- www.crainsny.com (local business news)

- sources previously mentioned, major newspapers and online news sites

Financials: SEC/EDGAR, investment sites (Quicken, MSN Money, ValueLine, S&P, Zacks, Multex, etc), personal brokerage accounts online services (e.g., TDWaterhouse offers analyst reports)

Search for industry-specific portals

Search for industry trade associations: try www.asaenet.org

For Further Study…Search Tactics and Tools

A few great sites to help you learn more about finding, evaluating and using specialized web search tools:

SearchEngineWatch.com

SearchEngineShowdown.com

Berkeley tutorial on internet search techniques http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/SearchEngines.html

A few tactics for aiding your search:

Master Boolean operators: AND, OR, NEAR

Use phrases, not single words; use unique words, most unique first

Seek specifics (easier to find than broad overviews)

Master search engine-specific refinements (menus, search tips)

Use wildcard symbol *

Book recommendation (updated annually):

Find It Online: The Complete Guide to Online Research, by Alan M. Schlein, which deserves much credit in the creation of this presentation.