Internet Librarian: Evaluating Search Tools

Evaluating Search Tools
Mary Ellen Bates, BatesInfo.com

Much of what Mary Ellen had to say can apply not only to search engines but also to all of those subscription databases that we are constantly trialing and evaluating.  This was a great session, and one of the most practical “chock-full-o-info” sessions I attended during the conference.

Using Your First Impressions
Don’t start searching/typing away.  What are your first impressions?  Does anything not make sense?  Can you see links to help files, advanced search, about us?  What options are available in the pull-down menus?  What options are available in the advanced search screen?  Does it require registration?  If so, what is their privacy policy?

Tests to Run
Run through a collection of test searches: something fairly current (within last month, but not yesterday), a specific fact (height of Eiffel Tower), quick-look up (URL for Colorado 14ers Initiative), something with common words (how does it handle ambiguity?).  Also search for something that you are personally interested in, consumer-type searches, a search that you would typically run during your workday, field searches (title, inURL, etc.), intentionally incorrect searches (typos, a specific URL, inappropriate punctuation).  Also do phrase searches, one-word searches, 5-word searches, nested logic, filetype searches, domain-limited searches, foreign language searches, country-specific searches (choose a webpage you know is based in another country but does not have that country’s domain name, search by page title and limit by country…does it come up?).  Can you search within your results?  What kind of Boolean logic is available?  What is the default?  What are the options?  Can you sort results by date?  What is the engine searching (open websites, content within databases, human-built directories, blogs, etc.)?  Try it on IE, Firefox, and other browsers.  Do they require a toolbar or plug-in that only works in one browser?  Compare the results with engines you’re already comfortable and familiar with to see how much overlap there is.  Do any of the results seem weird, like they shouldn’t be there?  Be sure to set your search results display default to 50 or more results per page.

Reviewing Results
See what’s included on your results pages: layout, images, text, links, citation or other data about each result, modify your search, related terms, see also, other ways to expand or narrow your results.  If there are paid ads, are they clearly labeled?  How intuitive are the search results?  How current are the search results?  Does it require the use of pop-ups or additional software or plug-ins?  Can you customize the display of the site?  Does it have the must-have features for your audience/organization?  Does it include links from a human-built directory and are they flagged?

Additional Check-List
How did you hear about the site (trusted source, web search, found it accidentally)?  Look for reviews of the site by the people you consider authoritative.  How long has the site been operating (check the Wayback Machine)?  How has it changed over time?  Is it in beta or just out of beta?  If you are reviewing a directory of other human-built resource, what’s the creator’s credibility?  How frequently is it updated?  Look for links to the site, particularly from authoritative people.   

Technorati tags: IL05, IL2005