Lecture Eight: Reading (URL) - Robert Harris: "Web Search Strategies"

• Search Engines. A search engine consists of the interface you use to type in a query, an index of Web sites that the query is matched with, and a software program (called a spider or bot) that goes out on the Web and gets new sites for the index. The bot crawls the Web at certain intervals, in order to index new material. When you use a search engine, you are asking it to look in its index to find matches with the words you have typed in. Some search engines index not only the World Wide Web, but also Usenet newsgroups. Many search engines are now becoming reference sites which contain much more than just search capability. They may also have news, weather, free software, picture indexes, ratings of web sites, and other features. There are several hundred search engines, but they fall into a handful of types:
  Global. This type of engine, typified by Google, Fast Search, Northern Light, HotBot, AltaVista, and others, reads pages from all over the world in many languages. These engines may index more than a billion pages.
  Regional. Some search engines are limited geographically. For example, only information on Web sites in Australia may be indexed.
  Targeted. These search engines limit themselves to one subject, like biography, medicine, graphics, art, fishing, and so forth.
  Reference. These provide information from a set of reference works, such as an encyclopedia. Britannica, Bartleby and xrefer are examples.

• Directories. Directories are categorized lists of sites picked out by human editors. Directory databases are therefore much smaller than those of search engines. However, the fact that the sites are hand picked often means that you will find very high quality sites or articles in the results. Example directories are Yahoo, Look Smart, and Snap.

• FOREST LOG. Here is a scheme, called FOREST LOG, to help generate search terms. Suppose, for example, you wish to research the validity of testing. Here is how you might use the Forest Log scheme to generate terms.
  FO Forms
  Forms or variants of the words you are thinking of. For example, if you search on validity of testing, you may miss a page that discusses validity of tests. So you should include the forms: test, testing, tests. Many search engines allow wild cards to cover most forms by using a wild card, often an asterisk, as in test*.
  RE Related Terms
  Related to testing is measurement, assessment, performance, criteria, judgment, evaluation, and so forth. A search only on validity of test* will miss validity of performance measures.
  ST Synonymous Terms
  What other words are synonymous or used in place of the word test? What about exam, examination, assessment, quiz, midterm? As another example, suppose you search under apple growing, and find a few items but not what you want. What other possible phrases might be used in an article that would cover this topic? You might search on fruit tree farming, fruit orchards, Washington Red Delicious, etc.
  LOG Ladder of Generalization
  The higher on the ladder, the more general or comprehensive the term, while the lower on the ladder, the more specific the term. Thus, for example, a list of terms related to testing, from most general to most specific, might include measurement, assessment, testing, performance test, weightlifting performance test. When you are developing a set of search terms and searching with them, if you do not get the results you want, move up or down the ladder of generalization and generate some more terms.