modified 05.10.96
Searching and Wandering the Web

"How do I find X?"

Turn to friendly spiders, robots, wanderers, and other web creepy crawlers...

The Internet is huge, vast, and some might say, in horrific disarry. How do you find things in all of this? There is no one single index or tool to find everything and anything. However, there are some resources out there to make it a little easier. We've tried to organize the various search engines into similar categories for the ways in which they search and the scope of what they search.

If you do not find a search type here, try the ever-growing list maintained at Yahoo or WebCrawler's List of Robots

Or, if you like to be shown the ropes, take a walk with our webhound...

No matter what, you will end up having to do some exploring....

One Page-One Click Meta-Searches

A "Meta-Search" will take your query and search for it by sending your request to other search engines, all in parallel. The "one-click" approach means doing all of this work from a single interface.

Savvy Search: has a single field for you to enter keywords and sends your request to more than 18 other major web search sites. SavvySearch has a built in mechanism to rank your search and send it to the most appropriate sites.
MetaCrawler: is a "Multi-Threaded Web Search Service" designed to take your keyword and submit it to several other search angines, returning a ranked list of sites in one form. MetaCrawler can also authenticate the returned links, which may take longer than other search engines, but offers more reliable results. The MetaCrawler is free and differs from other services in that it doesn't maintain any internal database. Rather, it relies on the databases of eight different services: Open Text, Lycos, WebCrawler, InfoSeek, Excite, Inktomi, Yahoo, and Galaxy.
IBM InfoMarket: can conduct parallel keyword searches by sending the request to Disclose SEC database, McKinley Internet Direct, OpenText Internet Search, USENET Newsgroups, and Yahoo. You have to register for the service to use it, but it is free.
W3 Catalog: The W3 catalog is a huge searchable collection of W3 resources. It is created daily by concatenating a list of resource databases collected by other net providers including the NCSA What's New Archives

One Page-Several Click Meta-Searches

These are pages that have have set up interfaces to search many other sites. Typically you will have to enter your keywords in several places and submit them individually. However, this is much easier than connecting to each search site seperately.

c|net search.com: Boasting more than 250 search engines, c|net has neatly organized an interface to the many other search sites on the web.
Find-It: offers an extensive array of search sites, nicely organized into tables.
All-In-One Internet Search: from its description, "This page is a compilation of various forms-based search tools found on the Internet. They have been combined here to form a consistent interface and convenient ALL-IN-ONE search point." The searches are categorized (i.e. Wordl Wide Web, Software, Publications/Literature, News) and is very comprehensive.
The Inquirer: provides a one page access to many of the major web search sites, grouped into the categories of Current Events, Software Files, Reference Materials, and People and Computers
The Internet Sleuth: provides a subject-oriented directory of hundreds of other searchable indexes and databases.
CUSI (Configurable Unified Search Engine): is a "one-page-does-all"-- a configurable search interface for many searcheable WWW resources. It allows you to quickly check related resources, without having to navigate an re-type the keywords.

Robots, Spiders, and Crawlers

All of these creatures are built up by sending out "agents" on the world wide web that follow every link possible, returning their results to a large, searchable database. For more information, take a look at Robots, Wanderers, and Spiders

Alta Vista: The Largest Web Index: The competition is on as far as what "largest" means, but this server is very efficient and flexible. It offers access to some 8 billion words found in over 16 million Web pages plus a full-text index of over 13,000 news groups updated in real-time.
Inktomi: also claims to be the "biggest" web search tool. Inktomi uses parallel computing technology to build a scalable web server using workstations; it is part of the Network of Workstations (NOW) project at the University of California at Berkeley.
Excite: Another claim as the biggest! "The Intelligent way to naviagte the net" includes NetSearch, an exhaustive net wide search of over 1.5 million web pages; NetReviews a searchable and category index of sites that have been reviewed; and Bulletin -- constantly updated news articles and editorials.
Open Text Index: is a fast, powerful search engine to a massive index of the Web. The results include not only links to sites that match the search criteria, but also a short description, keywords taken from the headers of documents, and an ability to conduct a search of similar sites. OpenText also offers a "Power Search" page where you can construct more complex queries.
Lycos: created at Carnegie Mellon University, is the catalog of the Internet. The Lycos web explorer searches the World Wide Web every day (including Gopher and FTP space), building a database of all the web pages it finds. The index is updated weekly. The search engine provides retrieval from this catalog, taking a user's query and returning a sorted list of hits, sorted by match score.
WebCrawler: is one of the most popular, fastest and easy-to-use Internet search tool available in the market today.
InfoSeek: "the most powerful and popular way to search the Web" offers full searches for commercial cleints, but anyone can use its search engine for a free 10 item search. The results are ranked and include a short description
World Wide Web Worm: allows you to locate almost any WWW hypertext or URL. WWWW provides four types of search databases: citation hypertext, citation addresses (URL), HTML titles and HTML addresses.
ALIWEB: searches the web and returns a ranked list with descriptions.
RBSE Spider: This index is a collection of url references built up and indexed with a hacked version of WAIS. The index is constructed by a spider that walks the web, building a graph in an Oracle database, and WAIS indexing the full text of the document.

Open Submission Sites- Yellow Pages

These sites allow public announcing of their web sites and then in turn , you can search their collections.

New Rider's Yellow Pages: can be searched by keyword or subject category
Virtual Yellow Pages: is a comprehensive and easy to use directory of Web sites and information. The VYPŽ uses a revolutionary patented search engine that gives you the power of natural language concept searching as well as keyword search.
What's New Too: one of the most up-to-date and fastest growing resources on the net, What's New Too! posts an average of over 300 announcements daily, all within 36 hours of submission. Search the collection based uopn desired interest area, length of description, and date of announcement

Search Big, Major Sites

Many of the large volume/high traffic web sites are worth searching, too.

Yahoo: Yahoo is a hierarchical subject-oriented catalogue for the World Wide Web and Internet.
TradeWave Galaxy: searches the extensive collections of the TradeWave Galaxy (formerly known as EINet). It references a large number of Web documents from around the world, including the home pages of most of the world's Web servers.

W³ InfoPage

Maricopa Center for Learning and Instruction (MCLI)
Maricopa County Community College District

The Internet Connection at MCLI is Alan Levine --}
Comments to levine@maricopa.edu

URL: http://www.mcli.dist.maricopa.edu/w3info/