By Cynthia Morgan

I've never yearned for my own personal spider, but then I never thought I'd spend half my time in a Web either. The personal spiders of CyberPilot Pro provide me with valuable Web site maps I can use to drill down to the information I really need.

Spiders, crawlers and bots, also known as intelligent agents, automate what has to be the worst Web task: finding fresh information. Large search sites such as Alta Vista, Lycos and Yahoo use them to "mine" Web documents for updates, following links to new pages and returning with up-to-date Web site outlines.

Similarly, you can point CyberPilot's personal spiders at a Web site and they'll assemble a map of the site's graphics, links, documents, multimedia elements, services and gateways. In collapsible outline form, the resulting NetCarta WebMap can be filtered and searched offline. You can update a site map, gather site statistics and view your map according to links, page element types or document names. It's a great tool for examining a Web or intranet site's structure, organization and content.

Put enough spiders on a server and you'll severely impact performance. CyberPilot gets around that by letting you link your WebMap to your Website; other Web mapping tools simply retrieve your map. You also can customize a map to hide certain pages or add labels to make your page outlines more understandable.

Exploring a Web site thoroughly is a slow process; CyberPilot lets you stop a crawl, if neccessary. In my tests on our own WinMag Web site, I "crawled" 153 pages using a 56Kb per second Internet connection. CyberPilot mapped 62 images, three gateways, 406 links to other pages/sites and four accesses to outside Internet services. That crawl took about 10 minutes. It took a bit over a half hour when I chose to verify links. Performance, of course, depends on bandwidth.