Digital Develops Internet's First 'Super Spider'
Next-Generation Technology Enables Most Comprehensive Index of WWW Text at Unprecedented Speed
Millions of Web Users Expected to 'Kick the Tires' in Trial Offer
PALO ALTO, CA -- December 15, 1995 -- Digital Equipment Corporation today introduced the Internet's first "super spider" software, as part of the most advanced information search and indexing technology available for the World Wide Web. Blazing fast, it conducts the most comprehensive search of the entire Web text at speeds up to 100 times faster than spiders used in conventional information search services.
Under development at Digital's Corporate Research Group here, the technology promises to surpass the limitations of current information services by delivering the most complete, precise, and up-to-date information of the Web's entire text. The technology's super spider and super indexer employ next-generation software and advanced networking, powered by the highest performing 64-bit Alpha computers.
The advanced technology is set to undergo its toughest testing starting today (Friday, December 15), as Digital makes a "beta" version available to tens of millions of Web users worldwide. Users can test the super spider's Web index and provide feedback through Digital's Internet address at: http://www.altavista.digital.com/
Thousands of Digital employees worldwide are testing the new technology behind the corporation's Internet firewall.
Cyber Search
"Finding a 'cyber needle' in an ever-growing 'cyber haystack' has long been a dream of Web users," said Samuel H. Fuller, vice president, Corporate Research. "This technology is a major step in that direction. It features the most complete and precise Web index. In fact, it's the only one with the potential to keep up with the phenomenal growth of the Web," he added.
Fuller said the super spider has been crawling the Web at up to 2.5 million pages per day, and already has found and indexed more information than any other spider or crawling service. It is enroute to finding every page and indexing every word of text on the Web. "We believe users will find Web pages they have never seen through any other indexing technology," he added.
Digital said the technology has implications for market research because of its precise search capabilities. For example, users for the first time can pinpoint the exact number of "hot-links" or references to their home page from other Web sites.
What It Does
The super spider creates and dispatches a "brood of spiders" that crawls the entire Web. Second-generation scalable software simultaneously locates and indexes text as it finds Web pages. A powerful search engine enables Web users to conduct precise searches for specific information by looking for phrases, specifying key words, using case-sensitive matches, and restricting searches to titles or other parts of a document. Network connectivity is provided through Digital's fiber optic and high-speed network switching technologies. Ultra-high-speed search and indexing is enabled by Digital's Alpha computers, the industry's highest-performing 64-bit servers. This powerful combination of software, networking, and hardware overcomes the limitations of conventional spider/indexing technologies. Patents are pending.
Fuller said current Internet search and indexing technology is much slower due to the constraints of 32-bit technology. "Users do not receive all of the information available on the entire Web, and frequently miss entire documents. Digital's second-generation technology indexes every word of every document," he added.
"Businesses will find this powerful indexing and search capability will assist them in instantaneously locating information on markets, customers, and competitors," said Rose Ann Giordano, vice president of Digital's Internet Business Group. "This new capability complements Digital's portfolio of industrial-strength Internet servers, security, and service offerings."
Digital Equipment Corporation is the world's leader in open client/server solutions from personal computing to integrated worldwide information systems. Digital's scalable Alpha platforms, storage, networking, software and services, together with industry-focused solutions from business partners, help organizations compete and win in today's global marketplace. Specific information on Digital's Alpha and Intel platforms, storage, networking, software, and services is available on the Internet and can be accessed through the Digital home page: http://www.digital.com
####
Note to Editors: Digital and the Digital logo are trademarks of Digital Equipment Corporation.
Editorial contacts:
Ed Canty 508.264.6672
Sarah Miller 508.264.6673
Tom Madden 508.264.6675
Don Bradley 508.264.6678
Patrick Ward 508.264.6696
Ethel Kaiden 508.486.2814