Covert Crawler Descends on Web


Websites get looked at by two different kinds of visitors: the human ones who peer around, look at the graphics, think about the links and click slowly; and the spiders, those automated scanners that come in from search engines like Google, or, more ominously, from malicious attackers, competing businesses and spammers looking for e-mail addresses.

Fortunately, it has always been pretty easy to tell the difference between the two in server logs, and block unwanted or anti-social crawlers. But research presented at the Shmoo Con hacker conference here Friday may change that. Wired News: Covert Crawler Descends on Web

Linked by shanmuga Sunday, 15th January 2006 12:20AM