Tuesday, February 19, 2008

Yahoo Block Google From Delicious Bookmarks

Yahoo Block Google From Delicious Bookmarks:

During the weekend, Yahoo!'s social bookmarking property del.icio.us seems to have prevented robots of other search engines including Google from spidering the website or fishing out fresh web pages, bookmarks and websites, observes Collin. It was clear that it was not a simple robots.txt exclusion, rather it was a 404 response that was being shown based on who the requesting User-Agent was.

From Internet Marketing

No comments: