Yahoo!, Google, Microsoft Clarify robots.txt Support
Today, Google, Yahoo!, and Microsoft have come together to post details of how each of them support robots.txt and the robots meta tag. While their posts use terms like "collaboration" and "working together", they haven't joined together to implement a new standard (as they did with sitemaps.org). Rather, they are simply making a joint stand in messaging that robots.txt is the standard way of blocking search engine robot access to web sites. They have identified a core set of robots.txt and robots meta tag directives that all three engines support:
For robots.txt, they all suport:
* Disallow
* Allow
* use of wildcards
* Sitemap location
For robots meta tags, they all support:
* noindex
* nofollow
* noarchive
* nosnippet
* noodpt
Google and Yahoo! already supported and documented each of these and Microsoft supported most of them before this announcement. In their posts, they also list the directives they support that may not be supported by the other engines.
We are providing visibility on Internet and covering online Advertising & Marketing and internet marketing news on Web.
Translate
Subscribe to:
Post Comments (Atom)
Create Content To #drawthecrowds
Create content to #drawthecrowds Content need to be relevant. If you’re looking to (#drawthecrowd) to your site this year, you ...
-
reportonbusiness.com: Microsoft, Yahoo would create advertising titan If the blockbuster takeover of Yahoo Inc. by Microsoft Corp. is allowe...
-
Google The Stealth Social Network? Google's plan to socialize its various applications continues. Google Operating System has spotted co...
-
Google's Supplemental Results What exactly are Google's supplemental results, why does everyone fear getting indexed in the suppleme...
No comments:
Post a Comment