Yahoo!, Google, Microsoft Clarify robots.txt Support
Today, Google, Yahoo!, and Microsoft have come together to post details of how each of them support robots.txt and the robots meta tag. While their posts use terms like "collaboration" and "working together", they haven't joined together to implement a new standard (as they did with sitemaps.org). Rather, they are simply making a joint stand in messaging that robots.txt is the standard way of blocking search engine robot access to web sites. They have identified a core set of robots.txt and robots meta tag directives that all three engines support:
For robots.txt, they all suport:
* Disallow
* Allow
* use of wildcards
* Sitemap location
For robots meta tags, they all support:
* noindex
* nofollow
* noarchive
* nosnippet
* noodpt
Google and Yahoo! already supported and documented each of these and Microsoft supported most of them before this announcement. In their posts, they also list the directives they support that may not be supported by the other engines.
We are providing visibility on Internet and covering online Advertising & Marketing and internet marketing news on Web.
Translate
Subscribe to:
Post Comments (Atom)
Create Content To #drawthecrowds
Create content to #drawthecrowds Content need to be relevant. If you’re looking to (#drawthecrowd) to your site this year, you ...
-
Create content to #drawthecrowds Content need to be relevant. If you’re looking to (#drawthecrowd) to your site this year, you ...
-
Where does Google go next? - May. 12, 2008 No matter what your attitude towards the Google-Yahoo-Microsoft triangle may be, such a title def...
No comments:
Post a Comment