Yahoo! Adopts Robots.txt Tag To Block Specific Content

Over at Search Engine Land a post says Yahoo! has adopted the Robots nocontent tag whose function is to block a certain part of your Website which you don't want to be included in an index.

"The new robots-nocontent tag now allows you to tell Yahoo to ignore the clutter. Simply use the tag (technically, it's an attribute) to surround text you do NOT want included in searchable content within Yahoo."

Priyank Garg at Yahoo's official blog writes, "This tag is really about our crawler focusing on the main content of your page and targeting the right pages on your site for specific search queries. Since a particular source is limited to the number of times it appears in the top ten, it's important that the proper matching and targeting occur in order to increase both the traffic as well as the conversion on your site. It also improves the abstracts for your pages in results by omitting unrelated text from search result summaries." The reason for the latest move being “we saw that it was compatible with existing standards and microformats and that makes it easier to gather broader support, including from the other search engines.”

Read "Search Illustrated: Blocking Search Engines With Robots.txt" for a comprehensive understanding of functions and usage of robots tag.

Click here to subscribe to our RSS feed to get a daily digest of news around search engine industry. PageTraffic SEO Blog is updated four times a day and is ranked as one of the best search engine resources blog by Pandia!