Tuesday, November 11, 2008

Bad, Bad Bot!!!


The definition of a "bad bot" varies depending on who you ask, but most would agree that bots are actually spiders that do a lot more harm than good on your site. Bad bots typically ignore the wishes of your robots.txt file, so you'll want to ban them using means such as .htaccess. But the tricky part is identifying the bot.

How to identify a bad bot: If you aren't already doing this, you really need to start checking your logs to see who is visiting your site, how often and how long they are there. From your logs you can start identifying the bad guys. There's too many to list here, but if you Google "how to identify bad bots" you will find blogs that list the bad guys and tools to help you protect your site.

Just keep in mind that having a search engine bot on your site is good for your site as they crawl and index your website faster; but you need to take the time to identify the bad bots and keep them from crawling your site. If you have someone doing optimization for you, you should make sure that this is part of their service offerings.

Want to learn more about SEO? Visit us on the web at mandarichmediagroup.com.

0 comments:

ShareThis Info