Do you know the feelign when sometimes there are more bots than visitors crawling over your site? You look at their websites and realize they are just some crapbots doing stuff that will not benefit you or your site at all. Instead they are just choking up your bandwidth. sometimes when this little server gets overwhelmed by requests the last requests in the log came from bots, especially baidu spider.
I decided that this has to stop!
„How to stop unwanted Crawlers, Bots and Spiders!“ weiterlesen