Website review: Stopping bad robots with honeytr...
johnblackmore discovered this in Web Development
•1 reviews since Aug 9, 2007
web-design
•davidnaylor.co.uk/stopping-bad-robots-with-ho...
People who like this website
StumbleUpon is the best way to discover great web sites, videos, photos, blogs and more - based on your interests.
Everything is submitted and rated by the community. Discover, share and review the best of the web!
Reviews of this website
johnblackmore discovered 9 months ago- Most search engines and spiders will responsibly crawl your site and only access the pages you define in robots.txt This page explains how to log and then ban the spiders that visit your site and don't follow your robots.txt rules.
