151.74.245.255 - - [25/Dec/2007:09:02:01 +0800] "GET /website/sell.php?member=ywtg26941&pg=2%20title= HTTP/1.1" 200 9166 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
91.117.64.108 - - [25/Dec/2007:09:02:00 +0800] "GET /website/sell.php?member=yw235532& HTTP/1.1" 200 9265 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
86.221.181.245 - - [25/Dec/2007:09:02:02 +0800] "GET /website/sell.php?member=ywtg26864&&pg=2%20title=%20title=%20title= HTTP/1.1" 200 9393 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
76.11.222.227 - - [25/Dec/2007:08:59:41 +0800] "GET /website/sell.php?member=wyltsj HTTP/1.1" 200 26007 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
91.117.64.108 - - [25/Dec/2007:09:02:00 +0800] "GET /website/sell.php?member=yw235532& HTTP/1.1" 200 9265 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
86.221.181.245 - - [25/Dec/2007:09:02:02 +0800] "GET /website/sell.php?member=ywtg26864&&pg=2%20title=%20title=%20title= HTTP/1.1" 200 9393 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
76.11.222.227 - - [25/Dec/2007:08:59:41 +0800] "GET /website/sell.php?member=wyltsj HTTP/1.1" 200 26007 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
What is MJ12bot doing on my site(s)?We do spider the Web for the purpose of building a distributed search engine with fast and efficient downloadable distributed crawler that will enable people with broadband connections to help contribute to, what we hope, will become the biggest search engine in the world.
Disallow: /可以将他拒之门外
Disallow: /
还是不行,不停在访问
189.163.82.126 - - [26/Dec/2007:11:04:15 +0800] "GET /website/new.php?member=daorenfeng&pg=2%20title=%20title=%20clas%20title=%20title=%20clas%20clas%20clas%20title=%20title=%20clas%20title=%20clas%20clas%20title=%20title=%20title=%20clas%20title=%20title=%20title=%20title=%20clas%20title=%20clas%20title=%20clas%20clas%20title=%20clas%20clas%20title=%20clas%20title=%20title=%20title=%20title=%20title=%20clas%20title=%20title=%20title=%20title=%20title=%20clas%20title=%20clas%20title=%20clas%20clas%20title=%20clas%20clas%20clas%20title=%20title=%20title=%20clas%20clas%20title=%20title=%20title=%20title=%20title=%20title=%20clas%20clas%20title=%20title=%20clas%20title=%20clas%20clas%20title=%20title=%20clas%20title=%20title=%20title=%20title=%20title=%20clas%20title=%20title=%20clas%20title=%20clas%20clas%20title=%20clas%20title=%20title=%20clas%20title=%20title=%20clas%20title=%20title=%20title=%20title=%20clas%20title=%20clas%20clas%20clas HTTP/1.1" 200 17213 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
Deny from 189.163.82.
Deny from 189.163.82.126
Allow from all
不知道你的程序有缓存没有. 每次都读数据会很慢的.