Spiders and System Performance
I’ve noticed a couple of occasions where the server has been slogged to the floor. This would appear to be due to search engines unfriendly spiders spamming the site with lots of requests in a short period. Altavista is particularly dastardly for this, and is presently in the banned list that doesn’t even know the …