Oh yeah...
Look what happened to Blanket the other day:
I had to take it offline, it was killing the server. I haven’t got around to looking to see what caused such a spike, but it’s definitely got something to do with search engines. I have moved the results page to a sub-url (/site/) and banned search engines from spidering that URL. It seems to have worked.
Hmmm, on second thoughts, a quick grep | awk | sort | uniq
at the log files makes it look like all 1.2m of the hits in that spike came from 130.94.202.16, which is just a plain old box in Verio’s network somewhere, one that is now down. Seems like whoever they are/were they requested some 812,802 unique pages through the blanket, some of them up to 300 times. Either they had a sudden Johnny-5 like thirst for knowledge or there’s something going on…