[gopher] Whole lotta hits
Jacob Dahl Pind
rachael at telefisk.org
Wed Sep 14 11:05:36 UTC 2011
On Tue, 13 Sep 2011, Jeff W wrote:
> Anyone else seeing unusually large numbers of hits on their servers
> today? Normally I get like a dozen or so; yesterday: 491, today:160.
> Most are from various eastern european countries or indirect via what
> looks to be some sort of WWW-gopher gateway at bytemark7.spotonnet.co.uk.
>> From the log it looks like intentional hammering to try to crash the
> server (it didn't).
>
> Jeff
I think it migth be baidubot, its quiet aggressiv in its crawling this
days, it crawlers my http sites every day, with 30-40sec between access.
But google and bing are also crawling through the gopherproxy.org at
bytemark7.spotonnet.co.uk.
As we cant ratelimit, or any other robots.txt things when those web
crawlers are hitting us through those proxies, I have opped for simply
redirecting access from those proxies to another selector.
is in the TELEFISK_FILTER ifdef in this patch
gopher://telefisk.org/0/gopher/gophernicus-1.2-beta-telefisk.diff
Regards
--
Jacob Dahl Pind | telefisk.org | fidonet 2:230/38.8
More information about the Gopher-Project
mailing list