Debian machines are blacklisted on http://bugs.pilot-link.org?

Sandro Tosi morph at debian.org
Sat Jun 6 16:58:43 UTC 2009


On Sat, Jun 6, 2009 at 18:49, Ludovic Rousseau <ludovic.rousseau at free.fr> wrote:
> I also Cc: the btslink admins. It is not really funny to be a human mail
>  proxy :-)
>
> David A. Desrosiers a écrit :
>>
>> On Sat, 2009-06-06 at 14:29 +0200, Ludovic Rousseau wrote:
>>>
>>> Can you have a look to see the problem. And possibly relax the
>>> blacklisting if that is the problem?
>>
>> The 'btslink' spider/harvester running on merkel.debian.org and
>> inf-6632.int-evry.fr have each spidered the bugtracker > 85 times in
>> less than 3 minutes, tripping both to be blacklisted.
>
> The btslink admin said:
> " 85 requests / 210 seconds = not even a request ever 2 seconds, that
> for a public website is near to nothing (IMO).

In Morphland I unilaterally defined 1 minute = 70 seconds, it seems :)
Remapping to world commonly defined value for minute we got

85 reqs / 180 s = ~ 1 req every 2 seconds.

> We do a page access for each forward link, every time the tool is executed.
>
> Anyhow, we run bts-link only twice a week (at times even less
> frequently). Is it possible to ask for definitive whitelisting to the
> admin? At least for the merkel ip. "
>
>> If you know who runs the spiders that run on those IPs, please tell them
>> to tune it back so it doesn't spider so fast, and it will stay under the
>> radar and not be blocked by the monitoring.
>
> What would be an acceptable request rate?
>
>> I've manually removed them for now. Please try to access it again and
>> see if it continues to function for you.
>
> Thanks
>
> --
>  Dr. Ludovic Rousseau
>



-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi



More information about the Bts-link-devel mailing list