List Info

Thread: Mailing List nutch-agent Reports of Bots Submitting Forms




Mailing List nutch-agent Reports of Bots Submitting Forms
user name
2006-05-30 21:03:35
Ken Krugler wrote:
>>> 2. Are the Nutch Devs replying to the emails
sent to this list? I could
>>> understand if they are replying off-list, but
to an outside observer 
>>> such as
>>> myself it appears as though webmasters are not
getting many replies 
>>> to their
>>> inqueries.
>>
>>
>> I can speak for myself only .. I'm not tracking
that list. What about 
>> others?

Folks who are running a nutch-based crawler that provides
this email 
address as the contact address should subscribe to this list
and respond 
to messages, especially those which may have been caused by
their 
crawler.  Others are also encouraged to subscribe and help
respond to 
messages here, as a bad reputation for the crawler affects
the whole 
project.  This list is actually fairly low-volume.

> This brings up an issue I've been thinking about. It
might make sense to 
> require everybody set the user-agent string, versus it
having default 
> values that point to Nutch.
> 
> The first time you run Nutch, it would display an error
re the 
> user-agent string not being set, but if the
instructions for how to do 
> this were explicit, this wouldn't be much of a
hardship for anybody 
> trying it out.

+1

That would be a better solution.

Doug
[1]

about | contact  Other archives ( Real Estate discussion Medical topics )