timeouts that never time out, aka 'the last problem I'm having...'

Luis Villa wrote:
> On 6/14/06, Sam Ruby <rubysintertwingly.net> wrote:
>> Luis Villa wrote:
>> > I'm still frequently having (which I did not
have before Sam started
>> > mucking around ;) web calls which never time
out; by 'never' I mean 'I
>> > left it sitting for 10 days by accident and it
still didn't time out'.
>> > It happens unpredictably, so I'm not sure
what is the cause yet, but I
>> > figured that while I'm trying to debug I'd
put it out there and see if
>> > anyone else is seeing it more
predictably/regularly than I am.
>>
>> FeedParser used to set timeouts globally on
sockets.  This broke
>> applications like Chandler which has some of their
own sockets - some of
>> which weren't merely used to make requests, some
of which are *waiting*
>> for requests.
>>
>> Thus, this logic was removed from FeedParser by
Mark Pilgrim.  I added
>> back in the identical logic (as well as adding some
logic and
>> configuration) to Planet.
>>
>> If you specify "log_level = DEBUG" in
your config.ini, you should see
>> something comforting in the top of the output, for
example:
>>
>> htt
p://planet.intertwingly.net/berkman/planet.log
> 
> I'm not seeing anything like that in either your
branch or Jeff's
> branch; I'll try investigating some more and see what
the problem is.
> Do you know offhand what revision you put that in?

revno: 37
committer: Jeff Waugh <jdubsvengali>
branch nick: trunk
timestamp: Wed 2006-05-10 00:16:42 +1000
message:
   merge from sam
    
------------------------------------------------------------
     merged: rubysintertwingly.net-20060509120437-841adf1934c62873
     committer: Sam Ruby <rubysintertwingly.net>
     branch nick: planet
     timestamp: Tue 2006-05-09 08:04:37 -0400
     message:
       (re-)implement request timeouts.  Also only process
permanent 
redirects if      a feed with entries found.

>> Of course, if you insist on running a FrankenPlanet
with only selected
>> revisions from various places, well then you will
likely find that life
>> is like a box of chocolates.
> 
> I have never once run a frankenplanet; I'm well aware
what mixing and
> matching patches does to a system. I'm pushed in that
direction
> *today* because I can't install new dependencies on
that !#!#ing old
> RHEL3 box, but as both you and Jeff pointed out in the
other thread,
> that just isn't going to work, so... <shrug> I
guess I'm just stuck
> with a mildly broken planet for a while. I'm very
appreciative of your
> work, and I'll be updating to your latest stuff on my
home server
> shortly.

If by new dependency, you are talking about Python 2.4, then
that's a 
bug that simply hasn't been tracked down yet.

FYI: I run my planet from my home server, running Ubuntu.  I
use rsync 
to push updates to my hosted account on intertwingly, thus:

  rsync -r -p -t /home/rubys/public_html/planet/ 
intertwingly.net:web/planet.intertwingly.net/

No fuss, no muss.  All planet.intertwingly.net ever does is
serve static 
web pages.

- Sam Ruby
-- 
devel mailing list
devellists.planetplanet.org

http://lists.planetplanet.org/mailman/listinfo/devel

On 6/14/06, Sam Ruby <rubysintertwingly.net> wrote:
> Luis Villa wrote:
> > On 6/14/06, Sam Ruby <rubysintertwingly.net> wrote:
> >> Luis Villa wrote:
> >> > I'm still frequently having (which I did
not have before Sam started
> >> > mucking around ;) web calls which never
time out; by 'never' I mean 'I
> >> > left it sitting for 10 days by accident
and it still didn't time out'.
> >> > It happens unpredictably, so I'm not
sure what is the cause yet, but I
> >> > figured that while I'm trying to debug
I'd put it out there and see if
> >> > anyone else is seeing it more
predictably/regularly than I am.
> >>
> >> FeedParser used to set timeouts globally on
sockets.  This broke
> >> applications like Chandler which has some of
their own sockets - some of
> >> which weren't merely used to make requests,
some of which are *waiting*
> >> for requests.
> >>
> >> Thus, this logic was removed from FeedParser
by Mark Pilgrim.  I added
> >> back in the identical logic (as well as adding
some logic and
> >> configuration) to Planet.
> >>
> >> If you specify "log_level = DEBUG"
in your config.ini, you should see
> >> something comforting in the top of the output,
for example:
> >>
> >> htt
p://planet.intertwingly.net/berkman/planet.log
> >
> > I'm not seeing anything like that in either your
branch or Jeff's
> > branch; I'll try investigating some more and see
what the problem is.
> > Do you know offhand what revision you put that in?
>
> revno: 37
> committer: Jeff Waugh <jdubsvengali>
> branch nick: trunk
> timestamp: Wed 2006-05-10 00:16:42 +1000
> message:
>    merge from sam
>     
------------------------------------------------------------
>      merged: rubysintertwingly.net-20060509120437-841adf1934c62873
>      committer: Sam Ruby <rubysintertwingly.net>
>      branch nick: planet
>      timestamp: Tue 2006-05-09 08:04:37 -0400
>      message:
>        (re-)implement request timeouts.  Also only
process permanent
> redirects if      a feed with entries found.

Hrm. So I was wrong; I *am* seeing that message (20
seconds), but I'm
still getting regular hangs (with both your branch, on my
home/Ubuntu
machine, and Jeff's branch, on the work/RHEL3 machine).
I'll keep
investigating.

> >> Of course, if you insist on running a
FrankenPlanet with only selected
> >> revisions from various places, well then you
will likely find that life
> >> is like a box of chocolates.
> >
> > I have never once run a frankenplanet; I'm well
aware what mixing and
> > matching patches does to a system. I'm pushed in
that direction
> > *today* because I can't install new dependencies
on that !#!#ing old
> > RHEL3 box, but as both you and Jeff pointed out in
the other thread,
> > that just isn't going to work, so...
<shrug> I guess I'm just stuck
> > with a mildly broken planet for a while. I'm very
appreciative of your
> > work, and I'll be updating to your latest stuff
on my home server
> > shortly.
>
> If by new dependency, you are talking about Python 2.4,
then that's a
> bug that simply hasn't been tracked down yet.

No, it's not just a bug, it is a dependency:

from htmlentitydefs import name2codepoint

fails in python < 2.3.

> FYI: I run my planet from my home server, running
Ubuntu.

Thanks for the tip, but I'm leaving this job in a week, so
making them
depend on my home server is not really an option.  I am
trying to
move it to an Ubuntu box we've got, but... urgh, long story
best told
over a beer when I don't want to stab a co-worker in the
face 

Thanks again for your time and patience, Sam- it has been
nice to see
planet development get a shot of adrenaline again.
Luis
-- 
devel mailing list
devellists.planetplanet.org

http://lists.planetplanet.org/mailman/listinfo/devel

Luis Villa wrote:
>>
>> If by new dependency, you are talking about Python
2.4, then that's a
>> bug that simply hasn't been tracked down yet.
> 
> No, it's not just a bug, it is a dependency:
> 
> from htmlentitydefs import name2codepoint
> 
> fails in python < 2.3.

That's a bug.  I've committed a fix for that on my branch,
and will work 
to get it into the next BeautifulSoup release.

Unfortunately, that's not *the* bug.  Running
feedparser.parse by itself 
on Lessig's feed produces truncated content.

- Sam Ruby
-- 
devel mailing list
devellists.planetplanet.org

http://lists.planetplanet.org/mailman/listinfo/devel

<quote who="Luis Villa">

> Hrm. So I was wrong; I *am* seeing that message (20
seconds), but I'm
> still getting regular hangs (with both your branch, on
my home/Ubuntu
> machine, and Jeff's branch, on the work/RHEL3
machine). I'll keep
> investigating.

That was the fix that stopped Planet GNOME from hanging, btw
(and it runs on
RHEL3).

- Jeff

-- 
linux.conf.au 2007: Sydney, Australia           http://lca2007.linux.org
.au/
 
    "I'm taking no part in your merry 5-way
clusterfuck - sort that mess
                 out between yourselves." - Alexander
Viro
-- 
devel mailing list
devellists.planetplanet.org

http://lists.planetplanet.org/mailman/listinfo/devel