Re: Future Perl development

[and y'all thought that this thread was dead. Well, no, it's
undead. ]

On Wed, Feb 07, 2007 at 04:02:00PM -0800, Jan Dubois wrote:

> Indeed. Surprisingly there seems to be more industry
funded Tcl
> development work than there is for Perl.  I'm just
guessing here, but my
> feeling about it is that the Tcl (enterprise) users
know they have a
> small community, so stuff they want won't get done
unless they pay
> someone to do it.  For Perl there seems to be a huge
community, so the
> expectation that something might get done anyways is
that much higher.

I forgot to say:

I think that from the "outside" this is a
reasonable assumption to make.
But it goes wrong because the huge community around Perl is
busy doing things
outside the core. I believe that the rate of code upload to
CPAN is
increasing.

Work on the core, particularly *work* (defined below, as
opposed to "fun")
is slowing.

On Thu, Feb 08, 2007 at 10:12:15AM +0100, Richard Foley
wrote:

> It would be a good target for a fund-raising group
though.  Specifically 
> tasked with:
> 	
> 	1. Raising corporate awareness of the _current_ perl,

> 
> 	2. Raising funds from these companies for further
development/support.
> 
> 	3. Feeding those funds directly to specific core
tasks.
> 
> Given the recent discussions on this list, it might
seem like an uphill 
> battle, but isn't that the point?  Something along
these lines might even 
> have a positive knock-on side-effect for perl in
general.

If the three steps can be made to loop into a virtuous
circle, that would
be good. But following chats on IRC and at the German Perl
Workshop, I'm
not sure how to make step 3 work.

Specifically, the core tasks that need doing are

1: answering bug reports
2: reviewing and where appropriate applying patches
3: fixing bugs
4: getting things release ready

and

1: I don't think that these constitute a full time job. I
think that steady
   state maint is about 1 day per week, and blead about 1
day per week.

2: The tasks above are not particularly fun jobs.

3: I'm not sure that there are really sufficient tasks in
the perl todo to
   keep someone busy for the other 3 days a week (for that
long) even if
   full time funding were to appear.

4: I'm unaware of any one person who is both capable and
keen to do work on
   the core full time (but alone) if funding were
available.


It has been suggested that it could be funded by offering
perl support
contracts. But in turn

1: I'm not sure how much that overlaps with what ActiveState
already do
2: As a one-man-band the thought of selling a "panic
level" service on the
   core would scare me because
   a: There are parts of the core I don't know at all well.
Without help from
      other (unpaid) people round here I'd be stuck
   b: Sod's law will ensure that sooner or later merchant
bank number 2 will
      phone up with a crisis whilst one is working flat out
on bank number 1's
      problem.

      In theory you solve this by having backup people. But
the number of
      people with core knowledge good enough currently seems
so small that
      you'd need pretty much all the active committers.
   (and note c: I don't actually want to do this, and I'm
aware of other
    active committers who don't, but not aware of any who
do)
3: There comes a conflict between the quick fix to get a
firm moving, and the
   best (portable) solution to commit to core. So does one
end up supporting
   many fragmented perl installations, or instead end up
compromising the
   core with rushed commits?


Maybe I'm pessimistic. But I'm aware from a friend that at
financial
institutions, the financial risk or losses from not having
some software
running (in his area because it's not yet written) can be
around $1,000,000
per day. The stakes are high.

Nicholas Clark

On Feb 27, 2007, at 12:32 PM, Nicholas Clark wrote:

>  But I'm aware from a friend that at financial
> institutions, the financial risk or losses from not
having some  
> software
> running (in his area because it's not yet written) can
be around  
> $1,000,000
> per day. The stakes are high.

That's why Yahoo employs Rasmus Lerdorf (of PHP fame).

I talked with Lerdorf at last year's OSCON, and he said that
Yahoo  
aggressively seeks to hire the developers of key open
software  
tools.  According to him, if you want to sell Yahoo software
services  
and you have an open source competitor, you're screwed --
because  
they'd much rather go with the open source solution and hire
the  
principle devs.

Maybe y'all ought to break Perl once.  Then all of a sudden,
some  
company will lose a million bucks and realize how insane it
is that  
they didn't keep a core Perl programmer on staff as an
insurance policy.

Marvin Humphrey
Rectangular Research
http://www.rectangular.co
m/

On 2/27/07, Nicholas Clark <nickccl4.org> wrote:
> [and y'all thought that this thread was dead. Well, no,
it's undead. ]
> 3: I'm not sure that there are really sufficient tasks
in the perl todo to
>    keep someone busy for the other 3 days a week (for
that long) even if
>    full time funding were to appear.

I think perltodo could be beefed up. I know in my area of
interest
there is quite a bit of work that could be done. Heres an
off the top
of my head list in no particular order.

1. Improve unicode charclass representation
2. Split match results out of the regexp structure
3. rework compilation so that further and more aggressive
optimisations are possible
4. trifurcate the match routine into 3,
octect-octet/ut8-utf8/octect-utf8
5. split up study_chunk in multiple passes that do different
things (see 3)
6. rework exact nodes so they store octect and utf8
representations
when necessary
7. rework trie code so that it doesnt use a hash for
handling utf8
charclass lookups
8. rework trie code so that it doesnt use a compressed
table
representation when the table is sufficiently small.
9. rework aho-corasick logic so it doesnt get invoked when
the string
being matched is very short.
10. implement reverse regops
11. change the way variable binding in regexps is done
12. fix the way (?{...}) (??{...}) interact with PL_curpm
13. introduce ways to handle safe signals during long
matches
14. introdcue resource limits on regexps so that they can
consume only
a certain amount of memory or run for a certain amount of
regops
15. add limited DFA capabilities for degenerate type
patterns.
16. clean up source code so that all regexp related code are
grouped
together in clearly identifiable groups.
17. make $REGERROR and $REGMARK dynamically scoped
18. rework the initialization stage of regmatch so that
initialization
is only done when necessary
19. improve the way regexp.extflags are ordered and
processed so that
accessing the flags is more efficient.
20. enhance pos code so that we store both character and
byte offsets
when matching a utf8 string.
21. cache charclass properties for localized matches.
22. come up with a way of mapping capture buffers into the
perl 6
nested capturing scheme and introduce a way to access them
in that
way.
23. implement char class set operations
24. super-linear cache for recursive regexps.

Thats just what i can come up with off the top of my head.

Also in the general codebase (again not in any particular
order):

1. mg_find is heavily overused throughout the code. Many
routines that
ive looked into throughout the code base do repeated
searches though
the magic linked list in the same routine. there must be a
way to make
this more efficient.
2. swash code for unicode is currently half-perl, the other
half could
probably be converted.
3. look into improving codepaths for unicode logic. many
times we dont
get to the core routine without going through multiple
layers of
wrapper routines
4. rework as much of the internal logic that uses char * as
part of
its interface to take some kind of lightweight string
structure that
knows its encoding and length.

Anyway, just some ideas.

cheers,
Yves


-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Nicholas Clark wrote:

> Specifically, the core tasks that need doing are
>
> 1: answering bug reports
> 2: reviewing and where appropriate applying patches
> 3: fixing bugs
> 4: getting things release ready

5: writing the summaries

On 27/02/07, Nicholas Clark <nickccl4.org> wrote:
> 3: There comes a conflict between the quick fix to get
a firm moving, and the
>    best (portable) solution to commit to core. So does
one end up supporting
>    many fragmented perl installations, or instead end
up compromising the
>    core with rushed commits?

(that's also apparent when you look at some of the patches
linux
vendors put in their perls)

Nicholas Clark wrote:
> 3: There comes a conflict between the quick fix to get
a firm moving, and the
>    best (portable) solution to commit to core. So does
one end up supporting
>    many fragmented perl installations, or instead end
up compromising the
>    core with rushed commits?
>   

Just track them all. Cherry pick in what's good.

Sam.