Web Application need to call a perl program.

             Hi,

I would like to know how can I make a daemon program in
perl, say DP . The
said program need to call a a program say A, which will to
parse huge text
file. The run time of A may vary say 5 to 6 hrs on average.
There will be a
lot of execution request reach the daemon program(DP)
parsing different text
files. so I need this daemon program to schedule each
request in proper
interval of time.

I need to call the program from a web interface. so that the
DP will take
the request from the web interface and invoke it.Since the
web user cant
wait for long time online, could any body tell me a idea how
a perl program
can survive in this scenario.

             Regards
             Shyam

 ------------------------------


[Non-text portions of this message have been removed]



Unsubscribing info is here: h
ttp://help.yahoo.com/help/us/groups/groups-32.html 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://g
roups.yahoo.com/group/perl-beginner/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http
://groups.yahoo.com/group/perl-beginner/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:perl-beginner-digest@yahoogroups.com 
    mailto:perl-beginner-fullfeatured@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    perl-beginner-unsubscribe@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.c
om/info/terms/
 

>>>>> "Shyam" == Shyam S
<pl.shyamgmail.com> writes:

Shyam> I would like to know how can I make a daemon
program in perl, say DP
Shyam> . The said program need to call a a program say A,
which will to parse
Shyam> huge text file. The run time of A may vary say 5
to 6 hrs on
Shyam> average. There will be a lot of execution request
reach the daemon
Shyam> program(DP) parsing different text files. so I
need this daemon program
Shyam> to schedule each request in proper interval of
time.

Shyam> I need to call the program from a web interface.
so that the DP will
Shyam> take the request from the web interface and invoke
it.Since the web
Shyam> user cant wait for long time online, could any
body tell me a idea how
Shyam> a perl program can survive in this scenario.

Create a daemon using POE (poe.perl.org) which can provide
either the retail
web connection (users talk directly to an HTTP server), or a
secondary HTTP
API that gets connections from a CGI script running on the
main server.  The
daemon can schedule the tasks, run them, and then notify the
users as the
results become available.  Use POE::Wheel::Run to manage the
long "A" program.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. -
+1 503 777 0095
<merlynstonehenge.com> <URL:http://www.ston
ehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy,
etc. etc.
See PerlTraining.Stonehenge.com for onsite and
open-enrollment Perl training!


Unsubscribing info is here: h
ttp://help.yahoo.com/help/us/groups/groups-32.html 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://g
roups.yahoo.com/group/perl-beginner/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http
://groups.yahoo.com/group/perl-beginner/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:perl-beginner-digest@yahoogroups.com 
    mailto:perl-beginner-fullfeatured@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    perl-beginner-unsubscribe@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.c
om/info/terms/
 

>>>>> "merlyn" == merlyn 
<merlynstonehenge.com> writes:

merlyn> Create a daemon using POE (poe.perl.org) which
can provide either the
merlyn> retail web connection (users talk directly to an
HTTP server), or a
merlyn> secondary HTTP API that gets connections from a
CGI script running on
merlyn> the main server.  The daemon can schedule the
tasks, run them, and
merlyn> then notify the users as the results become
available.  Use
merlyn> POE::Wheel::Run to manage the long "A"
program.

I should add that the reason this is familiar is that I just
designed and
implemented a system like this (scheduler system using POE
and a web API) for
one of my clients last summer.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. -
+1 503 777 0095
<merlynstonehenge.com> <URL:http://www.ston
ehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy,
etc. etc.
See PerlTraining.Stonehenge.com for onsite and
open-enrollment Perl training!


Unsubscribing info is here: h
ttp://help.yahoo.com/help/us/groups/groups-32.html 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://g
roups.yahoo.com/group/perl-beginner/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http
://groups.yahoo.com/group/perl-beginner/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:perl-beginner-digest@yahoogroups.com 
    mailto:perl-beginner-fullfeatured@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    perl-beginner-unsubscribe@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.c
om/info/terms/
 

On 2/12/07, Randal L. Schwartz <merlynstonehenge.com> wrote:
>
> >>>>> "merlyn" == merlyn 
<merlynstonehenge.com> writes:
>
>
> I should add that the reason this is familiar is that I
just designed and
> implemented a system like this (scheduler system using
POE and a web API)
> for
> one of my clients last summer.
>

Hi Schwartz.

I am so thankful for your reply. My program flow is like
this.


{php frontend} -->{perl cgi} <----->{ daemon /
scheduler }<-> { perl parser}

{php frontend }-from here the web user  enter the details of
the details he
want to process. Once he clicked the submit button. He will
be acknowledged
that,his request will be processed in such time and the
estimated run time
is some time(I think to calculate the estimated time in some
predefined
logics.)

The  data and specifications are suppose to be stored in a
database. Since
it is required for later report generatio/viewing.

Since multiple user can access the system in a time. I wish
to control the
{perl parser} module to be do four threads at a time. Rest
may be kept in
the database. Since the Client may close the window once he
get the info of
at which estimated time he can get the result.


So I would like to know whether the cgi can get the total
number of request
that is queued earlier in database via {damon/scheduler}
module and will be
able to display an estimated time by calculating on the
number of previous
records and display the result at {php frontend}.

The {daemon/scheduler} need to work seamlessly to process
each of the
database request.

Can POE can do the task for me. or do I need to write socket
program that
will communicate the  { daemon / scheduler } to the cgi.

Since I am new to cgi scripting I am in a big stop.please
help me.

Thanks In advance
Regards
Shyam


[Non-text portions of this message have been removed]



Unsubscribing info is here: h
ttp://help.yahoo.com/help/us/groups/groups-32.html 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://g
roups.yahoo.com/group/perl-beginner/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http
://groups.yahoo.com/group/perl-beginner/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:perl-beginner-digest@yahoogroups.com 
    mailto:perl-beginner-fullfeatured@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    perl-beginner-unsubscribe@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.c
om/info/terms/
 

>>>>> "Shyam" == Shyam S
<pl.shyamgmail.com> writes:

Shyam> Can POE can do the task for me. or do I need to
write socket program
Shyam> that will communicate the { daemon / scheduler }
to the cgi.

Well, there are two ways, as I said in the previous note:

browser -HTTP-> light CGI -HTTP-> POE Daemon
-forking-> tasks

browser -HTTP-> POE Daemon -forking-> tasks

The first involves creating a CGI script that gathers and
validates the
parameters, then uses LWP::Simple to talk to the POE Daemon
to actually post
the request or fetch the status.

The second is a bit simpler, but would require some way for
the browser to
find the POE daemon's URL, perhaps linked from a main
server's page,
and you don't want a complicated page to deliver to the
browser.

And yes, POE is a general framework, and everything you're
asking can
be done in a straightforward way.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. -
+1 503 777 0095
<merlynstonehenge.com> <URL:http://www.ston
ehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy,
etc. etc.
See PerlTraining.Stonehenge.com for onsite and
open-enrollment Perl training!


Unsubscribing info is here: h
ttp://help.yahoo.com/help/us/groups/groups-32.html 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://g
roups.yahoo.com/group/perl-beginner/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http
://groups.yahoo.com/group/perl-beginner/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:perl-beginner-digest@yahoogroups.com 
    mailto:perl-beginner-fullfeatured@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    perl-beginner-unsubscribe@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.c
om/info/terms/
 

Hi Schwartz.

Thank you very much for your timely help!!.

I had gone through the POE docs. Since I am very new to the
CGI please
excuse me if I am asking a wrong question. I am not able to
create the
scripts as I am confused and stuck up in the design phase
itself. Since http
is a stateless transfer protocol, how can POE hold the http
request
sessions?.

Well, there are two ways, as I said in the previous note:

browser -HTTP-> light CGI -HTTP-> POE Daemon
-forking-> tasks
browser -HTTP-> POE Daemon -forking-> tasks

The first involves creating a CGI script that gathers and
validates the
parameters, then uses LWP::Simple to talk to the POE Daemon
to actually post
the request or fetch the status.


With reference your first suggestion, I think, the http page
is transferring
data to CGI using POST method. But once the submissions are
over how can the
web page get result from the CGI?  As you said, LWP::Simple
can get the
values from the request. But I am querying some database and
doing
calculations, the response time can vary, say duration of 5
minutes. Will
the module hold it for that time?



The second is a bit simpler, but would require some way for
the browser to
find the POE daemon's URL, perhaps linked from a main
server's page, and you
don't want a complicated page to deliver to the browser.


 Also, I am confused, how to communicate between lwp and the
daemon. Can we
get the status, like how many child processes are currently
running on the
server?. If the request exceeds how can we queue up the
request?. Do we need
file or database operations since there are chances for the
user at http end
to close the browsing window or losing a network
connection?

And yes, POE is a general framework, and everything you're
asking can be
done in a straightforward way.


Since each request from LWP  to daemon is a new connection
how can we call
the daemon, since daemon  will be an independent service, do
we need any
socket connection for that?. Please advice.

Please correct me if I am having a wrong understanding on
this.  I am not
able to proceed with the design as I am not clear with the
underlying
processes and concepts.

Also could you please send me links that describes POE
operations with
example, if you have any. Your guidance is highly
appreciated. Once again,
thanks a lot!!


Have a Nice day!

Thanks and Regards
Shyam


[Non-text portions of this message have been removed]



Unsubscribing info is here: h
ttp://help.yahoo.com/help/us/groups/groups-32.html 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://g
roups.yahoo.com/group/perl-beginner/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http
://groups.yahoo.com/group/perl-beginner/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:perl-beginner-digest@yahoogroups.com 
    mailto:perl-beginner-fullfeatured@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    perl-beginner-unsubscribe@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.c
om/info/terms/