RFC: JobQueue Job Notification

Peter Edwards wrote:
> Hi Kiki
>
> I think there are two kinds of reports back.
> 1) Interim progress for a long running batch. (Not sure
how best to achieve
> that? Previously I've logged to a run file and offered
a "tail -f" web
> screen view until the user navigates away, after which
they can view again
> from a batch view web screen or command shell)
>   
Status report for jobs is an interesting idea, perhaps via
PoCo::Server::HTTP
> 2) Batch job completion report.
>   
> For the batch job completion report the sort of things
are
> - overall status (success/success-with-warning/fail)
> - error details
> - warning details
> - log details
> - full details
> In addition your engine might get a 402 (etc.) code
back if the handler
> fails and would need to generate its own completion
report in that case.
>
> The status could go in a header X-Batch-Complete-Status
or in the body.
> The other items I think would need to go in the body
and it might be easiest
> to define a DTD and include them as an XML report. Your
module could provide
> routines to help the handler to generate that.
> Then your jobqueue module can unpack and handle that in
a standard way.
>
>   
This data structure sounds quite reasonable. status and
errors are
implemented, log details can be had with Catalyst::Log,
full details can be extracted from stash with a
representation
convention, that leaves only warnings to be dealt with

I wanna point out that the engine deals with perl data
structures, not
the literal (rendered) HTTP request/response. So headers
and XML bodys would be a waste of parse/deparse cpu cycles.
> With that output I'd want to be able to 
> - log selectively those different streams via a
Log4perl configuration file
> (crontab only really lets you do stdout and stderr)
> - record the job running/complete and success status to
either a database or
> a file
> - trigger actions on conditions, e.g. on status fail
-> run a program to
>   
> send email/SMS to a sysadmin
>
>   
- you can directly log to Log4perl from your code, however
the engine
could provide a wrapper which passes log messages to
Log4perl while
keeping a copy of them for reports.

- reports to db can be done via plugin (cause we might not
have access
to a database)

- yes, however at first I plan to hardcode conditions ->
actions to keep
the code lightweight (as mentioned below too)

> Previously I've logged "normal" progress
to one file with the volume of
> messages filtered by a log level and all
errors/warnings to another. I used
> a batch job module to manage the process and record
overall job
> running/complete and success status values in SQL. I
also held the batch
> definitions in an SQL table.
>
>   
- again, SQL might not be available and I don't want to
make it a prereq.
- as for logging, it remains an open question at this time,
either let
the code do all the logging, or pipe logging through the
engine ...
> There is a bit more complexity you could add to do with
exclusivity and job
> chaining.
> Some batch jobs are exclusive and should only be run
singly. While you can
> implement this inside a job, it's better for them to
call the
> jobqueue/crontab replacement module to handle it, and
then you can pick up
> error-exited child processes and release locks and log
errors if they fail
> to do so.
> Then there's the idea of conditional job chaining
where you only run certain
> jobs if their predecessors work and have alternate
routes if they fail. Most
> data centres need that kind of chaining for overnight
batches to allow
> recovery or to stop at critical points, for example if
you're copy managing
> large volumes of data between legacy and new systems
and need to ensure they
> remain consistent and in step. That's the point where
crontab isn't enough
> and people need to use a batch job queue package.
>
>   
Hmm, while I see that this is quite the standard for batch
job systems,
I don't think this Engine quite fits the description. The
jobs for this
engine are supposed to do lightweight stuff, be modeled on
HTTP
request-response and to be related to the Web Application
not system
maintenance.

I'm not excluding the possibility to implement
semaphores/locks and
chaining, but this would be very low priority...

Thank you for the feedback,
Kiki

_______________________________________________
Catalyst-dev mailing list
Catalyst-devlists.rawmode.org
http://lists.rawmode.org/mailman/listinfo/catalyst-dev

Kiki wrote:
> Status report for jobs is an interesting idea, perhaps
via
PoCo::Server::HTTP
That would be pretty cool to have a batch monitor.

>- reports to db can be done via plugin (cause we might
not have access to a
database)
Sure, a choice of file, SQL or user-supplied plugin sounds
good.

>- yes, however at first I plan to hardcode conditions
-> actions to keep
the code lightweight (as mentioned below too)
Fair enough.

>> There is a bit more complexity you could add to do
with exclusivity and
job chaining.
>Hmm, while I see that this is quite the standard for
batch job systems,
>I don't think this Engine quite fits the description.
The jobs for this
>engine are supposed to do lightweight stuff, be modeled
on HTTP
>request-response and to be related to the Web
Application not system
>maintenance.

My guess is people will bend the limits pretty quickly; as
long as it's easy
to extend they could do it themselves 

Regards, Peter 



_______________________________________________
Catalyst-dev mailing list
Catalyst-devlists.rawmode.org
http://lists.rawmode.org/mailman/listinfo/catalyst-dev

Kiki wrote:
> Peter Edwards wrote:
>> There is a bit more complexity you could add to do
with exclusivity and job
>> chaining.
>> Some batch jobs are exclusive and should only be
run singly. While you can
>> implement this inside a job, it's better for them
to call the
>> jobqueue/crontab replacement module to handle it,
and then you can pick up
>> error-exited child processes and release locks and
log errors if they fail
>> to do so.
>> Then there's the idea of conditional job chaining
where you only run certain
>> jobs if their predecessors work and have alternate
routes if they fail. Most
>> data centres need that kind of chaining for
overnight batches to allow
>> recovery or to stop at critical points, for example
if you're copy managing
>> large volumes of data between legacy and new
systems and need to ensure they
>> remain consistent and in step. That's the point
where crontab isn't enough
>> and people need to use a batch job queue package.
>>
>>   
> Hmm, while I see that this is quite the standard for
batch job systems,
> I don't think this Engine quite fits the description.
The jobs for this
> engine are supposed to do lightweight stuff, be modeled
on HTTP
> request-response and to be related to the Web
Application not system
> maintenance.
> 
> I'm not excluding the possibility to implement
semaphores/locks and
> chaining, but this would be very low priority...

Locks are potentially important - or being able to say
"only one run of job 
type X at once" at least.

Job chaining shouldn't be the responsibility of the job
runner - since we're 
already intending a $c->schedule interface for apps to
request things to be 
run, a workflow job could simply schedule its parts via a
wrapper that invokes 
the inner job directly ($c->subreq would perhaps pretty
much do the trick if 
we can get trunk SubRequest to CPAN) and then schedules an
appropriate next 
stage itself.

-- 
      Matt S Trout       Offering custom development,
consultancy and support
   Technical Director    contracts for Catalyst, DBIx::Class
and BAST. Contact
Shadowcat Systems Ltd.  mst (at) shadowcatsystems.co.uk for
more information

+ Help us build a better perl ORM: http://dbix
-class.shadowcatsystems.co.uk/ +

_______________________________________________
Catalyst-dev mailing list
Catalyst-devlists.rawmode.org
http://lists.rawmode.org/mailman/listinfo/catalyst-dev