List Info

Thread: ruleflow trouble when process a large number of records




ruleflow trouble when process a large number of records
country flaguser name
United States
2008-04-29 09:13:25
Hello,

I'm using drools 4.0.4 on jboss 4.0.5.
I build a ruleflow and  a set of rules following the
documentation on
official drools site.
The ruleflow consist of about 50 ruleflow-group and 116
rules.
The ruleflow works correctly when processing a little number
of records
(100-1000).
Instead if the number of records is around 20000 (they are
processed in sets
of 100 record) many rules don't fire; so many records are
lost in output. 
If I process in sets of 50 records I lose less records than
in sets of 100
records. And  If I process in sets of 20 records I lose less
records than in
sets of 50 records.
Must I tweak the JVM memory or the number of JVM threads? Or
does this issue
concern the building of ruleflow (maybe it's bad built)?

Thanks in advance
-- 
View this message in context: http://www
.nabble.com/ruleflow-trouble-when-process-a-large-number-of-
records-tp16960999p16960999.html
Sent from the drools - user mailing list archive at
Nabble.com.

_______________________________________________
rules-users mailing list
rules-userslists.jboss.org

https://lists.jboss.org/mailman/listinfo/rules-users

Re: ruleflow trouble when process a large number of records
country flaguser name
Belgium
2008-04-29 16:20:28
There should be no tweaking to get it working correctly
(ruleflows don't 
even use separate threads during execution currently). 
Based on your 
explanation however, it is not obvious why things might be
going wrong.  I 
suggest trying to use the audit view or a custom logger to
figure out what 
is going on in both cases.  If you are losing records, there
must be some 
reason why the rules are not being executed for those
records (e.g. the 
record does not match the conditions of the rule, the
ruleflow-group is 
deactivated before all records have been processed, etc.). 
Try making a 
small use case where you enter a number of records and not
all records seem 
to be processed.  Then try looking at the audit log to see
which rules are 
activated and deactivated, which ruleflow-groups are
activated and 
deactivated, etc.  If you do find a problem where you thing
the ruleflow is 
not behaving as expected, it would be nice if you could send
me a 
self-countained example that shows the problem and I'll try
to look into it 
asap.

One additional piece of advice: it does seem a little
strange however that 
you are using 50 ruleflow-groups for 116 rules. 
Ruleflow-groups are used to 
specify the order in which large rule sets should be
executed.  If you want 
to specify the order for single rules, other features like
salience, agenda 
groups, etc. might also be useful.  But nevertheless,
ruleflows should even 
work in these circumstances.

Kris

----- Original Message ----- 
From: "Mazrim" <UE011434guest.telecomitalia.it>
To: <rules-userslists.jboss.org>
Subject: [rules-users] ruleflow trouble when process a large
number of 
records


>
> Hello,
>
> I'm using drools 4.0.4 on jboss 4.0.5.
> I build a ruleflow and  a set of rules following the
documentation on
> official drools site.
> The ruleflow consist of about 50 ruleflow-group and 116
rules.
> The ruleflow works correctly when processing a little
number of records
> (100-1000).
> Instead if the number of records is around 20000 (they
are processed in 
> sets
> of 100 record) many rules don't fire; so many records
are lost in output.
> If I process in sets of 50 records I lose less records
than in sets of 100
> records. And  If I process in sets of 20 records I lose
less records than 
> in
> sets of 50 records.
> Must I tweak the JVM memory or the number of JVM
threads? Or does this 
> issue
> concern the building of ruleflow (maybe it's bad
built)?
>
> Thanks in advance
> -- 
> View this message in context: 
> http://www
.nabble.com/ruleflow-trouble-when-process-a-large-number-of-
records-tp16960999p16960999.html
> Sent from the drools - user mailing list archive at
Nabble.com.
>
> _______________________________________________
> rules-users mailing list
> rules-userslists.jboss.org
> 
https://lists.jboss.org/mailman/listinfo/rules-users 

_______________________________________________
rules-users mailing list
rules-userslists.jboss.org

https://lists.jboss.org/mailman/listinfo/rules-users

Re: ruleflow trouble when process a large number of records
country flaguser name
United States
2008-04-30 02:24:03
Thanks for the quick response and advices.
I'll delve into with your suggestions and I let you know
when I solve the
issue.

Massimo




Kris Verlaenen wrote:
> 
> There should be no tweaking to get it working correctly
(ruleflows don't 
> even use separate threads during execution currently). 
Based on your 
> explanation however, it is not obvious why things might
be going wrong.  I 
> suggest trying to use the audit view or a custom logger
to figure out what 
> is going on in both cases.  If you are losing records,
there must be some 
> reason why the rules are not being executed for those
records (e.g. the 
> record does not match the conditions of the rule, the
ruleflow-group is 
> deactivated before all records have been processed,
etc.).  Try making a 
> small use case where you enter a number of records and
not all records
> seem 
> to be processed.  Then try looking at the audit log to
see which rules are 
> activated and deactivated, which ruleflow-groups are
activated and 
> deactivated, etc.  If you do find a problem where you
thing the ruleflow
> is 
> not behaving as expected, it would be nice if you could
send me a 
> self-countained example that shows the problem and I'll
try to look into
> it 
> asap.
> 
> One additional piece of advice: it does seem a little
strange however that 
> you are using 50 ruleflow-groups for 116 rules. 
Ruleflow-groups are used
> to 
> specify the order in which large rule sets should be
executed.  If you
> want 
> to specify the order for single rules, other features
like salience,
> agenda 
> groups, etc. might also be useful.  But nevertheless,
ruleflows should
> even 
> work in these circumstances.
> 
> Kris
> 
> ----- Original Message ----- 
> From: "Mazrim" <UE011434guest.telecomitalia.it>
> To: <rules-userslists.jboss.org>
> Subject: [rules-users] ruleflow trouble when process a
large number of 
> records
> 
> 
>>
>> Hello,
>>
>> I'm using drools 4.0.4 on jboss 4.0.5.
>> I build a ruleflow and  a set of rules following
the documentation on
>> official drools site.
>> The ruleflow consist of about 50 ruleflow-group and
116 rules.
>> The ruleflow works correctly when processing a
little number of records
>> (100-1000).
>> Instead if the number of records is around 20000
(they are processed in 
>> sets
>> of 100 record) many rules don't fire; so many
records are lost in output.
>> If I process in sets of 50 records I lose less
records than in sets of
>> 100
>> records. And  If I process in sets of 20 records I
lose less records than 
>> in
>> sets of 50 records.
>> Must I tweak the JVM memory or the number of JVM
threads? Or does this 
>> issue
>> concern the building of ruleflow (maybe it's bad
built)?
>>
>> Thanks in advance
>> -- 
>> View this message in context: 
>> http://www
.nabble.com/ruleflow-trouble-when-process-a-large-number-of-
records-tp16960999p16960999.html
>> Sent from the drools - user mailing list archive at
Nabble.com.
>>
>> _______________________________________________
>> rules-users mailing list
>> rules-userslists.jboss.org
>> 
https://lists.jboss.org/mailman/listinfo/rules-users 
> 
> _______________________________________________
> rules-users mailing list
> rules-userslists.jboss.org
> 
https://lists.jboss.org/mailman/listinfo/rules-users
> 
> 

-- 
View this message in context: http://www
.nabble.com/ruleflow-trouble-when-process-a-large-number-of-
records-tp16960999p16977116.html
Sent from the drools - user mailing list archive at
Nabble.com.

_______________________________________________
rules-users mailing list
rules-userslists.jboss.org

https://lists.jboss.org/mailman/listinfo/rules-users

[1-3]

about | contact  Other archives ( Real Estate discussion Medical topics )