bulk email

Hi all,
I have developed a php bulk email system which is integrated
with our
company website so that when you create a newsletter it
publishes the
newsletter on the website as well as creating the bulk mail
job.

I have never worked with bulk email before and am a bit
clueless about
it. Basically what I have done is used an open source class
to generate
the MIME code and include attached inline images then just
loop through
the entire mailing list sending one email at a time (I have
to update
link code each time in order to track users links). The
system manages
to send out about 2000 mails before it times out and stops.
my mailing
list is 2600 so I am looking for a simple way of fixing this
without
redoing the entire thing. I have added

set_time_limit(0);
ignore_user_abort(1);

to the top of the script. is there anything else I can do
that might
help? I would appreciate any assistance!
thanks in advance
Ben


--~--~---------~--~----~------------~-------~--~----~
This group is managed and maintained by the development
staff at 360 PSG. An enterprise application development
company utilizing open-source technologies for todays
small-to-medium size businesses.

For information or project assistance please visit :
http://www.360psg.com

You received this message because you are subscribed to the
Google Groups "Professional PHP Developers" group.
To post to this group, send email to Professional-PHPgooglegroups.com
To unsubscribe from this group, send email to
Professional-PHP-unsubscribegooglegroups.com
For more options, visit this group at http:
//groups.google.com/group/Professional-PHP
-~----------~----~----~----~------~----~------~--~---

You are in for a world of pain my friend.

It is mainly apache that times you out.... you get a page
error right,
not a php error.
It is like a 404 or something.

Apache will die if no output is sent back to the stream for
a while...
one way over this is each time through the loop, echo out a
single
white space like echo(' '); this will help trick apache to
think it is
receiving data from the script and not die... but wait it
still fails.
(make sure you are not buffering output)

Basically, not with mail, but another app... I wrote an
application
that imported hundreds of thousands of records and processed
them...
each record containing 3600 columns.

So needless to say it takes a few hours... but apache kept
dying...
sometimes at an hour or two... sometimes longer.

I had to break the imported file into batch groups of about
50,000
records each then execute my commands on one sub-file...
after
completion (about 5 minutes)... a javascript redirect was
echoed to the
screen calling the same page with the new 'sub-file' and it
was like a
brand new page call in Apache... it considered the previous
one done
and this a new page request... this did two things...

It insured no one single file caused the server to crash and
die... and
it gave me the ability to count the blocks of 'sub-files'
and there by
actually create a real-time status bar using file 3 of 25 to
calculate
a percentage complete and even provide time estimates to
finish... and
it is very functional very accurate...

But be warned !!!!
DO NOT LET A SCRIPT LIKE THIS BE AVAILABLE OPENLY !!!!
My god... imagine someone out there were malicious and able
to call 50
threads at once... or 100 or a 1000....

Mine as well plan on rebooting (killin apache) every 5
minutes

When working with long heavy processes... password protect
and keep'em
in-house!

Hope that helps... just some tidbits from my 'work-around'
college...
if anyone has other experience or help (when PHP isnt the
time-out
culprit) please chime in... always in the mood for more good
ideas !

Joel


--~--~---------~--~----~------------~-------~--~----~
This group is managed and maintained by the development
staff at 360 PSG. An enterprise application development
company utilizing open-source technologies for todays
small-to-medium size businesses.

For information or project assistance please visit :
http://www.360psg.com

You received this message because you are subscribed to the
Google Groups "Professional PHP Developers" group.
To post to this group, send email to Professional-PHPgooglegroups.com
To unsubscribe from this group, send email to
Professional-PHP-unsubscribegooglegroups.com
For more options, visit this group at http:
//groups.google.com/group/Professional-PHP
-~----------~----~----~----~------~----~------~--~---

Thanks, I thought about doing something like that. I have
been
considering using an AJAX script that delivers a running
total of
emails sent to the user. What are your views on that?

Alternatively I considered splitting it up into batches of
100 using
header('location') to
recall the script. Potentially this mailing
list could grow quite a lot are there any bad effects that
you know of
that may be caused by splitting it up like this? the length
of time it
takes to send out the mail job is not particularly crucial.

I also tried outputting text after each mail job but it
didn't actually
output anythign (in fact the browser hung on the previous
page, then
eventually, once it timed out displayed ALL the subscriber
ids in one
go (which is what I had been outputting to debug it).

Any more advice on the issue would be very helpful!
Ben


--~--~---------~--~----~------------~-------~--~----~
This group is managed and maintained by the development
staff at 360 PSG. An enterprise application development
company utilizing open-source technologies for todays
small-to-medium size businesses.

For information or project assistance please visit :
http://www.360psg.com

You received this message because you are subscribed to the
Google Groups "Professional PHP Developers" group.
To post to this group, send email to Professional-PHPgooglegroups.com
To unsubscribe from this group, send email to
Professional-PHP-unsubscribegooglegroups.com
For more options, visit this group at http:
//groups.google.com/group/Professional-PHP
-~----------~----~----~----~------~----~------~--~---

Ok...

AJAX could work really well for what you are doing... I
wrote these
things long ago (before the term... and web support
community for AJAX
really got moving)... so yes AJAX can do the same thing.

You should split in batches but forget the header( ) use the
AJAX
calls... be much more efficient.

As for output !!! ha the oldest problem in the scripting
language book
!
Apache handles your output to the browser... so you call a
php file
apache looks and finds that you need to run .php extenstions
through a
third-party (in other words dont just show code)... apache
calls php
(as a cgi) or executes it (as a module) and it loads and
parses the
file... apache waits patiently for its 'outsourced' request
to come
back.... in your script you output something... anything...
apache
happily accepts that as it's output...but... apache decides
when it has
enough information to warrant spitting it back to the
browser for
interpretation... so... sometimes you get a few pieces back
earlier,
but usually... apache waits for the whole document to
complete before
sending that output back to the browser...

The way i tend to get around it... (and i HATE to say it)...
frames.
Use frames (one to make the call... usually small/invisible
that can
call your AJAX processes and trigger the PHP to run.)
Then in the main frame use another AJAX call to update a
'status' area
with information from a flat-file log or database that you
'write' to
during your looping script.

This way your AJAX monitoring process reads the logs
generated by your
AJAX mailer and you have a complete system.

Keep the groupings small (test it on your server... assume
10 minutes
maximum execution... before apache craps out).
Plus keeping it small... if your browser crashes or there is
an
error.... you can create a 'resume' button to finish later.

So... all good ideas. i think your app calls for a dual
frame (uggggh i
hate frames) (or a popup.. just a pain)... dual AJAX
processes lets you
send infinite and monitor real-time.

Good Luck.

Joel


--~--~---------~--~----~------------~-------~--~----~
This group is managed and maintained by the development
staff at 360 PSG. An enterprise application development
company utilizing open-source technologies for todays
small-to-medium size businesses.

For information or project assistance please visit :
http://www.360psg.com

You received this message because you are subscribed to the
Google Groups "Professional PHP Developers" group.
To post to this group, send email to Professional-PHPgooglegroups.com
To unsubscribe from this group, send email to
Professional-PHP-unsubscribegooglegroups.com
For more options, visit this group at http:
//groups.google.com/group/Professional-PHP
-~----------~----~----~----~------~----~------~--~---

Thank you, you have been very helpful.


--~--~---------~--~----~------------~-------~--~----~
This group is managed and maintained by the development
staff at 360 PSG. An enterprise application development
company utilizing open-source technologies for todays
small-to-medium size businesses.

For information or project assistance please visit :
http://www.360psg.com

You received this message because you are subscribed to the
Google Groups "Professional PHP Developers" group.
To post to this group, send email to Professional-PHPgooglegroups.com
To unsubscribe from this group, send email to
Professional-PHP-unsubscribegooglegroups.com
For more options, visit this group at http:
//groups.google.com/group/Professional-PHP
-~----------~----~----~----~------~----~------~--~---

Hi Ben, have you considered tapping in to open source
mailing
libraries? A colleague of mine developed an application that
can handle
almost any mailing related tasks (including bulk sends)
without being
impacted by the timeout settings. if you are interested, the
package is
freely available at http://www.swiftmailer.or
g.

Robert

On Dec 5, 4:35 am, "Ben" <b.ep...gmail.com> wrote:
> Hi all,
> I have developed a php bulk email system which is
integrated with our
> company website so that when you create a newsletter it
publishes the
> newsletter on the website as well as creating the bulk
mail job.
>
> I have never worked with bulk email before and am a bit
clueless about
> it. Basically what I have done is used an open source
class to generate
> the MIME code and include attached inline images then
just loop through
> the entire mailing list sending one email at a time (I
have to update
> link code each time in order to track users links). The
system manages
> to send out about 2000 mails before it times out and
stops. my mailing
> list is 2600 so I am looking for a simple way of fixing
this without
> redoing the entire thing. I have added
>
> set_time_limit(0);
> ignore_user_abort(1);
>
> to the top of the script. is there anything else I can
do that might
> help? I would appreciate any assistance!
> thanks in advance
> Ben


--~--~---------~--~----~------------~-------~--~----~
This group is managed and maintained by the development
staff at 360 PSG. An enterprise application development
company utilizing open-source technologies for todays
small-to-medium size businesses.

For information or project assistance please visit :
http://www.360psg.com

You received this message because you are subscribed to the
Google Groups "Professional PHP Developers" group.
To post to this group, send email to Professional-PHPgooglegroups.com
To unsubscribe from this group, send email to
Professional-PHP-unsubscribegooglegroups.com
For more options, visit this group at http:
//groups.google.com/group/Professional-PHP
-~----------~----~----~----~------~----~------~--~---