sending >500 emails using send_email in a for loop

I've built a newsletter app that currently cycles through
500+ emails
and I'd like to have it scale to handle more.

Currently I just have code like this in my view:

for recipient in email_list:
    msg = EmailMultiAlternatives(email.subject,
email.message_text,
email.sender, [recipient])
    msg.attach_alternative(email.message_html, 'text/html')
    msg.send()

it seems to hang up the browser for quite a while with even
just a few
emails. What is the best way to do this and give the user
confirmation
that the email has been sent to everyone?

Should I fork off a new process? If so, how can I be sure it
finishes
successfully?

-- 
Pete

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Pete,

> I've built a newsletter app that currently cycles
through 500+ emails
> and I'd like to have it scale to handle more.

<snip>

> it seems to hang up the browser for quite a while with
even just a few
> emails. What is the best way to do this and give the
user confirmation
> that the email has been sent to everyone?

Do you have the ability to run a scheduled job (cronjob) on
the
server? If so, here's a good way to make this work. It's
basically a
batch job rather than an online one.

- Store your recipient email addresses in the database but
don't run
your mass mails inside the view.

- Instead, schedule a job to run at a suitable interval as
follows.
The job would be a Python/Django standalone script that
loops through
"unprocessed" email addresses and sends them the
newsletter. See James
Bennett's fantastic write up on standalone scripts here:
http://www.b-list.org/weblog/2007/sep/22/stan
dalone-django-scripts/

"Unprocessed email addresses": By that I mean
something like this,
have a "status" field in your email address
table/model. The status
would default to "Not processed". When your script
picks up a bunch of
email addresses to mass mail, first mark the status on those
addresses
as "Processing", then send your mass mail to those
addresses, and
lastly, mark them as "Processed". When your script
picks up email
addresses, just make sure it always fetches only the
"Not processed"
ones. So, even if a second instance of your script starts a
bit before
the first one has finished, the second instance would only
pick up
fresh addresses.

Also, maintain a "status_timestamp" field and
update it every time the
status is changed. This allows you to run another standalone
process
to check if any emails have been stuck in
"Processing" status for an
unexpectedly long period of time (most likely indicating
that your
mass mailer script failed in the middle.)

> Should I fork off a new process? If so, how can I be
sure it finishes
> successfully?

You can make your standalone script log its completion
status and
statistics in another table. You can then report to the user
how the
job is doing off of data from this table.

You can extend the=is idea by adding a "Job"
container that would
manage mass mailing of multiple issues of your newsletter.
In that
case, the Job would have its own status field with the same
idea (New,
Processing, Processed, Failed)

-Rajesh
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

You may want to have a look at http://g
roups.google.com/group/django-mailer

It seems to have died halfway complete but there is a bit
of
discussion and at least some code you could look at.

Jökull

On Dec 18, 9:18 pm, "Peter Baumgartner"
<sgt.hu...gmail.com> wrote:
> I've built a newsletter app that currently cycles
through 500+ emails
> and I'd like to have it scale to handle more.
>
> Currently I just have code like this in my view:
>
> for recipient in email_list:
>     msg = EmailMultiAlternatives(email.subject,
email.message_text,
> email.sender, [recipient])
>     msg.attach_alternative(email.message_html,
'text/html')
>     msg.send()
>
> it seems to hang up the browser for quite a while with
even just a few
> emails. What is the best way to do this and give the
user confirmation
> that the email has been sent to everyone?
>
> Should I fork off a new process? If so, how can I be
sure it finishes
> successfully?
>
> --
> Pete
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Peter,

On 19/12/2007, at 8:18 AM, Peter Baumgartner wrote:

>
> I've built a newsletter app that currently cycles
through 500+ emails
> and I'd like to have it scale to handle more.
>
> Currently I just have code like this in my view:
>
> for recipient in email_list:
>    msg = EmailMultiAlternatives(email.subject,
email.message_text,
> email.sender, [recipient])
>    msg.attach_alternative(email.message_html,
'text/html')
>    msg.send()
>
> it seems to hang up the browser for quite a while with
even just a few
> emails. What is the best way to do this and give the
user confirmation
> that the email has been sent to everyone?
>
> Should I fork off a new process? If so, how can I be
sure it finishes
> successfully?

I wrote a newsletter app that does this (and more) and is
about to  
become public, just as soon as I clean it up and make sure
it works  
outside the site it was developed for... might be too late
for you  
though.

It uses the method that Rajesh describes in his reply - it
create a  
Delivery record for each recipient, with a foreignkey
pointing to the  
newsletter to be mailed. An hourly cron job pulls a certain
amount of  
Delivery entries (can't just send all of them at once
because some  
hosting providers think you're a spammer if you do that),
sends them  
and marks them as "sent". The newsletter object
itself gets marked as  
"sent" as soon as the Delivery objects are
created, so you won't send  
it a 2nd by accident while its delivery is pending.

As for knowing that the delivery has finished, adding that
should be  
easy - just count unsent delivery objects. I use a
"send" button in  
the admin newsletter list view (changing to "sent"
after the send),  
the delivery count could easily be displayed next to it.

Let me know if you want a look at the mailing code before
the whole  
app is available.

Itai

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-usersgooglegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---