From: Ram on
We have a requirement to send some research analysis mails as quickly as
possible.



Everyday after the data is available my app generates the mails in eml
format in a directory.
These are personalized mails with attachments and have to reach the
recipients "instantly" ( in my customers lingo ... Research reports are
useless after the market opens )



What is the quickest way of pushing EML files to postfix for delivery.

Currently I have a perl script that makes parallel smtp connections on
localhost and sends the mails. Should I send the mails on command line.
There are currently around 50k mails to be delivered ideally within
5-10 mins.

I am only bothered about sending to postfix because delievery from there
is already taken care of.



Is there a better way , other than sending mails on command line or
SMTP. Something like and API to inject into postfix maildrop.




Thanks
Ram

From: Ralf Hildebrandt on
* Ram <ram(a)netcore.co.in>:
> We have a requirement to send some research analysis mails as quickly as
> possible.
>
>
>
> Everyday after the data is available my app generates the mails in eml
> format in a directory.

What is eml format?

> Currently I have a perl script that makes parallel smtp connections on
> localhost and sends the mails.

This sounds good!

> Should I send the mails on command line.

No, using the postfix sendmail binary is actually slower.

> There are currently around 50k mails to be delivered ideally within
> 5-10 mins.

How fast are you now?
50.000/10min = 5.000/min = 83/s = that's a lot
50.000/50min = 10.000/min = 186/s = that's even more

--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebrandt(a)charite.de | http://www.charite.de


From: Ram on
Hi ,

On Fri, 2010-08-13 at 09:39 +0200, Ralf Hildebrandt wrote:
> * Ram <ram(a)netcore.co.in>:
> > We have a requirement to send some research analysis mails as quickly as
> > possible.
> >
> >
> >
> > Everyday after the data is available my app generates the mails in eml
> > format in a directory.
>
> What is eml format?
>

Mail in plain text format , mime encoded message


> > Currently I have a perl script that makes parallel smtp connections on
> > localhost and sends the mails.
>
> This sounds good!
>
> > Should I send the mails on command line.
>
> No, using the postfix sendmail binary is actually slower.
>
> > There are currently around 50k mails to be delivered ideally within
> > 5-10 mins.
>
> How fast are you now?
> 50.000/10min = 5.000/min = 83/s = that's a lot
> 50.000/50min = 10.000/min = 186/s = that's even more
>
Currenlty I get 40/s - 45/s
But I want it to be atleast 100/s
Delivery is not at all an issue , because postfix gives it to further
relay boxes which are under our control again.


This is a 8GB Ram Centos 5.4 server with SAS discs

Do I need to increase the hardware



Thanks
Ram

From: Ralf Hildebrandt on
* Ram <ram(a)netcore.co.in>:

> Mail in plain text format , mime encoded message

OK!

> Currenlty I get 40/s - 45/s
That sounds normal. Any filtering (in these cases you should inject in
a way that bypasses and filters)
> But I want it to be atleast 100/s

Two machineS?
relay boxes
> Delivery is not at all an issue , because postfix gives it to further
> relay boxes which are under our control again.

Why not inject to the further relay boxes?

> Do I need to increase the hardware

It could be :)

--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebrandt(a)charite.de | http://www.charite.de


From: Wietse Venema on
Ralf Hildebrandt:
> * Ram <ram(a)netcore.co.in>:
>
> > Mail in plain text format , mime encoded message
>
> OK!
>
> > Currenlty I get 40/s - 45/s
> That sounds normal. Any filtering (in these cases you should inject in
> a way that bypasses and filters)
> > But I want it to be atleast 100/s
>
> Two machineS?
> relay boxes
> > Delivery is not at all an issue , because postfix gives it to further
> > relay boxes which are under our control again.
>
> Why not inject to the further relay boxes?
>
> > Do I need to increase the hardware
>
> It could be :)

Other options: increase input concurrency, or play with in_flow_delay.
Note that increasing your input rates will cause output rates to drop.
It's all about competing for disk access.

Wietse