From: Mike Abbott on
> In the mean time, the authors may want to enforce strict sanity
> checks on IMAP server output such as the reply payload length, and
> also make sure that their vstream_limit code behaves when reads
> are alternated with writes (either refuse to allow read/write/read,
> or enforce the limit as promised).

OK

> One question I have is why would anyone send an email message that
> is 100% identical to a message that is already sitting in an IMAP
> store?

Because the message in the IMAP server is brand-new, having just been composed by the user. For instance:
1. As the user composes a new message, the MUA saves the new message in the IMAP "Drafts" mailbox, possibly by using CATENATE (RFC 4469) and URLAUTH (RFC 4467) to compose it out of parts of other messages already on the IMAP server.
2. The MUA connects to Postfix and uses a BURL command instead of a DATA command.
3. Postfix connects to the IMAP server to retrieve the body of the message from the Drafts mailbox.
4. After receiving a positive response from Postfix, the MUA moves the message from the Drafts mailbox to the Sent mailbox.

Notice that in this example, the body of the message is pushed out of the client only once (to the IMAP server). Without BURL, it is pushed at least twice (once to Postfix and a second time to the Sent mailbox, and possibly also to the Drafts mailbox for auto-save during composition). RFC 4550 section 2 explains it better. On bandwidth-constrained devices such an optimization can have a significant effect.

> Why is there a problem with sending mail to the SMTP server directly,
> instead of taking an indirect route via an IMAP server?

There is no such problem. The problem is the double- or triple-transmission that is currently needed: once to SMTP, again to IMAP (for saving a copy in Sent, and auto-save to Drafts).

> explain why sending to an IMAP server is
> faster than sending that same message to an SMTP server.

It isn't, except for CATENATE (RFC 4469) and URLAUTH (RFC 4467). With these one can forward a huge attachment using minimal bandwidth. Then once the new huge message is complete, fast server-to-server communication (i.e., BURL) can be used to send it. The example in RFC 4550 section 2.4.1 shows how an arbitrarily-large attachment can be forwarded while using only a couple KB of bandwidth for mail headers, MIME headers, and plain-text comments.

The savings is in the reduction of bandwidth between the client and the server -- both servers, IMAP and SMTP. The benefit is in being able to forward a multi-megabyte document from a low-network-bandwidth, high-network-latency device using only 2-3 TCP packets rather than thousands.
From: Victor Duchovni on
On Mon, Apr 12, 2010 at 08:56:17AM -0500, Mike Abbott wrote:

> >> + case SMTP_ERR_EOF:
> >> + smtpd_chat_reply(state, "554 4.6.6 EOF from IMAP server");
> >> + vstream_longjmp(state->client, SMTP_ERR_QUIET);
> >> + break;
> >
> > Why is the DSN code 4.X.X when the SMTP reply code is 5XX? Is this a
> > permanent or a transient error code?
>
> It is a transient failure. The reasoning for these particular codes
> was as follows. RFC 4468 section 3.2 states "If the URL fetch fails, the
> server will fail the entire transaction." RFC 5321 section 4.2.2 uses
> code 554 for "Transaction failed." And the table in RFC 5248 section
> 2.4 implies that a 4.6.6 is valid with a 554. If this interpretation
> of the RFCs is incorrect, please propose corrected response codes.

This interpretation is incorrect, the 3-digit SMTP code, must match the
3-part DSN code. For transient errors use 454.

--
Viktor.

P.S. Morgan Stanley is looking for a New York City based, Senior Unix
system/email administrator to architect and sustain our perimeter email
environment. If you are interested, please drop me a note.

From: Charles Marcus on
On 2010-04-12 10:24 AM, Mike Abbott wrote:
> Because the message in the IMAP server is brand-new, having just been
> composed by the user. For instance:
> 1. As the user composes a new message, the MUA saves the new message
> in the IMAP "Drafts" mailbox, possibly by using CATENATE (RFC 4469)
> and URLAUTH (RFC 4467) to compose it out of parts of other messages
> already on the IMAP server.
> 2. The MUA connects to Postfix and uses a BURL command instead of a
> DATA command.
> 3. Postfix connects to the IMAP server to retrieve the body of the
> message from the Drafts mailbox.
> 4. After receiving a positive response from Postfix, the MUA moves
> the message from the Drafts mailbox to the Sent mailbox.

How would this work for messages that are never saved to the Drafts folder?

In fact, I rarely save it to the drafts folder before sending...

--

Best regards,

Charles

From: Charles Marcus on
On 2010-04-12 11:03 AM, Charles Marcus wrote:
> On 2010-04-12 10:24 AM, Mike Abbott wrote:
>> Because the message in the IMAP server is brand-new, having just been
>> composed by the user. For instance:
>> 1. As the user composes a new message, the MUA saves the new message
>> in the IMAP "Drafts" mailbox, possibly by using CATENATE (RFC 4469)
>> and URLAUTH (RFC 4467) to compose it out of parts of other messages
>> already on the IMAP server.
>> 2. The MUA connects to Postfix and uses a BURL command instead of a
>> DATA command.
>> 3. Postfix connects to the IMAP server to retrieve the body of the
>> message from the Drafts mailbox.
>> 4. After receiving a positive response from Postfix, the MUA moves
>> the message from the Drafts mailbox to the Sent mailbox.

> How would this work for messages that are never saved to the Drafts folder?
>
> In fact, I rarely save it to the drafts folder before sending...

I was actually hoping this could somehow result in satisfaction of the
past requests for a way to automatically save a copy of the message to
the Sent folder without having to transfer it twice over the wire (once
to send it and once to save it to the sent folder)...

--

Best regards,

Charles

From: Mike Abbott on
> How would this work for messages that are never saved to the Drafts folder?

Then instead the message could be saved to the Sent folder instead, and the submission server could fetch it from there. The MUA would have to remove it from the Sent folder if sending failed, of course. Nothing magic about the Drafts folder.