From: David Woodhouse on
On Wed, 2010-08-11 at 10:46 -0500, Jeffrey Hundstad wrote:
> Exchange 2010 does not handle IMAP "chunking" (partial message transfer)
> correctly. Any request after about 1 megabyte of total message size
> will fail.
>
> Thunderbird uses this "chunking" feature to give you a status update
> while downloading large messages. The IMAP statements are of this type:
> 11 UID fetch 244477 (UID RFC822.SIZE BODY[]<20480.12288>)
>
> When the 20480 is larger than 1MB Exchange "claims" there is no more.
> Sigh....

I think the problem is not with the fetching -- the problem is that
Exchange lies about RFC822.SIZE before the IMAP client even starts to
fetch the message. It reports a size which is smaller than the actual
size of the message, thus leading to truncated fetches.

In Evolution we have a workaround -- we don't just stop when we get to
the reported RFC822.SIZE; we continue fetching more chunks until the
server actually stops giving us any more. It's not as efficient (because
we fall back to having only one more chunk outstanding at a time rather
than the normal three in parallel), but at least it works around this
brokenness of Exchange.

http://git.gnome.org/browse/evolution-data-server/commit/?id=9714c064

--
David Woodhouse Open Source Technology Centre
David.Woodhouse(a)intel.com Intel Corporation

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Jeffrey Hundstad on
On 08/11/2010 02:01 AM, viresh kumar wrote:
> When i send a mail using git send-email then it receives fine on
> outlook but on thunderbird, tabs are converted to spaces.
>
> This doesn't happen with every patch on thunderbird, but only a few.
> And observation is that it happens only with big patches.
> (more than 500 lines)
>
> Any idea, how to solve issue now on thunderbird??

Hello,

Exchange 2010 does not handle IMAP "chunking" (partial message transfer)
correctly. Any request after about 1 megabyte of total message size
will fail.

Thunderbird uses this "chunking" feature to give you a status update
while downloading large messages. The IMAP statements are of this type:
11 UID fetch 244477 (UID RFC822.SIZE BODY[]<20480.12288>)

When the 20480 is larger than 1MB Exchange "claims" there is no more.
Sigh....

Fortunately you can disable this feature. To disable this in
Thunderbird you can go to the Advanced configuration and disable the
following feature, by setting it to false:
mail.server.default.fetch_by_chunks


--
Jeffrey Hundstad



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Jeffrey Hundstad on
On 08/11/2010 10:58 AM, David Woodhouse wrote:
> On Wed, 2010-08-11 at 10:46 -0500, Jeffrey Hundstad wrote:
>> Exchange 2010 does not handle IMAP "chunking" (partial message transfer)
>> correctly. Any request after about 1 megabyte of total message size
>> will fail.
>>
>> Thunderbird uses this "chunking" feature to give you a status update
>> while downloading large messages. The IMAP statements are of this type:
>> 11 UID fetch 244477 (UID RFC822.SIZE BODY[]<20480.12288>)
>>
>> When the 20480 is larger than 1MB Exchange "claims" there is no more.
>> Sigh....
>
> I think the problem is not with the fetching -- the problem is that
> Exchange lies about RFC822.SIZE before the IMAP client even starts to
> fetch the message. It reports a size which is smaller than the actual
> size of the message, thus leading to truncated fetches.
>
> In Evolution we have a workaround -- we don't just stop when we get to
> the reported RFC822.SIZE; we continue fetching more chunks until the
> server actually stops giving us any more. It's not as efficient (because
> we fall back to having only one more chunk outstanding at a time rather
> than the normal three in parallel), but at least it works around this
> brokenness of Exchange.
>
> http://git.gnome.org/browse/evolution-data-server/commit/?id=9714c064
>

In either case it can be used successfully by disabling
mail.server.default.fetch_by_chunks in Thunderbird.

--
Jeffrey Hundstad



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Avery Pennarun on
On Wed, Aug 11, 2010 at 11:58 AM, David Woodhouse <dwmw2(a)infradead.org> wrote:
> In Evolution we have a workaround -- we don't just stop when we get to
> the reported RFC822.SIZE; we continue fetching more chunks until the
> server actually stops giving us any more. It's not as efficient (because
> we fall back to having only one more chunk outstanding at a time rather
> than the normal three in parallel), but at least it works around this
> brokenness of Exchange.
>
> http://git.gnome.org/browse/evolution-data-server/commit/?id=9714c064

Out of curiosity, why fall back to one chunk at a time? It seems to
me that IMAP should be able to still support multiple outstanding
requests in that case, but you'd just get errors on the latter chunks.

It is just that there was no point optimizing the workaround case?

Have fun,

Avery
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: David Woodhouse on
On Wed, 2010-08-11 at 12:18 -0400, Avery Pennarun wrote:
>
> Out of curiosity, why fall back to one chunk at a time? It seems to
> me that IMAP should be able to still support multiple outstanding
> requests in that case, but you'd just get errors on the latter chunks.
>
> It is just that there was no point optimizing the workaround case?

There wasn't a lot of point in optimising it.

The current logic, shown in the patch I referenced, is to keep fetching
new chunks while the stream position matches the end of the previous
chunk we attempted to fetch.

To handle multiple outstanding requests, especially if they can be
satisfied out-of-order, would have been more complex because the stream
position (in the 'really_fetched' variable) wouldn't necessarily match
anything interesting. We'd have to keep more state, and the whole thing
would get a lot more intrusive.

Also, for the common case where the server isn't broken and the mail
size happens not to fall on a chunk boundary, the current implementation
results in no extra fetch requests. Doing otherwise would either mean
extra fetch requests even for this common case, or would mean even more
complexity to 'catch up' by issuing additional fetch requests as soon as
we realise the server lied about RFC822.SIZE (which is when we receive
the last chunk, and it runs over the size we expected).

It may be that there's a neat and simple way to handle all of the above,
and if so then patches would be welcome -- but personally, I just
couldn't be bothered to think too hard about it. There were more
pressing matters to attend to, like implementing QRESYNC support.

--
David Woodhouse Open Source Technology Centre
David.Woodhouse(a)intel.com Intel Corporation

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/