Can't search mail messages that are base64 encoded?

Question

Level 1

9 points

Can't search mail messages that are base64 encoded?

I get a lot of emails that are either multi-part base64-encoded, or that use a content-transfer-encoding of base64. Unless I'm mistaken, Mail seems unable to search through these encoded messages at all.

I can search the headers, such as the To, From, Subject, etc. But I can't search the entire message...that option is disabled when such an email is selected, and if I deselect it and search, nothing turns up. Similarly, a Smart Mailbox filtered by the entire message doesn't turn up results for the emails in particular, either. It does for non-base64 encoded emails, even those on a remote server; but not for local messages that are encoded as such.

Is there any way to fix this? Or do I need to use a "proper" email client like Entourage?

iMac 24", Mac OS X (10.5.4)

Posted on Aug 29, 2008 5:47 PM

Reply

Answer 1

Mulder

Level 6

8,990 points

Aug 29, 2008 7:49 PM in response to EvanK

You can search on any text in the email, as long as you know what you're looking for, and you haven't disabled Spotlight. But since Base64 is not human-readable, what's the point in searching for anything with that encoding? It's most likely an attachment, so the email client should decode it for you, thus rendering the need to search for it moot.

As for a "proper" email client like Entourage, perhaps you should define what is proper about it, and improper about Mail.

Mulder

Reply

Answer 2

EvanK Author

Level 1

9 points

Aug 29, 2008 11:16 PM in response to Mulder

Mulder wrote:
You can search on any text in the email, as long as you know what you're looking for, and you haven't disabled Spotlight. But since Base64 is not human-readable, what's the point in searching for anything with that encoding? It's most likely an attachment, so the email client should decode it for you, thus rendering the need to search for it moot.

I suppose I wasn't clear enough. +A handful+ of the messages in question are multi-part mime and have attachments, but the message parts of these are also base64-encoded. Most of the messages concerned are not multi-type and have no attachments, they are instead using a Content-Transfer-Encoding. This would be due either to the originating mail server, or another server somewhere along the delivery route.

My issue is that the message itself is apparently not searchable due to the encoding, while any messages not similarly encoded can be searched just fine.

Mulder wrote:
As for a "proper" email client like Entourage, perhaps you should define what is proper about it, and improper about Mail.

Yes, I was a bit unclear about that. The account these messages are coming from is a work account, which is hosted on an Exchange server. What I meant by "proper" was native Exchange support versus the pseudo-support Mail offers through IMAP. I despise Entourage, however, since the last time I used it, it ate its own database and took half my emails with it. But it also has no issue searching through content-encoded messages, whereas Mail apparently does.

Reply

Answer 3

Tom Gewecke

Level 10

122,240 points

Aug 30, 2008 4:44 AM in response to EvanK

Most of the messages concerned are not multi-type and have no attachments, they are instead using a Content-Transfer-Encoding. This would be due either to the originating mail server, or another server somewhere along the delivery route.

I think all email uses some kind of content-transfer-encoding, generated by the sending mail client, to transform the text into ascii. Normally ordinary text would be sent with 7bit (the default) or Quoted-Printable. I would expect Base64 to be used for text only when some kind of search protection was desired during transmission, or by a defective sending mail client.

Are you sure that other mail clients will search messages that arrive on your machine in Base64? If so, then Mail should also do so, and you can ask for this via this link:

http://www.apple.com/feedback/macosx.html

More info:

http://en.wikipedia.org/wiki/MIME#Content-Transfer-Encoding

Reply

Answer 4

Barney-15E

Level 10

122,252 points

Aug 30, 2008 5:23 AM in response to EvanK

What I meant by "proper" was native Exchange support

That's pretty funny. A non-standards based, proprietary system that goes out of its way to prevent competition is " proper."

Reply

Answer 5

EvanK Author

Level 1

9 points

Aug 30, 2008 11:15 AM in response to Barney-15E

Barney-15E wrote:
That's pretty funny. A non-standards based, proprietary system that goes out of its way to prevent competition is " proper."

Unfortunately, yes. A non-standards based proprietary system that almost everyone in the business world uses, so it is sort of the de-facto standard.

Reply

Answer 6

EvanK Author

Level 1

9 points

Aug 30, 2008 11:18 AM in response to Tom Gewecke

{quote:title=Tom Gewecke wrote:}
Are you sure that other mail clients will search messages that arrive on your machine in Base64? If so, then Mail should also do so, and you can ask for this via this link

Entourage has no problem searching them, and Thunderbird always worked for me in the past (but now I need something with some kind of Exchange integration, of which Mail does a little). Thanks for the link, I'll try that.

Reply

Answer 7

abates25

Level 1

0 points

Sep 15, 2008 10:36 PM in response to EvanK

As posted in a different topic, but....

It appears there is some underlying problem in the mail.app code which deals with MIME decoding, particularly, it would appear, to base64-encoded MIME attachments. I support a ~60 clients who use Mail.app as their e-mail client. A growing handful have had sporadic instances of receiving incorrectly decoded (read=not decoded) messages, though some have had it happen for a while, others have it happen with a greater frequency. The message generally, if not always, has an attachment (attachment type does not appear to be relevant), though the attachment is masked by base64 code, a database rebuild will not help, it is apparent on intel machines running 10.4.11 -> 10.5.4, viewing the same message on other clients and it looks fine, so it is specific to a Mail.app problem. An example would be this string of text:

--Apple-Mail-33--881167460
Content-Disposition: inline;
filename=xxxxxxxxInvite.jpg
Content-Transfer-Encoding: base64
Content-Type: image/jpeg;
x-mac-creator=3842494D;
x-unix-mode=0644;
x-mac-type=4A504547;
name="xxxxxxxxInvite.jpg"
Content-Id: <352143E2-EE39-45F3-85B6-8ED1068FA99A>

/9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAZAAA/+4ADkFkb2JlAGTAAAAAAf/b
AIQAAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQICAgICAgICAgIC
AwMDAwMDAwMDAwEBAQEBAQECAQECAgIBAgIDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMD
AwMDAwMDAwMDAwMDAwMDAwMD/8AAEQgEAAKTAwERAAIRAQMRAf/EAQAAAQACAwADAQEBAAAAAAAA
AAAHCAUGCQIDBAoBCwEBAAIDAQEBAQEAAAAAAAAAAAMEAQIFBgcICQoQAAAGAgECBAEECgwHCgoD
EQECAwQFBgAHCBESIRMUCRUxQSIWUWEjtTZ2puYXGHGBMpMkVNRmtjdnd5GhQlIzlgqxwWJykiVX
1xkagqKywtJDUzQmWKM1VVa3ONFzg7PDREUneOFjtHWGEQACAQMDAgMDCAcFAwUOAA8BAgMAEQQh
EgUxE0EiBlEyFGFxgaEjMxUHwdHhQlJicvCRJBYIsUM0ktLiUyXxgqKyY3OTs3S0NXU2F0RUtcKj
02TUVTcYg8M4Vv/aAAwDAQACEQMRAD8A6zZ fapUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKU
xSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUx
SmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxSmKUxS
etc., etc. where the length of the code is dependent on the size of the attachment which is supposed to be displayed. I'd love to blame something else - the sending or receiving mail server, the sending mail client, a M'soft issue...something other than Mail.app - but it is a Mail.app problem, and it has been a problem for a long time now and Apple is not addressing it :-/

Reply

Answer 8

Sep 16, 2008 6:53 AM in response to EvanK

FWIW, I have just had a related experience with base64 in Mail.app. We are running tests of a multipart newsletter sent from a php-based CMS (Typo3). The text showed up fine in almost any conceivable email-client and webmail-system. But it failed in Mail. The first small part of the Mail with some graphics was fine, but only about 1/6 of the mail was shown. I copied the base64-part and decoded it in TextMate, and only the first part of the html-code came out. I discovered the base64 was cut up by 7 sets of LF(and/or)CR + space. When I removed those the base64 converted beautifully to valid HTML. The php-script was corrected so linefeeds were removed before encoding and all is fine now.

I'm not sure if this can be helpful to you, but I think we can gather from this that Mail does not seem to be quite as forgiven to the base64 as other mail clients or decoders. Also this is not the only situation where LF/CR is an issue in data exchange between OS X and the rest of the world. I wished this would be solved once and for all.

Reply

Answer 9

abates25

Level 1

0 points

Sep 16, 2008 11:55 AM in response to jespersoholm

Thanks for the response - in the case of html, yes, removing the extra CR/LF's does clean up the code and allow it to be displayed correctlty, but unfortunately, in the case of the base64 code for other attachments (jpegs, .doc's, etc), there are no interjected CR/LF's to remove...it is just straight (and what looks like clean) code.

Reply