Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Broken character encoding when viewing .eml attachments #662

Closed
vivia opened this issue Nov 12, 2022 · 9 comments
Closed

Broken character encoding when viewing .eml attachments #662

vivia opened this issue Nov 12, 2022 · 9 comments
Labels
bug Something isn't working enhancement New feature or request

Comments

@vivia
Copy link

vivia commented Nov 12, 2022 •
edited
Loading

Describe the bug
When opening an .eml attachment with non-ascii characters, they are displayed wrong. It looks like the utf-8 is interpreted as iso-8859-1 and converted back to utf-8.

To Reproduce
Steps to reproduce the behavior:

  1. Go to an email with an .eml attachment (like an email forwarded from Evolution)
  2. Click on the .eml attachment to view it
  3. The attachment appears garbled

Expected behavior
The attachment is displayed properly

Screenshots
I sent this email and told the recipient to forward it back to me from Evolution:
image
This is the email in my inbox:
image
This is what appears when I click on the attachment:
image

Please complete the following information:

  • Browser: Firefox 106, Linux
  • IMAP daemon: dovecot
  • PHP version: 7.4.30
  • SnappyMail Version: 2.21.0 (running inside NextCloud)

Additional context

Copy-pasting from the email source:

--=-iOF5it9AMPUdK7+kBapl
Content-Disposition: inline
Content-Description: Forwarded message =?UTF-8?Q?=E2=80=94?=
 =?UTF-8?Q?_=CE=B5=CE=BB=CE=BB=CE=B7=CE=BD=CE=B9=CE=BA=CE=AC?= UTF-8
Content-Type: message/rfc822

Return-Path: [snip]
Delivered-To: [snip]
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=[snip];
 s=default; t=1668262491;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:mime-version:mime-version:content-type:content-type;
 bh=RZni3PSQyGiBGNe/LkeUdbQPaLYhS8MhkL5oDx8E7aQ=;
 b=DnUio4l2xuriYxbajf43008JC+jzzu2/zPH2Qf7/+OjmKRPS+EsJdXIWlfuMIEIedXCC1P
 gxKwbpveuOY/QJQjQ/wxvvidM0zjRHIXtgozgAGjbGGjcH4V96ZMBCDpvJ4Yjaynm7WvRE
 bRgepmqlne/dp8QUUxyb8KJKwcSkYlOELLzUlmVzA+MoZzu6IPTa2WeAidYuBdUOiDy2ow
 8pOGQ4tO8w4GMBGIRXBlHK4gEGBg7GShfJaiNjTrT0ornAGv75CVMHWCuWVbol8zgnxzFJ
 lMfpmQ8Zpoxa2n6rnWjxwSS3nrJGJQyle+1tQ5JG3I9Ki7o/FXv3lt2hp1AtIQ==
MIME-Version: 1.0
Date: Sat, 12 Nov 2022 14:14:50 +0000
Content-Type: multipart/alternative;
 boundary="a7404208-bd3e-4bf8-87ae-d977130d7ecb-1"
From: [snip]
Message-ID: [snip]
Subject: =?utf-8?B?zrXOu867zrfOvc65zrrOrCBVVEYtOA==?=
To: [snip]

--a7404208-bd3e-4bf8-87ae-d977130d7ecb-1
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

plain ascii
=CE=B5=CE=BB=CE=BB=CE=B7=CE=BD=CE=B9=CE=BA=CE=AC
=E6=97=A5=E6=9C=AC=E8=AA=9E
=F0=9F=90=B1
--a7404208-bd3e-4bf8-87ae-d977130d7ecb-1
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><meta http-equiv=3D"Content-Type" content=3D"t=
ext/html; charset=3Dutf-8"></head><body><div>plain ascii<br></div><div>=
=CE=B5=CE=BB=CE=BB=CE=B7=CE=BD=CE=B9=CE=BA=CE=AC</div><div>=E6=97=A5=E6=
=9C=AC=E8=AA=9E</div><div>=F0=9F=90=B1<br></div></body></html>
--a7404208-bd3e-4bf8-87ae-d977130d7ecb-1--

--=-iOF5it9AMPUdK7+kBapl--

This is how I recovered the email text, after iconv-ing the text from utf-8 to iso-8859-1 and letting my terminal interpret the result as utf-8 (right before the second plain ascii line I pressed ^D which cannot be pasted):

$ iconv -c -f utf-8 -t iso-8859-1
plain ascii
ελληνικά
���
�
plain ascii
ελληνικά
日本語
🐱
@vivia
Copy link
Author

vivia commented Nov 12, 2022 •
edited
Loading

I wanted to exclude the possibility that Evolution is doing something wrong, so I did the following tests:

  1. I viewed the same email from K-9 Mail on my phone. It appeared just fine.
  2. I forwarded the initial email as an attachment using SnappyMail insted of Evolution. It was broken.
  3. I forwarded the initial email as an attachment using SnappyMail, but to my gmail this time. It appeared just fine.

So I conclude that the problem is with SnappyMail.

@the-djmaze
Copy link
Owner

The EML is parsed in JavaScript by:
https://github.com/the-djmaze/snappymail/blob/master/dev/Mime/Parser.js

This is because of support for PGP/MIME decrypt.

I know the parser is simple and should be improved to handle this better, and support attachments inside the (decrypted) EML.

However, since it is a lot of work to modify SnappyMail source to get this all properly working, it could take a while.

@vivia
Copy link
Author

vivia commented Nov 12, 2022

How can I test my changes, is it enough to change the source inside the NextCloud apps directory and restart apache, or do I have to run something extra for changes to take effect? I found a few hardcoded ISO_8859_1 instances scattered in the code and I want to see if changing those will have any effect.

@the-djmaze
Copy link
Owner

To make modifications you must edit the /static/app.js
It's not in PHP to parse EML attachments.

the-djmaze pushed a commit that referenced this issue Nov 14, 2022
Support UTF8 in EML files #662
bc5194a
@the-djmaze
Copy link
Owner

I've added UTF-8 charset conversion.
Other character sets are currently not supported.

@vivia
Copy link
Author

vivia commented Nov 14, 2022

Thanks! I manually applied this to static/js/app.js and (after restarting apache and trying in a private window) it still doesn't work, am I missing something?

the-djmaze pushed a commit that referenced this issue Nov 15, 2022
Improved MIME charset decoding #662
f23e0d7
@the-djmaze
Copy link
Owner

Probably, just wait for next release?

@the-djmaze the-djmaze added bug Something isn't working enhancement New feature or request labels Nov 15, 2022
@the-djmaze
Copy link
Owner

https://github.com/the-djmaze/snappymail/releases/tag/v2.21.2

@vivia
Copy link
Author

vivia commented Nov 16, 2022

Oh, that was very fast, thanks a lot! I can confirm that it works fine in v2.21.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vivia @the-djmaze