Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hebrew Characters turn to Gibberish when ticket is received via 0Auth with Office365 #17396

Open
2 tasks done
NegroniBlast opened this issue Jul 1, 2024 · 8 comments
Open
2 tasks done

Hebrew Characters turn to Gibberish when ticket is received via 0Auth with Office365 #17396

NegroniBlast opened this issue Jul 1, 2024 · 8 comments
Labels
bug

Comments

@NegroniBlast
Copy link

NegroniBlast commented Jul 1, 2024 •
edited
Loading

Code of Conduct

  • I agree to follow this project's Code of Conduct

Is there an existing issue for this?

  • I have searched the existing issues

Version

10.0.12

Bug description

I installed GLPI 10.0.12 and after connecting our ticketing office365 tenant mailbox via 0auth, I tried opening tickets via email with Hebrew characters and I get gibberish instead in the subject and the body.

When I open it manually on the platform there's no problem and Hebrew is shown fine.

Relevant log output

No response

Page URL

No response

Steps To reproduce

  1. Connect receiver office365 mailbox with 0Auth to GLPI
  2. Setup automatic mailgate to retrieve the new emails as tickets
  3. Send email with Hebrew characters to this mailbox
  4. The new ticket is created, but with Hebrew content in Gibberish

Your GLPI setup information

Information about system installation & configuration
GLPI 10.0.12 ( => /var/www/glpi)
Installation mode: TARBALL
Current language:en_US
Server
 

Operating system: Linux REDACTED WEBSITE ADDRESS 5.15.0-113-generic #​123-Ubuntu SMP Mon Jun 10 08:16:17 UTC 2024 x86_64

PHP 8.1.2-1ubuntu2.18 fpm-fcgi (Core, FFI, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apcu, bz2, calendar, cgi-fcgi,

ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imagick, imap, intl, json, ldap, libxml,

mbstring, memcache, mysqli, mysqlnd, openssl, pcre, pdo_mysql, posix, pspell, readline, session, shmop, sockets, sodium,

standard, sysvmsg, sysvsem, sysvshm, tidy, tokenizer, xml, xmlreader, xmlrpc, xmlwriter, xsl, zip, zlib)

Setup: max_execution_time="30" memory_limit="128M" post_max_size="8M" safe_mode="" session.save_handler="files"

upload_max_filesize="2M" disable_functions=""

Software: nginx/1.18.0

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36

Server Software: Ubuntu 22.04

Server Version: 10.6.18-MariaDB-0ubuntu0.22.04.1

Server SQL Mode: STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

Parameters: glpi_admin@localhost/glpi_db

Host info: Localhost via UNIX socket


PHP version (8.1.2-1ubuntu2.18) is supported.

Sessions configuration is OK.

Allocated memory is sufficient.

mysqli extension is installed.

Following extensions are installed: dom, fileinfo, filter, libxml, json, simplexml, xmlreader, xmlwriter.

curl extension is installed.

gd extension is installed.

intl extension is installed.

zlib extension is installed.

The constant SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES is present.

Database engine version (10.6.18) is supported.

No files from previous GLPI version detected.

The log file has been created successfully.

Write access to /var/www/glpi/files/_cache has been validated.

Write access to /var/www/glpi/files/_cron has been validated.

Write access to /var/www/glpi/files has been validated.

Write access to /var/www/glpi/files/_dumps has been validated.

Write access to /var/www/glpi/files/_graphs has been validated.

Write access to /var/www/glpi/files/_lock has been validated.

Write access to /var/www/glpi/files/_pictures has been validated.

Write access to /var/www/glpi/files/_plugins has been validated.

Write access to /var/www/glpi/files/_rss has been validated.

Write access to /var/www/glpi/files/_sessions has been validated.

Write access to /var/www/glpi/files/_tmp has been validated.

Write access to /var/www/glpi/files/_uploads has been validated.


Web server root directory configuration seems safe.

Sessions configuration is secured.

OS and PHP are relying on 64 bits integers.

exif extension is installed.

ldap extension is installed.

openssl extension is installed.

Following extensions are installed: bz2, Phar, zip.

Zend OPcache extension is installed.

Following extensions are installed: ctype, iconv, mbstring, sodium.

Write access to /var/www/glpi/marketplace has been validated.

Timezones seems loaded in database.
GLPI constants
 

GLPI_ROOT: "/var/www/glpi"

GLPI_CONFIG_DIR: "/var/www/glpi/config"

GLPI_VAR_DIR: "/var/www/glpi/files"

GLPI_MARKETPLACE_DIR: "/var/www/glpi/marketplace"

GLPI_USE_CSRF_CHECK: "1"

GLPI_CSRF_EXPIRES: "7200"

GLPI_CSRF_MAX_TOKENS: "100"

GLPI_USE_IDOR_CHECK: "1"

GLPI_IDOR_EXPIRES: "7200"

GLPI_ALLOW_IFRAME_IN_RICH_TEXT: false

GLPI_SERVERSIDE_URL_ALLOWLIST: ["/^(https?|feed):\/\/[^@:]+(\/.*)?$/"]

GLPI_TELEMETRY_URI: "https://telemetry.glpi-project.org"

GLPI_INSTALL_MODE: "TARBALL"

GLPI_NETWORK_MAIL: "glpi@teclib.com"

GLPI_NETWORK_SERVICES: "https://services.glpi-network.com"

GLPI_MARKETPLACE_ALLOW_OVERRIDE: true

GLPI_MARKETPLACE_MANUAL_DOWNLOADS: true

GLPI_USER_AGENT_EXTRA_COMMENTS: ""

GLPI_DISABLE_ONLY_FULL_GROUP_BY_SQL_MODE: "1"

GLPI_AJAX_DASHBOARD: "1"

GLPI_CALDAV_IMPORT_STATE: 0

GLPI_DEMO_MODE: "0"

GLPI_CENTRAL_WARNINGS: "1"

GLPI_TEXT_MAXSIZE: "4000"

GLPI_DOC_DIR: "/var/www/glpi/files"

GLPI_CACHE_DIR: "/var/www/glpi/files/_cache"

GLPI_CRON_DIR: "/var/www/glpi/files/_cron"

GLPI_DUMP_DIR: "/var/www/glpi/files/_dumps"

GLPI_GRAPH_DIR: "/var/www/glpi/files/_graphs"

GLPI_LOCAL_I18N_DIR: "/var/www/glpi/files/_locales"

GLPI_LOCK_DIR: "/var/www/glpi/files/_lock"

GLPI_LOG_DIR: "/var/www/glpi/files/_log"

GLPI_PICTURE_DIR: "/var/www/glpi/files/_pictures"

GLPI_PLUGIN_DOC_DIR: "/var/www/glpi/files/_plugins"

GLPI_RSS_DIR: "/var/www/glpi/files/_rss"

GLPI_SESSION_DIR: "/var/www/glpi/files/_sessions"

GLPI_TMP_DIR: "/var/www/glpi/files/_tmp"

GLPI_UPLOAD_DIR: "/var/www/glpi/files/_uploads"

GLPI_INVENTORY_DIR: "/var/www/glpi/files/_inventories"

GLPI_NETWORK_REGISTRATION_API_URL: "https://services.glpi-network.com/api/registration/"

GLPI_MARKETPLACE_PLUGINS_API_URI: "https://services.glpi-network.com/api/marketplace/"

GLPI_I18N_DIR: "/var/www/glpi/locales"

GLPI_VERSION: "10.0.12"

GLPI_SCHEMA_VERSION: "10.0.12@b902ef5f5da967c1a9d0826ee8a7dce07d57cf3d"

GLPI_MARKETPLACE_PRERELEASES: false

GLPI_MIN_PHP: "7.4.0"

GLPI_MAX_PHP: "8.4.0"

GLPI_YEAR: "2024"
Libraries
 

htmlawed/htmlawed version 1.2.14 in (/var/www/glpi/vendor/htmlawed/htmlawed)

phpmailer/phpmailer version 6.8.0 in (/var/www/glpi/vendor/phpmailer/phpmailer/src)

simplepie/simplepie version 1.5.8 in (/var/www/glpi/vendor/simplepie/simplepie/library)

tecnickcom/tcpdf version 6.6.2 in (/var/www/glpi/vendor/tecnickcom/tcpdf)

michelf/php-markdown in (/var/www/glpi/vendor/michelf/php-markdown/Michelf)

true/punycode in (/var/www/glpi/vendor/true/punycode/src)

iamcal/lib_autolink in (/var/www/glpi/vendor/iamcal/lib_autolink)

sabre/dav in (/var/www/glpi/vendor/sabre/dav/lib/DAV)

sabre/http in (/var/www/glpi/vendor/sabre/http/lib)

sabre/uri in (/var/www/glpi/vendor/sabre/uri/lib)

sabre/vobject in (/var/www/glpi/vendor/sabre/vobject/lib)

laminas/laminas-i18n in (/var/www/glpi/vendor/laminas/laminas-i18n/src)

laminas/laminas-servicemanager in (/var/www/glpi/vendor/laminas/laminas-servicemanager/src)

monolog/monolog in (/var/www/glpi/vendor/monolog/monolog/src/Monolog)

sebastian/diff in (/var/www/glpi/vendor/sebastian/diff/src)

donatj/phpuseragentparser in (/var/www/glpi/vendor/donatj/phpuseragentparser/src/UserAgent)

elvanto/litemoji in (/var/www/glpi/vendor/elvanto/litemoji/src)

symfony/console in (/var/www/glpi/vendor/symfony/console)

scssphp/scssphp in (/var/www/glpi/vendor/scssphp/scssphp/src)

laminas/laminas-mail in (/var/www/glpi/vendor/laminas/laminas-mail/src/Protocol)

laminas/laminas-mime in (/var/www/glpi/vendor/laminas/laminas-mime/src)

rlanvin/php-rrule in (/var/www/glpi/vendor/rlanvin/php-rrule/src)

ramsey/uuid in (/var/www/glpi/vendor/ramsey/uuid/src)

psr/log in (/var/www/glpi/vendor/psr/log/Psr/Log)

psr/simple-cache in (/var/www/glpi/vendor/psr/simple-cache/src)

psr/cache in (/var/www/glpi/vendor/psr/cache/src)

league/csv in (/var/www/glpi/vendor/league/csv/src)

mexitek/phpcolors in (/var/www/glpi/vendor/mexitek/phpcolors/src/Mexitek/PHPColors)

guzzlehttp/guzzle in (/var/www/glpi/vendor/guzzlehttp/guzzle/src)

guzzlehttp/psr7 in (/var/www/glpi/vendor/guzzlehttp/psr7/src)

glpi-project/inventory_format in (/var/www/glpi/vendor/glpi-project/inventory_format/lib/php)

wapmorgan/unified-archive in (/var/www/glpi/vendor/wapmorgan/unified-archive/src)

paragonie/sodium_compat in (/var/www/glpi/vendor/paragonie/sodium_compat/src)

symfony/cache in (/var/www/glpi/vendor/symfony/cache)

html2text/html2text in (/var/www/glpi/vendor/html2text/html2text/src)

symfony/css-selector in (/var/www/glpi/vendor/symfony/css-selector)

symfony/dom-crawler in (/var/www/glpi/vendor/symfony/dom-crawler)

twig/twig in (/var/www/glpi/vendor/twig/twig/src)

twig/string-extra in (/var/www/glpi/vendor/twig/string-extra)

symfony/polyfill-ctype not found

symfony/polyfill-iconv not found

symfony/polyfill-mbstring not found

symfony/polyfill-php80 not found

symfony/polyfill-php81 not found

symfony/polyfill-php82 in (/var/www/glpi/vendor/symfony/polyfill-php82)

league/oauth2-client in (/var/www/glpi/vendor/league/oauth2-client/src/Provider)

league/oauth2-google in (/var/www/glpi/vendor/league/oauth2-google/src/Provider)

thenetworg/oauth2-azure in (/var/www/glpi/vendor/thenetworg/oauth2-azure/src/Provider)

phpCas version 1.3.8 in (/usr/share/php/CAS/source)
SQL replicas
 

Not active
Notifications
 

Way of sending emails: SMTP+TLS (REDACTED@octopuscs.com@smtp.office365.com)
Plugins list
 

news                 Name: Alerts                         Version: 1.12.2     State: Installed / not activated

Install Method: Marketplace

behaviors            Name: Behaviours                     Version: 2.7.3      State: Installed / not activated

Install Method: Marketplace

datainjection        Name: Data injection                 Version: 2.13.5     State: Installed / not activated

Install Method: Marketplace

archimap             Name: Diagrams                       Version: 3.3.4      State: Installed / not activated

Install Method: Marketplace

financialreports     Name: Financial reports              Version: 3.0.0      State: Installed / not activated

Install Method: Marketplace

addressing           Name: IP Adressing                   Version: 3.0.2      State: Installed / not activated

Install Method: Marketplace

mfa                  Name: MFA                            Version: 1.0.2      State: Installed / not activated

Install Method: Marketplace

mreporting           Name: More Reporting                 Version: 1.8.6      State: Installed / not activated

Install Method: Marketplace

oauthimap            Name: Oauth IMAP                     Version: 1.4.3      State: Enabled

Install Method: Marketplace

onetimesecret        Name: OneTimeSecret                  Version: 2.1.3      State: Installed / not activated

Install Method: Marketplace

pdf                  Name: Print to pdf                   Version: 3.0.0      State: Installed / not activated

Install Method: Marketplace

reports              Name: Reports                        Version: 1.16.0     State: Installed / not activated

Install Method: Marketplace

screenshot           Name: Screenshot                     Version: 2.0.3      State: Installed / not activated

Install Method: Marketplace

ticketfilter         Name: Ticketfilter                   Version: 1.3.2      State: Installed / not activated

Install Method: Marketplace

timelineticket       Name: Timeline of tickets            Version: 10.0+1.2   State: Installed / not activated

Install Method: Marketplace

vip                  Name: VIP                            Version: 1.8.2      State: Installed / not activated

Install Method: Marketplace

Anything else?

Screenshot 2024-07-01 at 14 07 51
@NegroniBlast
Copy link
Author

these might be relevant but i dont know if these fixes were implemented into 10.0.12 so i decided to open a new issue here
#7748
#4844

@NegroniBlast
Copy link
Author

this is the closest issue I found on the subject #7709

@cedric-anne
Copy link
Member

Hi,

Please check if the problem persists in GLPI 10.0.15. If problem persists, please attache the message into .eml format so we could add it to our test suite and see the results.

@NegroniBlast
Copy link
Author

Hi Cedric thanks for the quick reply. we upgrade to 10.0.15 and now the gibberish is different haha, still an encoding issue.
Screenshot 2024-07-01 at 16 23 32
any recommendations?

@NegroniBlast
Copy link
Author

Testing after 10.0.15 בדיקה נוספת אחרי מעבר לגרסה חדשה.eml.zip

it was blocking the .eml file so it's attached inside this compressed zip file.

@cedric-anne
Copy link
Member

I validate the issue. It sees that the problem is related to usage of iso-8859-8-i encoding.

*** PHP Warning (2): iconv(): Wrong encoding, conversion from "iso-8859-8-i" to "UTF-8//IGNORE" is not allowed in /var/www/glpi/src/MailCollector.php at line 2470
  Backtrace :
  src/MailCollector.php:2470                         iconv()
  src/MailCollector.php:1808                         MailCollector->getDecodedContent()
  src/MailCollector.php:1150                         MailCollector->getBody()
  src/MailCollector.php:811                          MailCollector->buildTicket()
  front/mailcollector.form.php:106                   MailCollector->collect()
  public/index.php:82                                require()

@NegroniBlast
Copy link
Author

NegroniBlast commented Aug 1, 2024 •
edited
Loading

Issue solved.

By editing Mailcollector.php I was able to edit the functions decodeMimeHeader and getDecodedContent to be able to handle Hebrew characters. the decoding function that works for incoming emails from M365 is mb_convert_encoding.

Attaching my code snippets of Mailcollector.php for your consideration. I will test some more next week but you can close this thread if you want.

  1. Subject wasn't using the subject decoding function - it was going to 'cleanSubject' instead. I replace cleanSubject with my decodeMimeHeader function.
  2. getDecodedContent had too many conditions at first, then i replaced it with a leaner version which is suitable for our needs.
    GLPI Mailcollector.php updated functions and code for Hebrew Encoding email messages.zip

@trasher
Copy link
Contributor

trasher commented Aug 2, 2024

Could you please open a pull request with your changes? Test suite will run with your changes; and it's easiest for us to review changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@trasher @cedric-anne @NegroniBlast