How to access original recipient in sieve scripts

[SapuSeven]

Hi,
I have docker-mailserver configured with a catch-all mailbox as described in the FAQ by editing postfix-virtual.cf:

@example.com user1@example.com

Now I want to perform some actions on specific recipient-addresses, e.g. messages to fun@example.com should be moved to the folder Fun.

I tried a sieve script like this:

if envelope :localpart "To" "fun"{
[... do stuff]
}

but it didn't work.

To debug, I tried the following script:

if envelope :matches "To" "*" {
  fileinto "${1}";
}

just to see where it ends up, and looking at the result it always ends up in user1@example.com.
In the logs, I can also see a line like this:

postfix/smtp-amavis/smtp[131332]: 312321321: to=<user1@example.com>, orig_to=<fun@example.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=0.48, delays=0.27/0.02/0.01/0.18, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 123123123)

Apparently, the envelope filter in the sieve script is always looking at the final to-address.

Is there any way to make filtering with a catch-all by to-address possible?

[polarathene]

I think I came across a solution for that when I was writing up these notes, skip to near the end under the user-and-cc-local-alias.txt section.

Those parts should have relevant links.

You can shell into the DMS container running and use nano /etc/postfix/master.cf to edit that config, with postfix reload after saving to refresh those settings.

Virtual aliases like you'd use for your catch-all should be managed by virtual, which we configure by default to deliver through lmtp to Dovecot:

docker-mailserver/target/scripts/startup/setup.d/postfix.sh

Line 82 in a815bf5

postconf 'virtual_transport = lmtp:unix:/var/run/dovecot/lmtp'

Thus the advice should be related to master.cf adding those extra config settings to lmtp:

docker-mailserver/target/postfix/master.cf

Line 74 in a815bf5

lmtp unix - - n - - lmtp

When you know the settings you need, they can be added to our postfix-master.cf config file to apply for you when the container starts. (EDIT: Seems like you will need to handle a change that it doesn't support unfortunately)

postfix-master.cf is fed into a postconf command (postconf -P each_line_here). So instead you'll have to opt for our user-patches.sh support for now (let us know if you get it working):

#!/bin/bash

# Required for the Postfix to send with original recipient headers for Sieve to use:
# Change from default limit of 50 to 1,
# Postfix will send mail with multiple recipients to Dovecot one at a time
# instead of delegating to Dovecot to manage all at once:
postconf lmtp_destination_recipient_limit=1

# Modify the `lmtp` service to use the `lmtp` transport command with extra `flags=DO` arg:
# This will enable both `Delivered-To:` and `X-Original-To` headers for Sieve to have access:
postconf -F 'lmtp/unix/command=lmtp flags=DO'

Those two commands you could use in the DMS container like I described earlier for testing, and if they work then persist that via user-patches.sh.

There might be some other settings you'd need to look into, and the Sieve script maybe needs to look at those newly added headers instead, should be the X-Original-To that you want.

I don't have time to investigate further than I did in my link, or apply the advice I'm providing here to verify unfortunately, but I'd appreciate hearing back what works for you so others can benefit, it'd also help towards a feature request issue for someone to contribute a feature/docs for 👍

[SapuSeven]

Thanks a lot for your detailed investigation and explanation!
I tested the proposed changes (lmtp_destination_recipient_limit and lmtp flags), now there's an X-Original-To header present in the message, but it has the same value as the To-address (in my case user1@example.com).

[polarathene]

@SapuSeven sorry for the late answer, but I've found time to actually investigate now. Amavis service seems to affect the X-Original-To header not being the expected value. That can probably be fixed (such as preventing the related master.cf service for Amavis from modifying that header), but in the meantime if you disable that (or switch to rspamd I think), it should work correctly.

You'll need to adjust your sieve script as well. I've provided a compose.yaml you can run via copy/paste to reproduce: https://github.com/orgs/docker-mailserver/discussions/3898#discussioncomment-10263045

[SapuSeven]

Alright, I am really not qualified enough to further debug this issue, so until someone has a better idea I will use a global sieve script that I found here:
https://community.mailcow.email/d/1081-catchall-and-x-original-to-header/5

It takes the original to-address from the X-Spamd-Result header.
It's quite an ugly hack and I have no idea how it handles multiple recipients, but it works well enough for my purpose.

I also needed to add the editheader sieve extension in dovecot config: sieve_global_extensions = +editheader

[polarathene]

Yeah sorry, I'm not aware of any other maintainer familiar with Sieve and related service config to get the headers applied. I assume it's possible, just might require a fair amount of time experimenting with configs.

You only mentioned the X-Original-To header, and not the Delivered-To header? The descriptions I shared above indicate they're potentially different. The envelope recipient should be the original one, and from the sounds of it with X-Original-To it may be after it resolved the alias.

---

It takes the original to-address from the X-Spamd-Result header.

That actually might be worth looking into too. If you disable anti-spam so that header isn't present (ENABLE_AMAVIS=0), this will skip a little dance of sending the mail arriving to Amavis, which then sends it back to Postfix afterwards re-queuing it for actual delivery.

Potentially the virtual alias may have already been resolved before that latter step, so disabling Amavis may reveal that issue. Then we just need to figure out the extra config changes to accommodate that. The alternative anti-spam service is rspamd which is implemented differently via a Postfix milter instead of content filter, thus doesn't do that transport dance.

Clearly that information was at least available earlier if you're getting it from that header :)

---

Beyond those two avenues to investigate, I only shared how to configure that final delivery via lmtp. There is similar settings supported by other transports/services in master.cf that could occur earlier where it may be relevant if neither of the above suggestions are successful.

[polarathene]

I also got the same values in the headers

Context? Did you have Amavis disabled?

Did you try any of my other suggestions?

[polarathene]

@FeeeeK disable Amavis, follow the working example I provided here: https://github.com/orgs/docker-mailserver/discussions/3898#discussioncomment-10263045

[SapuSeven]

Another update:
For some reason, the Spamd header workaround stopped working. I looked at the headers and noticed that PREVIOUSLY_DELIVERED was missing in spamd-results.
However, "To" now contains the address the email was originally sent to, and "Delivered-To" the final destination.
So no idea why, but now everything works as expected.

[SapuSeven]

I applied all the settings from your original post, then I added the spamd workaround from my last post.
I haven't changed anything since then.
Good idea with writing it down in a separate answer, I will do that when I get around to it.

[FeeeeK]

For me, the default installation already allows me to get the original address from the To header without having to change anything in the config

[polarathene]

Good idea with writing it down in a separate answer, I will do that when I get around to it.

👍 Thanks!

without having to change anything in the config

For aliases with more than one recipient? (eg: marketing@example.com => john.doe@example.com, jane.doe@example.com), or as the original example for the discussion mentions, a catch-all address @example.com => john.doe@example.com. You're getting the original full @example.com address?

[matrixes]

I'm using Docker Mailserver 12.1, having set up a catch-all address through aliases and I experience issues with Sieve filtering.
I'm not sure where Sieve puts its logs, so I'm only looking at "/var/log/mail.log" and the headers of the received mail, and "X-Original-To" is missing, which means my filter for that header never gets processed.

I have done the following:

• Read through @polarathene's notes in post https://github.com/orgs/docker-mailserver/discussions/3898#discussioncomment-8526599 and to the best of my ability, interpreted what I can do to get my Sieve filter to work. It took a couple reads before I groked that the mention of "/etc/postfix/master.cf" is just that, a mention of it, and not something that should/needs to be edited.

1. Edited "./[DMS]/config/user-patches.sh" and added

#!/bin/bash

# Required for the Postfix to send with original recipient headers for Sieve to use:
# Change from default limit of 50 to 1,
# Postfix will send mail with multiple recipients to Dovecot one at a time
# instead of delegating to Dovecot to manage all at once:
postconf lmtp_destination_recipient_limit=1

# Modify the `lmtp` service to use the `lmtp` transport command with extra `flags=DO` arg:
# This will enable both `Delivered-To:` and `X-Original-To` headers for Sieve to have access:
postconf -F 'lmtp/unix/command=lmtp flags=DO'

2. Restarted DMS
3. Sent a test email that hits my catch-all
4. Verified that at least "X-Original-To" is included in the headers ("Delivered-To" has, as far as I know, always been included (I assume as part of defaults/spec), and it's still there containing values.)

I don't know how this will affect multiple recipients, as I don't have my mail server set up for that.

Thanks to everyone involved in this discussion, and especially to @polarathene for the detailed suggestions and instructions. I had already put in a day's worth of googling without getting anywhere.

[polarathene]

Thanks for chiming in @matrixes is there a reason you've not upgraded to a newer release of DMS yet btw? (be sure to go through our CHANGELOG.md file for any breaking changes documented for each major version to be aware of)

---

I had already put in a day's worth of googling without getting anywhere.

The sieve docs have had some improvements added for upcoming DMS v15.0 (specifically the sub-addressing examples and notes), but I've still got to revisit the info covered in this discussion as it is something I'd like to be easier for users to discover.

I managed to take a look and it was fairly simple to get working if I understood correctly what users are wanting to do in this discussion with Sieve. With the DMS docs link I provided and this other reference for Sieve syntax, you should be able to adjust this as you'd like?

# DISCLAIMER: This is an example config to demonstrate functionality,
# It does not represent any considerations for secure production deployments.
services:
  dms:
    image: mailserver/docker-mailserver:edge # :14.0
    hostname: mail.example.test
    environment:
      # Additional changes seem to be required if you have amavis enabled (maybe with rspamd too?)
      ENABLE_AMAVIS: 0
    # NOTE: I am using the Docker Compose `configs` feature here instead of `volumes` that you'd normally use for files.
    # This is embeds the config files into the same `compose.yaml` example so you can copy/paste to try it yourself.
    configs:
      # User accounts and virtual aliases (the catch-all wildcard) are all configured in advance for the demo:
      - source: dms-accounts
        target: /tmp/docker-mailserver/postfix-accounts.cf
      - source: dms-aliases
        target: /tmp/docker-mailserver/postfix-virtual.cf
      # Sieve script is associated to a specific account/mailbox:
      - source: dms-sieve
        target: /tmp/docker-mailserver/john.doe@example.test.dovecot.sieve
      - source: user-patches
        target: /tmp/docker-mailserver/user-patches.sh

  # OPTIONAL: A webmail service for quick and easy local testing via a web browser:
  roundcube:
    image: roundcube/roundcubemail:latest
    ports:
      - 80:80
    environment:
      - ROUNDCUBEMAIL_DB_TYPE=sqlite
      - ROUNDCUBEMAIL_SKIN=elastic
      - ROUNDCUBEMAIL_DEFAULT_HOST=tls://mail.example.test
      - ROUNDCUBEMAIL_SMTP_SERVER=tls://mail.example.test

# NOTE: The Docker Compose `configs.<name>.content` feature is supported in Docker Compose releases since 2024.
#
# Docker Compose has a variable interpolation feature which matches the `$` character, which we do not want to mistakenly use.
# - File content below avoids that feature via it's escape syntax of repeating that character (`$$`).
# - If you copy/paste to separate files for `volumes` make sure you revert that change.
configs:
  # Created via `setup email add john.doe@example.test secret`
  # - Username: `john.doe@example.test`
  # - Password: `secret`
  dms-accounts:
    content: |
      john.doe@example.test|{SHA512-CRYPT}$$6$$sbgFRCmQ.KWS5ryb$$EsWrlYosiadgdUOxCBHY0DQ3qFbeudDhNMqHs6jZt.8gmxUwiLVy738knqkHD4zj4amkb296HFqQ3yDq4UXt8.

  dms-aliases:
    content: |
      @whatever.test john.doe@example.test

  # Enable the `X-Original-To` header by forcing Postfix to deliver to individual recipients,
  # instead of delegating Dovecot to manage delivery for multiple recipients on it's end.
  user-patches:
    content: |
      postconf lmtp_destination_recipient_limit=1
      postconf -F 'lmtp/unix/command=lmtp flags=DO'

  # The sieve script will check for a `address` in the `X-Original-To` header and:
  # 1. Take the `:localpart` (left-side of `@` in the address) of that header value.
  # 2. Ensure it matches the regex pattern and then provide capture groups within the branch.
  # 3. The remainder of the example is adapted from the DMS docs linked.
  dms-sieve:
    content: |
      require ["fileinto", "mailbox", "variables", "regex"];

      # Check if the mail was originally from expected `azure` or `aws` addresses:
      if address :localpart :regex "X-Original-To" "^cloud-(azure|aws)$" {
        # Create a variable `tag`, with the captured value normalized to lowercase (SoCIAL => Social)
        set :lower :upperfirst "tag" "$${1}";

        # Store the mail into a folder with the tag name, nested under your inbox folder:
        if mailboxexists "INBOX.$${tag}" {
          fileinto "INBOX.$${tag}";
        } else {
          fileinto :create "INBOX.$${tag}";
        }
      }

Test sending mail to an address that the pattern would match, you can do so via the roundcube web UI or via CLI:

# Authenticated mail submission on port 587 with the john.doe@example.test account:
# NOTE: No TLS configured thus `--tls` option omitted, normally that would be required.
docker compose exec dms swaks --ehlo example.test --server mail.example.test --port 587 --auth PLAIN --auth-user john.doe@example.test --auth-password secret --from jane.doe@example.test --to cloud-azure@whatever.test

You'll find that delivers mail to the mailbox of john.doe@example.test, the Sieve script then matches cloud-azure and creates + stores it in the azure sub folder of the inbox.

• Amavis is configured with a separate service in master.cf, it "re-delivers" the mail after processing back into Postfix, which is why it seems to lose that original recipient address with X-Original-To... something like that.
• Rspamd is implemented differently (as a milter), so I don't think it'd have the same problem. It'll be the default in a future major release of DMS, so best to configure for that if you want anti-spam support.

---

I don't know how this will affect multiple recipients, as I don't have my mail server set up for that.

We tend to discourage this as DMS generally assumes aliases only ever map to a single recipient. This is now better documented for DMS v15.0 (see account management page in :edge version docs). Users can try it, but if they do run into any issues it's probably out of scope for us to support.

Other than that, for sieve there is global vs user scripts. I've demonstrated user-specific usage, so only mail arriving for that user is processed with the sieve script 👍

---

which means my filter for that header never gets processed.

Yup, this seems to be due to envelope and header to being matched in prior examples? Better to match on the actual intended X-Original-To header :)