Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] privacy_consent: Avoid race condition when sending emails #29

Conversation

yajo
Copy link
Member

@yajo yajo commented May 7, 2019

It could happen that, while Odoo is still sending emails, a subject receives it and clicks on accept/reject links.

In such case, he'd get a 404 error because the record wouldn't exist yet in the database. That's because the DB commit was made only after processing all the sent emails.

We need to commit in advance to make sure that doesn't happen.

@Tecnativa

It could happen that, while Odoo is still sending emails, a subject receives it and clicks on accept/reject links.

In such case, he'd get a 404 error because the record wouldn't exist yet in the database. That's because the DB commit was made only after processing all the sent emails.

We need to commit in advance to make sure that doesn't happen.
@yajo yajo self-assigned this May 7, 2019
@yajo yajo added this to the 10.0 milestone May 7, 2019
@yajo yajo added the bug label May 7, 2019
Copy link
Member

@rafaelbn rafaelbn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍

@pedrobaeza
Copy link
Member

You must never commit inside a transaction. This can generate other side effects, like errors in the middle. Please use other technique.

@yajo
Copy link
Member Author

yajo commented May 8, 2019

Using commits when sending emails is a supported feature. It's an exception to the rule, because when an email is sent, a rollback could leave the database unsynced with the real world.

Another approach I thought about was to generate consents in a method and send them in another. However, that just wouldn't work, because the UX of this addon includes a button to "send all remaining consents now", and that's what should happen when the button is clicked.

I'd have preferred also to use the auto_commit feature, but it is missing when calling message_post_with_template, as this module does, so no luck either. Maybe for v13 if they merge odoo/odoo#33235 I'll be able to do it better.

@pedrobaeza
Copy link
Member

Please confirm me that this commit doesn't put in the database data outside this method (that's to say, it commits all unsaved data from other threads on threaded mode or from other workers on worker mode).

For me the ideal solution is:

  • Disable at the beginning the cron "Email Queue Manager". If not possible because it's running, then return an error message (the same as Odoo now does when trying to install a module and a cron is running).
  • Enqueue the messages as mail.mail in this process. This is done through the mail_mode field/context argument if I don't remember bad.
  • Enable the cron.
  • And now, they will flow at their time with no danger of unsynced operations.

@yajo
Copy link
Member Author

yajo commented May 8, 2019

Please confirm me that this commit doesn't put in the database data outside this method (that's to say, it commits all unsaved data from other threads on threaded mode or from other workers on worker mode).

Odoo is the one that takes care of giving a different cursor to each thread/worker, so the worst that could happen is that something from before is committed. It shouldn't happen as for this addon because the uses for this method are 1) user clicks button to send mails, or 2) cron sends mails.

Enqueue the messages as mail.mail in this process. This is done through the mail_mode field/context argument if I don't remember bad.

This module has specific requirements regarding sending mails. Namely: it must be logged without the token URL, but sent with it. I already spent quite a few hours in the past to get to a point where this works fine, but as always it's not documented and my memory is not so good, so... do you really want me to spend all that time in this addon?

Disable at the beginning the cron [...] Enable the cron.

I noticed that, although the user doesn't get a 404 while the emails are being sent, the welcome controller won't render until the queue is emptied. It seems like, even with this commit, the 2nd transaction (email click processing) has to wait for the mails to be sent, so with current patch the situation is not ideal yet.

I could split this into 2 crons:

  1. create missing requests
  2. mail them. This one could be actually the standard one (if I manage to use the mail queue and still fill the requirements, see above), or I could just use the method used now with a new cron and leave that for future improvements.

There would be then the problem of UX. The user expects immediate sending of mails when clicking the button. Maybe I should relabel it?

@pedrobaeza
Copy link
Member

OK, if you say there's no problem in proper commit isolation, let's continue this way then.

@pedrobaeza pedrobaeza merged commit 030d5f6 into OCA:10.0 May 8, 2019
@pedrobaeza pedrobaeza deleted the 10.0-privacy_consent-avoid_race_sending_emails branch May 8, 2019 18:04
yajo added a commit to Tecnativa/data-protection that referenced this pull request May 13, 2019
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.

Note to self: never use again `post_message_with_template()`.
yajo added a commit to Tecnativa/data-protection that referenced this pull request May 13, 2019
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.

Note to self: never use again `post_message_with_template()`.
yajo added a commit to Tecnativa/data-protection that referenced this pull request May 13, 2019
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.

Note to self: never use again `post_message_with_template()`.
yajo added a commit to Tecnativa/data-protection that referenced this pull request May 14, 2019
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.
fkantelberg pushed a commit to initOS/data-protection that referenced this pull request Jul 4, 2019
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.
yajo added a commit to Tecnativa/data-protection that referenced this pull request Jul 8, 2019
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.
techultrasolutions pushed a commit to techultrasolutions/data-protection that referenced this pull request Aug 2, 2020
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.
yajo added a commit that referenced this pull request Jan 15, 2021
Before #29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.
valtterilattu pushed a commit to valtterilattu/data-protection that referenced this pull request Jul 27, 2021
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.
victoralmau pushed a commit to Tecnativa/data-protection that referenced this pull request Nov 10, 2021
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
victoralmau pushed a commit to Tecnativa/data-protection that referenced this pull request Nov 11, 2021
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
victoralmau pushed a commit to Tecnativa/data-protection that referenced this pull request Nov 15, 2021
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
victoralmau pushed a commit to Tecnativa/data-protection that referenced this pull request Oct 10, 2022
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
tde-banana-odoo pushed a commit to tde-banana-odoo/oca-data-protection that referenced this pull request Oct 24, 2022
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
tde-banana-odoo pushed a commit to tde-banana-odoo/oca-data-protection that referenced this pull request Oct 24, 2022
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
tde-banana-odoo pushed a commit to tde-banana-odoo/oca-data-protection that referenced this pull request Oct 25, 2022
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
tde-banana-odoo pushed a commit to tde-banana-odoo/oca-data-protection that referenced this pull request Oct 25, 2022
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
tde-banana-odoo pushed a commit to tde-banana-odoo/oca-data-protection that referenced this pull request Oct 26, 2022
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
tde-banana-odoo pushed a commit to tde-banana-odoo/oca-data-protection that referenced this pull request Oct 26, 2022
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
victoralmau pushed a commit to Tecnativa/data-protection that referenced this pull request Nov 3, 2022
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/

Update privacy_consent.pot

Update translation files

Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants