-
-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FIX] privacy_consent: Avoid race condition when sending emails #29
[FIX] privacy_consent: Avoid race condition when sending emails #29
Conversation
It could happen that, while Odoo is still sending emails, a subject receives it and clicks on accept/reject links. In such case, he'd get a 404 error because the record wouldn't exist yet in the database. That's because the DB commit was made only after processing all the sent emails. We need to commit in advance to make sure that doesn't happen.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks 👍
You must never commit inside a transaction. This can generate other side effects, like errors in the middle. Please use other technique. |
Using commits when sending emails is a supported feature. It's an exception to the rule, because when an email is sent, a rollback could leave the database unsynced with the real world. Another approach I thought about was to generate consents in a method and send them in another. However, that just wouldn't work, because the UX of this addon includes a button to "send all remaining consents now", and that's what should happen when the button is clicked. I'd have preferred also to use the |
Please confirm me that this commit doesn't put in the database data outside this method (that's to say, it commits all unsaved data from other threads on threaded mode or from other workers on worker mode). For me the ideal solution is:
|
Odoo is the one that takes care of giving a different cursor to each thread/worker, so the worst that could happen is that something from before is committed. It shouldn't happen as for this addon because the uses for this method are 1) user clicks button to send mails, or 2) cron sends mails.
This module has specific requirements regarding sending mails. Namely: it must be logged without the token URL, but sent with it. I already spent quite a few hours in the past to get to a point where this works fine, but as always it's not documented and my memory is not so good, so... do you really want me to spend all that time in this addon?
I noticed that, although the user doesn't get a 404 while the emails are being sent, the welcome controller won't render until the queue is emptied. It seems like, even with this commit, the 2nd transaction (email click processing) has to wait for the mails to be sent, so with current patch the situation is not ideal yet. I could split this into 2 crons:
There would be then the problem of UX. The user expects immediate sending of mails when clicking the button. Maybe I should relabel it? |
OK, if you say there's no problem in proper commit isolation, let's continue this way then. |
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. Note to self: never use again `post_message_with_template()`.
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. Note to self: never use again `post_message_with_template()`.
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. Note to self: never use again `post_message_with_template()`.
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`.
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`.
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`.
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`.
Before #29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`.
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`.
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent. To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient. So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met: * E-mail must be enqueued, no matter if there are less or more than 50 consents to send. * The template must be processed per record. * In an ideal world, a `cr.commit()` must be issued after each sent mail. The metod that was being used: * Didn't allow to use `auto_commit` mode. * Only allowed to render the template per record if called with `composition_mode="mass_mail"`. * Only allowed to enqueue emails if called with `composition_mode="mass_post"`. Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used. I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default: * Renders the template per record * Enqueues the email * The email queue is cleared in `auto_commit=True` mode. So, from now on, problems are gone: * The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them. * The mail queue manager will send them later, in `auto_commit` mode. * After sending the e-mail, this module will set the `privacy.consent` record as `sent`. * Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster. * Instructions in the README and text in the "generate" button are updated to reflect this new behavior. * Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot. * No hardcoded commits. * No locked transactions. * BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message. Note to self: never use again `post_message_with_template()`. Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/ Update privacy_consent.pot Update translation files Updated by "Update PO files to match POT (msgmerge)" hook in Weblate. Translation: data-protection-10.0/data-protection-10.0-privacy_consent Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
It could happen that, while Odoo is still sending emails, a subject receives it and clicks on accept/reject links.
In such case, he'd get a 404 error because the record wouldn't exist yet in the database. That's because the DB commit was made only after processing all the sent emails.
We need to commit in advance to make sure that doesn't happen.
@Tecnativa