Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[13.0][MIG] privacy_consent #44

Closed

Conversation

techultrasolutions
Copy link
Member

Privacy - Consent

This module allows the user to define a set of subjects (partners)
affected by any data processing activity, and establish
a process to ask them for consent to include them in that activity.

For those that need explicit consent as a lawfulness base for personal data
processing, as required by GDPR (article 6.1.a), this module provides the
needed tools to automate it.

yajo and others added 30 commits August 2, 2020 16:23
Currently translated at 100,0% (74 of 74 strings)

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/pt/
Currently translated at 98.6% (73 of 74 strings)

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/pt/
Currently translated at 100.0% (74 of 74 strings)

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/pt/
The purpose of this `@post_install` no longer exists, and by removing these decorators, we avoid test failures in case some custom addon customizes the welcome message.
Before this patch, all privacy consent emails were forced to be sent under a context with `mark_consent_sent=True` to track the state change and trigger the attached server action.

When using the automatic mode, if a separate cron job started running the mail queue, it could happen that mails were being sent from that another worker, losing the context key and, as such, being sent without token and without being marked as sent nor executing the attached server action (if any).

To avoid this problem, now the context dependency is removed. After all, the `mail.mail` object is only created when sending in `mass_mail` model, so other kind of notifications or messages are not affected. When a mail of type `mass_mail` is sent, we can assume that it is asking for consent and we can move draft consent requests to sent.
Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
It could happen that, while Odoo is still sending emails, a subject receives it and clicks on accept/reject links.

In such case, he'd get a 404 error because the record wouldn't exist yet in the database. That's because the DB commit was made only after processing all the sent emails.

We need to commit in advance to make sure that doesn't happen.
Currently translated at 79.5% (58 of 73 strings)

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/de/
Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Before OCA#29 there was a race condition where an email could be sent while the same transaction that created the `privacy.consent` record still wasn't committed, producing a 404 error if the user clicked on "Accept" or "Reject" before all mails were sent.

To avoid that, a raw `cr.commit()` was issued, but this produced another situation where the user had to wait until the full email queue is cleared to get his page loaded. It wasn't an error, but a long queue meant several minutes waiting, and it's ulikely that an average human is so patient.

So, here's the final fix (I hope!). The main problem was that I was looking in the wrong place to send the email. It turns out that the `self.post_message_with_template()` method is absolutely helpless in the case at hand, where these criteria must be met:

* E-mail must be enqueued, no matter if there are less or more than 50 consents to send.
* The template must be processed per record.
* In an ideal world, a `cr.commit()` must be issued after each sent mail.

The metod that was being used:

* Didn't allow to use `auto_commit` mode.
* Only allowed to render the template per record if called with `composition_mode="mass_mail"`.
* Only allowed to enqueue emails if called with `composition_mode="mass_post"`.

Obviously, I cannot set 2 different values for `composition_mode`, so a different strategy had to be used.

I discovered that the `mail.template` model has a helpful method called `send_mail()` that, by default:

* Renders the template per record
* Enqueues the email
* The email queue is cleared in `auto_commit=True` mode.

So, from now on, problems are gone:

* The user click, or the cron run, will just generate the missing `privacy.consent` records and enqueue mails for them.
* The mail queue manager will send them later, in `auto_commit` mode.
* After sending the e-mail, this module will set the `privacy.consent` record as `sent`.
* Thanks to *not* sending the email, the process the user faces when he hits the "generate" button is faster.
* Instructions in the README and text in the "generate" button are updated to reflect this new behavior.
* Thanks to the `auto_commit` feature, if Odoo is rebooted in the middle of a mail queue clearance, the records that were sent remain properly marked as sent, and the missing mails will be sent after the next boot.
* No hardcoded commits.
* No locked transactions.
* BTW I discovered that 2 different emails were created when creating a new consent. I started using `mail_create_nolog=True` to avoid that problem and only log a single creation message.

Note to self: never use again `post_message_with_template()`.
Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-10.0/data-protection-10.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-10-0/data-protection-10-0-privacy_consent/
- Partner's `opt_out` no longer exists. Using `mail.blacklist` now.
- Tests updated to support that change.
- Test workarounds removed.
- Duplicated-field-name-in-model warning removed.
- Use create multi where possible.
Co-Authored-By: Alexandre Díaz <alexandre.diaz@tecnativa.com>
oca-transbot and others added 7 commits August 2, 2020 16:23
Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: data-protection-12.0/data-protection-12.0-privacy_consent
Translate-URL: https://translation.odoo-community.org/projects/data-protection-12-0/data-protection-12-0-privacy_consent/
Due to a bug present in the `_postprocess_sent_message` code, if a consent wasn't successfully sent, still the consent got marked as sent.

It should stay as draft. Modify tests to test this new behavior.

@Tecnativa TT24457
@techultrasolutions techultrasolutions mentioned this pull request Aug 2, 2020
3 tasks
@techultrasolutions
Copy link
Member Author

Depends on #43

@treylade
Copy link

treylade commented Sep 14, 2020

I would really love to use this module. Any ideas on how soon this could be migrated to v13, @techultrasolutions?
Since the dependent module "privacy" looks like its shortly to be released?
Thank you for your work!

@techultrasolutions
Copy link
Member Author

I would really love to use this module. Any ideas on how soon this could be migrated to v13, @techultrasolutions?
Since the dependent module "privacy" looks like its shortly to be released?
Thank you for your work!

Thank for the appreciation.
We are trying our best. Expect this module in next week but depends on approval.
Cheers.

@yajo
Copy link
Member

yajo commented Jan 15, 2021

You're gonna kill me... it happened again.

I was going to open a PR and I suddenly realized I pushed to OCA 13.0 branch. Well, the good part is that the module is migrated already at ce67324. The bad one is that I got hit by OCA/odoo-module-migrator#47 yet again and we had no review again.

Sorry... closing.

@yajo yajo closed this Jan 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.