Difficult to test for consent to custom vendors when trying to set vendor exceptions

[spormeon]

Type of issue

bug or "i'm missing something"

Description

In V 8.16.0 this is being thrown "Activity control: TCF2 denied 'transmitEids' for 'bidder.pubmatic'" , its not been there until V8.16.0, quantcast CMP now uses tcf2.2

Its something to do with userIds: as when I comment userIds out, it goes away.

Its for video, no clue if effects display etc

Steps to reproduce

can see here in professor prebid extension: https://cdn.adysis.com/VideoDemo/DemoSite/pages/BBH.html

Test page

https://cdn.adysis.com/VideoDemo/DemoSite/pages/BBH.html

Expected results

no: Activity control: TCF2 denied 'transmitEids' for 'bidder.pubmatic' to be thrown

Actual results

Activity control: TCF2 denied 'transmitEids' for 'bidder.pubmatic' being thrown

Platform details

chrome, mac, V8.16.0

Other information

guessing something to do with #10435 , no docs, so no clue what to set , where etc

[dgirardi]

Docs are still in draft: prebid/prebid.github.io#4876

This is the relevant section:

By default, sending user IDs to bid adapters or RTD modules (the transmitEids activity) is not tied to a single TCF Purpose; instead it is allowed if one of these scenario is true:

• We have evidence of consent for the vendor and evidence of consent for any purpose between 2 and 10;
• We have evidence of consent for any purpose between 2 and 10, and the vendor is excepted through softVendorException in at least one of: basicAds, personalizedAds, or measurement;
• The vendor is excepted through vendorExceptions in at least one of basicAds, personalizedAds, or measurement.

This behavior can be changed to the same "basic enforcement" algorithm described above, tied to TCF Purpose 4, by setting eidsRequireP4Consent: true on a personalizedAds rule:

  ...
  rules: [{
    purpose: "personalizedAds",
   eidsRequireP4Consent: true
 }]

In your setup, even when clicking "accept all", pubmatic (GVLID 76) has neither consent nor LI for any purpose 2-10, so it fails the check above. Auctions (fetchBids) still work because you have it configured with enforceVendor: false - but the enforce flags are not taken in consideration for the default transmitEids rule.

As an immediate fix I suggest adding:

setConfig({
  consentManagement: {
    gdpr: {
       rules: [
         // ...
         {
            purpose: 'personalizedAds',
            enforcePurpose: false,
            enforceVendor: false,
            eidsRequireP4Consent: true
         }
       ]
    }
  }
})

I am actually surprised to see this issue since the default rule is so lax - I'll make sure to bring this up for review. Do you know why "accept all" is not enough to consent to pubmatic? Is it something you configure with the CMP?

[lcorrigall]

Going to move this to Needs Req, so we can have a further discussion on this with the PMC.

[bretg]

In short, this was thought of as a bug fix. Here's the background for discussion in the committee:

Over 2 years ago, there was discussion about whether EIDs were for ad personalization or for broader functionality. At the insistence of one of Prebid's User ID members, it was decided to compromise and allow EIDs to be sent whenever the TCF string contained consent for any purpose 2-10. This seems quite broad, lax even - if the user is giving permission for anything, then EIDs may be passed.

This consent-for-P2-to-P10 feature was implemented on Prebid Server, but never got implemented in Prebid.js. When this was recognized, we opened an issue (#10184) to close the gap. We also pushed to give publishers (and their lawyers) a way to define that P4 is required rather than the more lax P2-P10. This is the eidsRequireP4consent option.

We were thinking of this more as a bug fix than a breaking change, and we figured that getting consent for P2-P10 should be easy, but the algorithm does explicitly reject vendorexceptions, so that may affect more parties than expected. The goal is to flip the default to requiring P4 in a later major release.

And in any case, I would agree that the update could certainly have been documented more clearly in the release notes.

[spormeon]

@dgirardi Nothing specific to pubmatic has been set in the quantcast cmp by me. The CMP basically has "all" vendors allowed and i'm pressing "accept all" when consenting.

What does look different in the cmp is that Pubmatic is included in "IAB vendors", but not in "IAB vendors 2.2 "(tcf 2.2), CMP is set to use tcf 2.2. Quantcast messed up something?

[spormeon]

@dgirardi I flicked the CMP back to tcf 2.0 and the "error" goes away then, looks like quantcast might of messed up and not included pubmatic in their tcf 2.2 vendor list and thats whats throwing it all out?

[dgirardi]

@spormeon that's a question for Qantcast - I don't know why pubmatic is not in their 2.2 CMP.

[spormeon]

@dgirardi ye i've got onto their support but that means its potentially on 000's of pubs as they been sending emails to move to tcf2.2, so no idea if it opens a can of worms for Prebid V 8.16.0 users . They might fix in a day, or 3 months, who knows.

[spormeon]

apparently their not tcf 2.2 "operational" from quantcast support, thats why not on the list, so back to the drawing board if anyone has Q CMP tcf2.2 on and on V8.16.0 , including Pubmatic as bidder.
https://iabeurope.eu/vendor-list-tcf/

[patmmccann]

You seem to be advocating for illegal behavior. If pubmatic isn't consented to nor has it expressed legitimate interest in your CMP, don't send them anything , they have no legal bases for even getting a request

Why do you bother to enforce tcf at all?

@pm-harshad-mane as fyi

[spormeon]

@patmmccann I'm not "advocating" anything, its a "test" page, to show the issue thrown, If its switched to enforceVendor: true, with tcf2.2. Then fetchBids starts getting thrown "Activity control: TCF2 denied 'fetchBids' for 'bidder.pubmatic'". I think the real prblem is pubmatic arnt "operational" on tcf2.2 by the looks of it but your getting the info/ issue as its prebid throwing the info. I would guess its happening with "all" cmp's thats are now using, or got people to switch to use tcf2.2 @pm-azhar-mulla

[patmmccann]

I would suggest not giving them an override; if you do so, it will also affect users who deny all

[spormeon]

there seems to be another potential bug/ problem, when using tcf2.2 &
purpose: "basicAds",
enforcePurpose: true,
enforceVendor: true,
vendorExceptions: ['smaato','amx','sharethrough','pubmatic','pwbid','adform','rhythmone','minutemedia','smaradserver','sovrn','33across','openx','gumgum','ozone','ix']

pubmatic is allowed to fetchBids

i can put it on the test page , if you want?

[patmmccann]

You gave them a vendor exception in the above, why wouldn't you expect them to fetch bids?

[spormeon]

becasue its tcf2.2 and their not even in the list/operational, so thats the truth of source, otherwise we are sort of back to your point above of "advocating illegal acivity". I never quite get/ got why these "vendor exception" even exist, its either "enforce" or "dont enforce". Your stand is to enfocre to not "run the risk" of illegal activity. There should be "no choice" and that "clears everyone", no "work around" , no "ambiguity"

[bretg]

Prebid's regulatory philosophy is that the default behavior should be conservative, but that special cases exist and publishers need to have controls. We assume that if they research the exception scenarios, they want to be responsible for their actions.

Specifically in this case, say there's a certain bidder that doesn't pay the IAB fee to have a GVLID but the publisher wants to trust them because there's been a contract with due diligence. That bidder cannot be in the TCF string without a GVLID, so the vendor exception feature exists to allow them.

[spormeon]

@bretg Ok, I get this "scenario" but in the cmp they havent given "consent", as they cant consent, as their no longer in the cmp, this could be a "massive can of worms" for the pub, their the ones "responsible" but the "ad providor" is circumventing it, maybe as a "bad actor" or even just blind nievly/ oblivious to the fact their "not in the cmp" anymore

[pm-azhar-mulla]

@pm-manasi-moghe @pm-nitin-nimbalkar Can you please check if anything is needed for Pubmatic's adapter.
@kapil-tuptewar FYI

[spormeon]

follow on of this, if cmp module is set to 'iab', does that also pick up what are now "non iab vendors" such as pubmatic and amazon, as there is the ability in the quantcast cmp to add/ allow "non iab vendors" to gain "consent" on them, even when "allowing all" etc, pubmatic still throws the errors/ warnings:

no "exceptions" used:

no "activities" set:

its on a test page here:
https://cdn.adysis.com/VideoDemo/DemoSite/pages/BBH.html

[bretg]

does that also pick up what are now "non iab vendors"

Definitely not. Unclear what mechanism QC is using -- perhaps Google's 'additional consent'? Anyhow, Prebid doesn't parse anything other than TCF.

This is why the 'vendor exceptions' feature exists, though apparently 'vendor exceptions' aren't currently supported on this activity.

One thing you could do in your page code is read 'additional consent' from the CMP and dynamically add a vendor exception.

[patmmccann]

Given that you're pubmatic to your cmp but Prebid doesn't understand that your users have consented to that vendor, I now think your request to make the transmitEids activity control honor your vendor exceptions is quite reasonable, marking ready for dev.

[spormeon]

@patmmccann ok, but what happens here if say a user hasnt consented specifically to pubmatic or any one of the other 50 odd that have become "non operatational" and/ or has "pulled back consent" down the line. The "vendor exception" is gonna allow them, if consented, but allow them even if "pulled back consent". Equals a situation of an absolutle minefield

[patmmccann]

ok so you will have to access the cmp consent object to find if your custom vendor has been consented to before setting the vendor exception; or wait for pubmatic to get their paperwork in order

[dgirardi]

@spormeon this thread got very confusing, is an accurate summary of this "consent for Pubmatic does not work if you use this particular CMP and TCF 2.2"? IMO the only action Prebid can take is warn Pubmatic, which we did. My suggestion for you is to do the same; maybe also talk to your CMP vendor, and if it's an option roll back to the CMP version that works.

[spormeon]

pretty much ye, i'd go " "prebid" consent for Pubmatic does not work if you use this particular CMP and TCF 2.2" but its not a "prebid" problem, more a prob of 50 odd vendors not getting "operational" if thats becasuse of lax ( need to pull their fingers out) or a concious desicussion to not go tct.2.2 operational, I got no clue but they got a couple of weeks to do it, like patrick said with "get their papwerwork in order" and pull out their cheque books. I'll wait till 20th November see where the land lies then.

Quantcast are having a look at it anyway. Amazon came back with: "As a TCF Vendor, Amazon Advertising is working on adopting TCFv2.2 and is committed to the IAB’s deadline of 20th November 2023 to support the new version. We will inform you when our adoption of TCFv2.2 is complete. Until that time, we will not be able to monetize traffic containing TCFv2.2 consent strings. We will continue to monetize valid TCFv2.1 consent strings until November 2024."

[spormeon]

pubmatic have become "operational" now, so tcf2.2 prob for them sorted now.

[patmmccann]

I think we can close this with documentation at this point, where setting the vendor exception is dependent on examining the tcfapi response for a custom vendor opt in