Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prebid 7: auction-specific first party data and refactoring of FPD internals #8338

Merged
merged 4 commits into from
May 3, 2022
Merged

Prebid 7: auction-specific first party data and refactoring of FPD internals #8338

merged 4 commits into from
May 3, 2022

Conversation

dgirardi
Copy link
Collaborator

@dgirardi dgirardi commented Apr 28, 2022 •
edited
Loading

Type of change

  • Feature
  • Refactoring (no functional changes, no api changes)

Description of change

This restructures the way Prebid handles first party data to enable auction-specific FPD (#7651)

The changes are:

  • pbjs.requestBids now accepts an ortb2 parameter, which will get merged into (and overrides) the global ortb2 config only for that particular auction
  • requestBids now "freezes" FPD for a particular auction into global (getConfig('ortb2') + the auction-specific ortb2) and bidder fragments that get passed down to a new hook, startAuction (and further down the stack to auctionManager, adapterManager, etc)
  • FPD modules and RTD modules have been refactored to hook into startAuction (instead of requestBids), and they read from and modify its FPD parameters instead of calling getConfig / setConfig
  • bid requests and bidder requests (the parameters passed to bidders' buildRequests) are now augmented with an ortb2 property that replaces the use of getConfig('ortb2') for bid adapters. All adapters have been refactored to read from those instead of calling getConfig.
  • FPD submodules (currently just enrichment and validation) now run on every auction, and the global refreshFpd has been removed - with FPD now being potentially different for each one, I don't think the previous run-once approach makes sense. That said, both submodules can be improved to memoize some of their processing - that's not included in this PR.

The choice to put ortb2 into every bid request is due to some adapters doing their FPD logic in the validation phase (isBidRequestValid) or in transformBidParameters, where the bidder request is not available.

Note that two RTD providers - weborama and jwPlayer - were already populating bid.ortb2 under some circumstances. Most adapters were blind to it (I know only of proxistore looking for it - related is also #8320). I have refactored those two to update the bidder-level ortb2 config instead.

Other information

Docs PR TBD

@dgirardi
This restructures the way Prebid handles first party data to enable a…
a7f79dc 
…uction-specific FPD (prebid#7651)

The changes are:

 - `pbjs.requestBids` now accepts an `ortb2` parameter, which will get merged into (and overrides) the global `ortb2` config only for that particular auction
 - `requestBids` now "freezes" FPD for a particular auction into global (`getConfig('ortb2')` + the auction-specific `ortb2`) and bidder fragments that get passed down to a new hook, `startAuction` (and further down the stack to auctionManager, adapterManager, etc)
 - FPD modules and RTD modules have been refactored to hook into `startAuction` (instead of `requestBids`), and they read from and modify its FPD parameters instead of calling `getConfig` / `setConfig`
 - bid requests and bidd*er* requests (the parameters passed to bidders' `buildRequests`) are now augmented with an `ortb2` property that replaces the use of `getConfig('ortb2')` for bid adapters. All adapters have been refactored to read from those instead of calling `getConfig`.

The choice to put `ortb2` into every bid request is due to some adapters doing their FPD logic in the validation phase (`isBidRequestValid`) or in `transformBidParameters`, where the bidd*er* request is not available.

Note that two RTD providers - weborama and jwPlayer - were already populating `bid.ortb2` under some circumstances. Most adapters were blind to it (I know only of proxistore looking for it - related is also prebid#8320). I have refactored those two to update the bidder-level ortb2 config instead.

Squashed commit of the following:

commit 171b5ad6e23837ebbe964c7539345bd825b5545f
Merge: f9d3b96a fa20923
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu Apr 28 12:17:28 2022 -0700

    Merge branch 'pb7-merge' into auction-fpd

commit f9d3b96afd2cb5477d488209c450802ca5c135da
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu Apr 28 11:48:22 2022 -0700

    Revert "Revert "Prohibit getConfig('ortb2')""

    This reverts commit 439e926ed0323e8128e82fa3e6968eefb6ee5daf.

commit 6667c1568dcaf6aaeee3ec02b7609466f5f1ade1
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu Apr 28 11:44:49 2022 -0700

    Fix lint

commit 1f3d7cabf462a3ad287acd93087a9e13f3756f84
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu Apr 28 11:42:10 2022 -0700

    convert invalid setting of bid.ortb2 from RTD providers: weborama

commit 6e7f7d031c5a5da484f14d27a79dce2131d8ff22
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu Apr 28 11:08:40 2022 -0700

    convert invalid setting of bid.ortb2 from RTD providers: jwPlayer

commit 5d9d3b02de35f61b7959a2a923a9116c594fcb91
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu Apr 28 10:34:24 2022 -0700

    convert RTD: weborama

commit b27aada8b9659f0565c66bb3b26ae883358738aa
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu Apr 28 10:11:19 2022 -0700

    convert rtd: sirdata

commit fca00b89588454092240feb89d77e35e90882fa4
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu Apr 28 10:01:03 2022 -0700

    convert rtd: permutive

commit 6ba257a04788a09a214b58d6df1500c2441bf49a
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 26 09:15:52 2022 -0700

    convert rtd: jwPlayer

commit a0fd8b0128b071cddf8e3a81cca552a2ae284271
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 26 09:14:07 2022 -0700

    convert rtd: im

commit 3e2aa858cae8334c3fb670d2094686bb7e689a0f
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 26 09:12:39 2022 -0700

    convert rtd: idWard

commit 39d2139c9617c305c1c0d68948ad0c447342e2da
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 26 09:09:57 2022 -0700

    convert rtd: halo

commit c6f3852795c25e59814c34ad9d0a202127132658
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon Apr 25 13:48:17 2022 -0700

    convert rtd: hadron

commit 39365892ccf7c4aa5acc19913e01ab6e3a4915f6
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon Apr 25 13:31:44 2022 -0700

    convert rtd: dgkeyword

commit 0966b92d68122e7677b75f9b5dfad91788ec5ffa
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon Apr 25 13:25:55 2022 -0700

    convert rtd: brandmetrics

commit 54fe3211ed7967b32bc4d1c81a24d2eb44a806ea
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon Apr 25 13:17:26 2022 -0700

    convert rtd: akamai

commit c569fb7cf364318991700a01874797b58f492475
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon Apr 25 13:09:41 2022 -0700

    convert rtd: airgrid

commit d70ad20d59ff63d9e22b7efdd4c87fafe21fddd0
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon Apr 25 12:58:45 2022 -0700

    convert rtd: adloox

commit 11086f1b400c7f4870afd7f8979309747f0d2eff
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon Apr 25 12:07:50 2022 -0700

    Group global and bidder level ortb2 in startAuction

commit b77fc06954a78902c5abbad583c31ef49e7cab12
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon Apr 25 11:41:27 2022 -0700

    Convert fpdModule

commit 439e926ed0323e8128e82fa3e6968eefb6ee5daf
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Mon Apr 25 10:25:13 2022 -0700

    Revert "Prohibit getConfig('ortb2')"

    This reverts commit 1d93427b271f7b9da490c33c099522a54f1bc614.

commit 1d93427b271f7b9da490c33c099522a54f1bc614
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 12:34:40 2022 -0700

    Prohibit getConfig('ortb2')

commit 118420f7bf58c5d7d63a6456009100efdb6543bd
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 11:20:15 2022 -0700

    Convert PBS adapter

commit 8b85b9bd378f0bae4b10da5a00cdcc2854fac414
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 11:06:28 2022 -0700

    Pass FPD to PBS adapter

commit 3d577a861b01c34ebf9e818d415caf9596b98d7b
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 10:28:57 2022 -0700

    Group ortb2 fragments into one object when passing them down the stack

commit 0199ad2a8a1af033f6787d5c9c113638f755947f
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 09:23:00 2022 -0700

    convert bidders: improveddigital

commit 13ea77111409e1ec7c2608d950bab9195af23337
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 09:14:54 2022 -0700

    Fix lint

commit e16f7045e998c66d76a90ba59c4f204db5064170
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 09:09:05 2022 -0700

    convert bidders: yieldlab

commit 38158faf94561337f218887410e253517c8e914e
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 09:06:05 2022 -0700

    convert bidders: trustx

commit 9f5b4d5dabaa2df3b406d755b79f8b91e804e258
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 09:01:52 2022 -0700

    convert bidders: improveddigital

commit 357dc821f83d64e6b434e73ffa77eea98c065928
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 08:56:55 2022 -0700

    convert bidders: grid

commit 386692ebd28467dad70bae35f47f9470495186e6
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 08:47:38 2022 -0700

    convert bidders: pubmatic

commit 5bdb5ec3995e26a6fd209e2c7d1301a2028404ab
Merge: cf662cb8 d8b0509
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Fri Apr 22 08:27:42 2022 -0700

    Merge branch 'master' into auction-fpd

commit cf662cb80078c3fea5532d0eb56d2148abecca94
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:36:31 2022 -0700

    convert bidders: openx

commit 1485cf33297dd7116c09557f39a0eb5102590eeb
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:30:57 2022 -0700

    Fix lint

commit 33ffd2b1bdb2a7a8c0f27344ec34354696ffd0f6
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:29:27 2022 -0700

    convert bidders: complete for all bidders calling getConfig('ortb2')

commit 409b79b8152751993b0c9fbadfc8c9669e3e7a39
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:28:15 2022 -0700

    convert bidders: yahoo

commit 5d12c257818ec82f1d4f0f3c9f5d8ccda14f556c
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:22:28 2022 -0700

    convert bidders: ttd

commit 30ff4373558c0f0028514e6a0e389f6874ad694a
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:19:44 2022 -0700

    convert bidders: triplelift

commit de1d97382bed073aad6ff06e381f5bccad8c7586
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:18:06 2022 -0700

    convert bidders: sovrn

commit 85a4db11ac70c0c00546cc177ef4b7206066dfe0
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:14:39 2022 -0700

    convert bidders: sonobi

commit 13cf0352f9374a50da9b627fdc53cb34f1a2b090
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:13:41 2022 -0700

    convert bidders: smaato

commit d31679020084b4b8277faa87adbff6c10402ff12
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:12:13 2022 -0700

    convert bidders: sharethrough

commit aac7150b1133c78cc37f9918d109af760c4b7134
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:10:12 2022 -0700

    convert bidders: rubicon

commit 1af8b5103c9004ff3342c8a1d5a7e13b1365ae04
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:05:04 2022 -0700

    convert bidders: pubmatic

commit 85e7de147fd868cb01d46d24bb8bf56400949ed2
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 15:03:51 2022 -0700

    convert bidders: pubmatic

commit 50d106168206de6a88581731af2b2883c9b85a16
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:59:57 2022 -0700

    convert bidders: ozone

commit 821d995e0d57e7027035962a6fb2d304f45b880c
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:56:54 2022 -0700

    convert bidders: optout

commit 16cf6b4ac9d9a1ff259f9e365043ca2ec88eb07b
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:55:31 2022 -0700

    convert bidders: nobid

commit 0c2c350fb33c25d24017d2e59385c4204e37caf0
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:53:50 2022 -0700

    convert bidders: minutemedia

commit 19f253486dab65d65a15a4b7bd758dd27c95a373
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:52:11 2022 -0700

    convert bidders: mediakeys

commit ba66be0f2c596fabaeb9380245334fa208318840
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:50:24 2022 -0700

    convert bidders: livewrapped

commit e25e3587dbae8ff12adcd57c55a234c918f5f42b
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:47:18 2022 -0700

    convert bidders: ix

commit 9a65715a1b3604036f073288048bbecc81a70b85
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:32:07 2022 -0700

    convert bidders: glimpse

commit 1a3cff66882ca56093fd38e3a054fb764c92078d
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:31:18 2022 -0700

    convert bidders: glimpse

commit 00bf5aa75c3a1b29299f2ecc9e0ee1416afc2024
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:25:32 2022 -0700

    convert bidders: criteo

commit 893ff7ed635e81266b6910d7c1623a4b7b86484c
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:19:49 2022 -0700

    convert bidders: conversant

commit 6b44388bae02d364464dab3c3360d9a0ce8d289e
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:06:25 2022 -0700

    convert bidders: beachfront

commit 92c6f9780f51f6508c36c2dd20701492fb979db1
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 14:01:01 2022 -0700

    convert bidders: amx

commit afc57dd4b2ba911820ea3f19ab43a73f2e7a4b87
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 13:58:37 2022 -0700

    convert bidders: adxcg

commit bcb8955f4edd597696a5fe70196976b20a2e611e
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 13:51:09 2022 -0700

    convert bidders: adrelevantis

commit aeade3ee9da816d74b02af28aa67a474b0a23920
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 13:46:02 2022 -0700

    convert bidders: adnuntius

commit 675ae0830526b44b82d5060f0b6ba687943695e8
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 13:27:07 2022 -0700

    convert bidders: adkernel, admixer

commit c7e68088ffda095a6be88b492ef31d45c98df36a
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 13:24:40 2022 -0700

    Convert bidders: adagio, adf

commit 4991a79c7aa88131d77ea97ed8458bb36212178b
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 12:56:51 2022 -0700

    Make FPD available in bidderRequest

commit 9224806f2457001b12763d7493de7422025880fa
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Tue Apr 19 12:28:18 2022 -0700

    Pass ortb2 down to adapterManager

commit 532539815910448cf4d758ff2328fc00af2d1179
Author: Demetrio Girardi <dgirardi@prebid.org>
Date:   Thu Apr 14 10:45:05 2022 -0700

    Make auction FPD-aware
@dgirardi
Merge branch 'prebid-7' into auction-fpd-merge
a8b59ca
@dgirardi
Copy link
Collaborator Author

Tagging:

@dgirardi dgirardi changed the title Prebid Core & multiple modules: auction-specific first party data and refactoring of FPD internals Prebid 7: auction-specific first party data and refactoring of FPD internals Apr 28, 2022
@peczenyj
Copy link
Contributor

peczenyj commented Apr 29, 2022 •
edited
Loading

LGTM

@dgirardi I am currently working on the weborama RTD module, should l wait to rebase it properly in the next days? Or this will take more time?

@karimMourra karimMourra mentioned this pull request Apr 29, 2022
Closed
2 tasks
karimMourra
karimMourra reviewed Apr 29, 2022
View reviewed changes
modules/jwplayerRtdProvider.js Outdated
@@ -158,7 +158,7 @@ export function enrichAdUnits(adUnits) {
const contentId = getContentId(vat.mediaID);
const contentData = getContentData(vat.segments);
const targeting = formatTargetingResponse(vat);
enrichBids(adUnit.bids, targeting, contentId, contentData);
enrichBids(ortb2Fragments?.bidder || {}, adUnit.bids, targeting, contentId, contentData);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dgirardi why the ? ? Based on the function signature, ortb2Fragments is always defined

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true, this or the default param value is superfluous - I missed it

@dgirardi
Copy link
Collaborator Author

@peczenyj this is a breaking change for version 7; you'll probably want to fix for 6 independently, using the current getBidderConfig / merge / setBidderConfig strategy.

@peczenyj
Copy link
Contributor

@peczenyj this is a breaking change for version 7; you'll probably want to fix for 6 independently, using the current getBidderConfig / merge / setBidderConfig strategy.

Well, I try to structure this refactor to be easier to rebase #8350

@dgirardi dgirardi linked an issue May 2, 2022 that may be closed by this pull request
Site.content.data for a specific [auction] #7651
Closed
patmmccann
patmmccann reviewed May 2, 2022
View reviewed changes
src/config.js
const conf = _getConfig();
Object.defineProperty(conf, 'ortb2', {
get: function () {
throw new Error('invalid access to \'orbt2\' config - use request parameters instead');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is applicable to bidders, but what if publishers what to access the config, for example, to achieve #6240

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the publisher API does not hit this code path - the global pbjs.getConfig and pbjs.readConfig are aliased to getAnyConfig and readAnyConfig.

@patmmccann patmmccann merged commit 3b3c8a5 into prebid:prebid-7 May 3, 2022
@patmmccann patmmccann mentioned this pull request May 23, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@desbo desbo desbo approved these changes

@patmmccann patmmccann patmmccann approved these changes

@karimMourra karimMourra karimMourra approved these changes

@mmoschovas mmoschovas mmoschovas approved these changes

Assignees
No one assigned
Labels
7.0 API Change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Site.content.data for a specific [auction]
6 participants
@dgirardi @peczenyj @desbo @patmmccann @karimMourra @mmoschovas