Module dependencies on open source libraries

[deepthivenkat]

Type of issue

There are two main issues that have been brought to our attention as something that other modules do not do:

1. external dependency and 2) auto-sends

Description

Liveintent prebid user id module adds a dependent open source library called liveconnect. Liveconnect is a fully configurable open source library and was predominantly added as a dependency to avoid having to write duplicate code and maintain two separate repositories overtime. Right now we are able to recommend prebid and liveconnect interchangeably primarily because of this.
We have a small team of developers and maintaining two separate code bases that support email newsletter use cases and providing stable identifiers to the web. Separating them out will be a huge overload for us and the main reason we wanted to have this is to prevent our publishers from having both liveconnect and prebid user id module on the page at the same time. The original idea was discussed in this PR and approved by prebid: #4803

The cookies set primarily is required to create mappings used to serve subsequent resolution requests

Other information

Objections and Mitigation steps:

1. We will reintroduce liveconnect minimal mode and the corresponding documentation and promise to keep it alive for all newer versions of prebid. I saw the usage of prebid module minimal mode reduce recently and decided to not add it to our documentation anymore - because i did not have context on the pr listed above.
2. Every id module will need to make a call to an external endpoint to resolve for identifiers and what liveintent module is doing is not different in any manner compared to other user id modules.
3. We will also provide configurable options and making cookie setting configurable and turned off by default and add it to our documentation and maintain them.

To summarize, this is still what liveconnect does:
depending on the configuration, does or does not set a 1st party identifier(you can choose between LS, CookieJar, or no cookie setting at all)
also depending on the configuration, lets you synchronize first party identifiers
issues a pixel call towards LiveIntent collector by default. One can event implement its own data collection and use it for that. The data that's being sent as a plain image pixel can be found here
additionally, it provides a simple resolve function that will use the data already being used for collection to resolve the defined identifiers to a stable one, and that is the only AJAX call. Of course, the timeout can be specified via config.

and all the arguments made here are still valid: #4803

Creating this issue to discuss options and get guidance on the best path forward.

[premesh]

I think my concern here is just two fold

• the module allows the ability to make an external api calls outside of the prebid framework
• the module allows the ability to read cookies and/or local storage outside of the prebid framework

Though I can resonate with your reasons to use an external library to do this, it does not follow the same principals and guidelines that the prebid group has put forth and upheld for others in the same situation. It is the prebid team's job to ensure the framework is able to unilaterally protect it's users and provide an even playing field for all contributors.

[patmmccann]

Of note, liveconnect adds 40k to the bundle compressed, which if a publisher did say 10x, would make prebid unmanagable. Our request has often been to support a publisher loading live-connect independently from the build. Whichever the default is, the flexibility would be nice.

[patmmccann]

Related: #10838; if we split a subset a modules into a second part. Eg uid2 cstg and other ids in a second phase, would start to solve some the the growing bundle size issue

[3link]

Hi @premesh. Nice to meet you.

the module allows the ability to read cookies and/or local storage outside of the prebid framework

During initialization, LiveConnect gets passed Prebid's storage manager (../src/storageManager.js#getStorageManager) and interacts with both, cookies and local storage, only using this manager. This should be inline with Prebid priciple and guidelines. To be able to better address your concern, can you please provide more information about what specifically you are referring to?

the module allows the ability to make an external api calls outside of the prebid framework

I assume you are referring to auto-collect when the module is loaded.

This call sends the same data as the call that happens when the identity resolution is requested from our module. The goal is to start incorporating this data into our model as soon as possible since this step is time consuming and thus to be able to respond to resolve requests as quickly as possible with the most recent data.

This is also documented in https://docs.prebid.org/dev-docs/modules/userid-submodules/liveintent.html - including configuration parameters for the auto-collect. LiveIntent makes sure to share this documentation when on-boarding new publishers.

provide an even playing field for all contributors

I hope we can work out a course of action here.

[3link]

Of note, liveconnect adds 40k to the bundle compressed, which if a publisher did say 10x, would make prebid unmanagable. Our request has often been to support a publisher loading live-connect independently from the build. Whichever the default is, the flexibility would be nice.

@patmmccann This is a great proposal. I will discuss it with my team.

[3link]

Related: #10838; if we split a subset a modules into a second part. Eg uid2 cstg and other ids in a second phase, would start to solve some the the growing bundle size issue

@patmmccann Definitely something to consider. The grows in the recent past can be mostly attributed to supporting more and more identifiers.

[premesh]

Hey @3link ,

During initialization, LiveConnect gets passed Prebid's storage manager (../src/storageManager.js#getStorageManager) and interacts with both, cookies and local storage, only using this manager. This should be inline with Prebid priciple and guidelines. To be able to better address your concern, can you please provide more information about what specifically you are referring to?

I see the use of the storageManager which should govern the access to reading data however the practice of doing this outside of the liveIntent module itself in an external module is as I understand it against the policy that Prebid has. As to why third party modules are not allowed i would defer to @patmmccann as that would be a broader rule not directed only at this use case.

This call sends the same data as the call that happens when the identity resolution is requested from our module. The goal is to start incorporating this data into our model as soon as possible since this step is time consuming and thus to be able to respond to resolve requests as quickly as possible with the most recent data.

I think this again issue here is on principal as no other module is allowed to make calls to any end point outside of the prebid framework. I believe there other other identity modules that work similarly to yours that face this same issue and are forced to rely on the publisher implementing their module independently of the prebid identity module. I would suggest working with the prebid.js team to accommodate the functionality you need with calling an end point in a generic fashion so that other modules can do this as well .

In my opinion asking the publisher to install a separate 3rd party script on the page is a hurdle along with your stated concerns around identity resolution speed which given the current implementation is an disadvantage that any other identity resolution module has for which the LiveIntent does not.

Can you work with the prebid group to bring your needed functionality into the framework if possible ? If not it just seems like an unfair advantage as the Prebid group would not approve a module today doing the same thing.

[deepthivenkat]

Meeting notes from identity PMC: Liveintent module calls external libraries. They will be formally asked to stop.
we decided to go ahead with the rewrite features to prebid and break it into chunks and work on it consistently.