Increased timeouts with v0.17.0

[klinde-inuvo]

We have seen an increase timeouts with v0.17.0. We have rolled back to our previous version, v0.14.0 becuase of the issue. Included is the percent increase by bidder:

[CarsonBanov]

@klinde-inuvo We have seen a decrease in impressions going from 0.15 to 0.17.
Will report here if we find a cause, but it could be the timeouts you mention or something else that would cause issues for both of us.
Any other folks seeing issues?

[edit] #915 (comment)

[mkendall07]

@klinde-inuvo
Can you let me know how your measuring timeouts? Thanks

[klinde-inuvo]

Using GA event information, I compared yesterday's timeouts (the only day we ran v0.17) to the 7-day average of the previous week.

[Darijusch]

we have the same issue we went back to 0.14.0 since all versions afterwards were broken im terms of building or the impressions went down

[headerbidding]

Which version was this "bug" introduced?

[mkendall07]

I'm not sure how a bug in Prebid.js would impact timeouts from bidders but I suppose it's possible. Seems more likely to me that something with the timeout event could be the culprit - is anyone measuring timeouts other than that event that could point to something else?

[agabel]

We noticed a dramatic decrease in fill rates (cut in half) after upgrading from 0.15 to 0.16 and then to 0.17 as well. We were not measuring timeouts until 0.17 so I cannot give you exact numbers to compare.

[protonate]

@klinde-inuvo it may be that reporting of "no bid" responses are now more accurate. In version 0.14.0 most bidders did not pass a bidObject when calling bidmanager.createBid for a "no bid" case, and this meant it couldn't be reconciled to the originating request. With each successive version more bidders and all new bidders conformed to the standard of passing the bidObject when creating a "no bid" Bid.

Here's the timeout difference you supplied with the bidder support for "no bid" Bids:

Bidder	Timeout Increase	v0.14.0	v0.17.0
sovrn	0%	Yes	Yes
pulsepoint	62%	Yes	Yes
appnexus	136%	No	Yes
sonobi	138%	No	Yes
sekindo	151%	No	Yes
brealtime	220%	No (alias)	Yes
springserve	1059%	No	Yes

There are other factors impacting these numbers I'm sure but if you (and everyone reporting issues here) could test for that with your data it would be very helpful.

[klinde-inuvo]

@protonate - Are you saying "no bid" responses are being counted as timeouts with v0.17 and before they they weren't? Can "no bid" responses be tracked as a separate GA event?
I contacted our account rep at springserve about the issue.. He said other pubs had had the same problem. They are investigating.
Unfortunately, we have other priorities and will not be able to look at v0.17 again for a while.

[protonate]

@klinde-inuvo understandable, thanks for reporting your findings, and any additional info about your data, such as total requests / responses, will be helpful as well.

@prebid/springserve - please let us know if you are aware of any impact v0.17.0 is having on timeouts.

@headerbidding - there is not yet a known bug, I've applied the label to bring attention to the issue and get additional community feedback.

@Darijusch - not sure what you mean in terms of "broken in terms of building", if you can provide more info here.

@mmilleruva @agabel @CarsonBanov - if you are able to provide additional info it will be appreciated. You can email me, address is in my profile.

[Darijusch]

@protonate broken i ment only version 0.15.X versions which didn't work of the webpack misconfiguration and then with version 0.16.0 and higher we noticed the drop of impressions and rolled back to 0.14.0

[mkendall07]

@Darijusch
Can give us the website so we can take a look? Email or posting here is fine. Thanks

[CarsonBanov]

@protonate
In our most recent test, we found there to be negligible difference between Prebid 15 and Prebid 17 in terms of impressions and revenue.
The previous impression difference was attributable to our own experiment design and due to less caching of our Prebid 17 bundle (compared with existing Prebid 15 bundle). I would encourage other folks who have seen similar issues to consider their experiment design and caching (and/or perhaps just wait longer in their AB tests). Or results have 6,639,302 impressions for Prebid 15 and 6,641,925 impressions for Prebid 17 over the same time preriod (so identical).

[protonate]

Changes made with #989 and #990 address some issues that may have caused an auction to end early or a "bids back" callback to not fire. Prebid 0.19.0 is released now and can be tested to see if discrepancies are reduced.

[protonate]

Closing this as resolved, if any further info please reopen or create a new issue.