Master merge (#3)

* Use the correct labels for cache performance metric (prebid#904) * PubMatic Adslot validation (prebid#886) * testing a few changes to validate adslot, more to follow * Changed AdSlot to optional parameter, modified validation and test cases for the same * removed imp id from adslot validation error msgs * assign banner size * remove TrimSpace where it is not needed as recommended in the review * Implementation of Categories Http fetcher (prebid#882) * Implementation of Categories Http fetcher -Added code to fetch data using http -Added previously loaded categories to run http request just at the first time * Moved common Category struct to stored_request * Added comments * Minor refactoring * Minor refactoring -Added verification if category exist in map * Minor refactoring * Minor refactoring -strings.Replace changed to strings.TrimSuffix * Improved error handling on Beachfront adapter (prebid#873) * Removed a redundant error message that was causing some confusion. * trying to turn '[]' into null and 200 inti 404 * not a goot path * looking at the status codes * I think I have covers all these bases. * checking for empty array response a failing sanely * removed an attempt to pointlessly reset the status code. * corrected and added test cases * Fix Rubicon bidder for multi-format impression processing (prebid#902) * Replace Dockerfile Alpine linux with Ubuntu (prebid#885) * Ubuntu dockerfile works, includes backups and out file * Ubuntu Dockerfile creates image successfully and prebid-server was tested with a sample request. Seems to work fine * Ubuntu 18.04 now builds, validates.sh, and curls sample request successfully * Removed sed command * [fix] broken Go 1.9 compatibility (prebid#910) This CL addresses Go 1.9 compatibility issue along with adding back Go 1.9 to travis testing setup to prevent such bugs. Issue: prebid#895 * Fixed "invalid BidType: " error for lifestreet adapter (prebid#893) * Fixed "invalid BidType: " error for lifestreet adapter * After run gofmt * Used standard bid.ext type to get bid.ext.prebid.type * [Currency support] Activate multi-currencies support (prebid#894) This CL is the last piece to activate currency conversion support in PBS. It activates currency support per default. Currency rates will be fetched once per hour (PrebidJS file is updated once a day). Issue: prebid#280 * ImproveDigital adapter: (prebid#887) - Add support for video - Add support for mobile/app - Add support for multibid responses - Extend optional parameters * Add a PI exemption environment variable to PBS (prebid#916) * Refactored to official name of config item * Changes suggested by Mansi Nahar * [Sharethrough] Add new Sharethrough Adapter (prebid#903) * wip * wip * wip * exploration for sharethrough prebid-server * WIP: updating sharethrough adapter to latest prebid version Co-authored-by: Chris Nguyen <cnguyen@sharethrough.com> * WIP: adding butler params to the request #164291358 Co-authored-by: Josh Becker <jbecker@sharethrough.com> * Manage bid sizes [#164291358] Co-authored-by: Josh Becker <jbecker@sharethrough.com> * Manage GDPR [#164291358] Co-authored-by: Josh Becker <jbecker@sharethrough.com> * Populate prebid-server version if provided [#164291358] Co-authored-by: Josh Becker <jbecker@sharethrough.com> * Refactor gdpr data extraction from request [#164291358] Co-authored-by: Eddy Pechuzal <epechuzal@sharethrough.com> * Add instant play capability [#164291358] Co-authored-by: Eddy Pechuzal <epechuzal@sharethrough.com> * Split in multiple files [#164291358] Co-authored-by: Eddy Pechuzal <epechuzal@sharethrough.com> * Add s2s-win beacon todo? replace server name by server id? [#164291358] Co-authored-by: Eddy Pechuzal <epechuzal@sharethrough.com> * Removing `server` param in s2s-win beacon (will be added in imp req) [#164291358] * Clean up code + enable syncer (prebid#4) [#165257793] * Proper error handling (prebid#6) * Proper error handling [#165745574] * Address review (baby clean up) + catch error that was missed [#165745574] * Implement Unit Tests (prebid#7) * Proper error handling [#165745574] * Address review (baby clean up) + catch error that was missed [#165745574] * Implement unit tests for utils [#165891351] * Add UT for utils + butler [#165891351] Co-authored-by: Michael Duran <mduran@sharethrough.com> * Attempt for testing Bidder interface function implementations [#165891351] Co-authored-by: Michael Duran <mduran@sharethrough.com> * Finalizing Unit tests [#165891351] Co-authored-by: Chris Nguyen <cnguyen@sharethrough.com> Co-authored-by: Josh Becker <jbecker@sharethrough.com> * Fixing sharethrough.yaml capabilities [#165891351] Co-authored-by: Josh Becker <jbecker@sharethrough.com> * Send supplyId to imp req instead of hbSource (prebid#5) [#165477915] * Finalize PR (prebid#8) [#164911891] Co-authored-by: Josh Becker <jbecker@sharethrough.com> * Remove test setting * Add Sharethrough in syncer_test * Update deserializing of third party partners * Refactor/optimize UserAgent parsing (prebid#9) following josephveach's review in prebid#903 * Addressing June 3rd review from prebid#903 Optimizations, clean up suggested by @mansinahar * Addressing June 4th review from prebid#903 (prebid#10) * Addressing June 4th review from prebid#903 Clean up canAutoPlayVideo + hardcode bhVersion to unknown for now... * Removing hbVersion butler param since it's not accessible * Fix adMarkup error handling * [Mgid] Add new Mgid Adapter (prebid#907) * new mgid adapter * increase coverage * remove extra imports * Cache validation fix (prebid#911) * Cache validation fix if no bids returned - don't throw cache error, just return empty result * Cache validation fix - optimization: do not run auction logic if no bids returned * Cache validation fix: minor refactoring * Cache validation fix: minor refactoring * Cache validation fix: added unit test for no bids returned * Cache validation fix: minor refactoring * Remove hard coded targeting keys (prebid#923)
spotx · Jun 14, 2019 · aaff53c · aaff53c
1 parent 993f2f7
commit aaff53c
Show file tree

Hide file tree

Showing 102 changed files with 4,259 additions and 216 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -1,6 +1,7 @@
 language: go
 
 go:
+  - '1.9'
   - '1.10'
   - '1.11.1'
 

diff --git a/Dockerfile b/Dockerfile
@@ -1,20 +1,24 @@
-FROM alpine:3.8 AS build
+FROM ubuntu:18.04 AS build
 WORKDIR /go/src/github.com/prebid/prebid-server/
-RUN apk add -U --no-cache go git dep musl-dev
+RUN apt-get update && \
+    apt-get -y upgrade && \
+    apt-get install -y git golang go-dep && \
+    apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
 ENV GOPATH /go
 ENV CGO_ENABLED 0
 COPY ./ ./
-RUN dep ensure
-RUN go build .
+RUN dep ensure && \
+    go build .
 
-
-FROM alpine:3.8 AS release
-MAINTAINER Hans Hjort <hans.hjort@xandr.com>
+FROM ubuntu:18.04 AS release
+LABEL maintainer="hans.hjort@xandr.com" 
 WORKDIR /usr/local/bin/
 COPY --from=build /go/src/github.com/prebid/prebid-server/prebid-server .
 COPY static static/
 COPY stored_requests/data stored_requests/data
-RUN apk add -U --no-cache ca-certificates mtr
+RUN apt-get update && \
+    apt-get install -y ca-certificates mtr && \
+    apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
 EXPOSE 8000
 EXPOSE 8080
 ENTRYPOINT ["/usr/local/bin/prebid-server"]

diff --git a/adapters/beachfront/beachfront.go b/adapters/beachfront/beachfront.go
@@ -3,11 +3,13 @@ package beachfront
 import (
 	"encoding/json"
 	"errors"
+	"fmt"
+	"github.com/prebid/prebid-server/errortypes"
 	"net/http"
+	"strconv"
 	"strings"
 
 	"github.com/prebid/prebid-server/adapters"
-	"github.com/prebid/prebid-server/errortypes"
 	"github.com/prebid/prebid-server/openrtb_ext"
 
 	"github.com/mxmCherry/openrtb"
@@ -22,7 +24,7 @@ const VideoEndpoint = "https://reachms.bfmio.com/bid.json?exchange_id="
 const VideoEndpointSuffix = "&prebidserver"
 
 const beachfrontAdapterName = "BF_PREBID_S2S"
-const beachfrontAdapterVersion = "0.2.2"
+const beachfrontAdapterVersion = "0.3.0"
 
 type BeachfrontAdapter struct {
 }
@@ -140,15 +142,14 @@ func (a *BeachfrontAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapte
 	var beachfrontRequests BeachfrontRequests
 	var reqJSON []byte
 	var uri string
-	var errs = make([]error, 0)
+	var errs = make([]error, 0, len(request.Imp))
 	var err error
 	var imps int
 
 	uri = getEndpoint(request)
 
 	beachfrontRequests, errs, imps = preprocess(request, uri)
 
-	// These are fatal errors -------------
 	if uri == VideoEndpoint {
 		reqJSON, err = json.Marshal(beachfrontRequests.Video)
 		uri = uri + beachfrontRequests.Video.AppId + VideoEndpointSuffix
@@ -175,6 +176,15 @@ func (a *BeachfrontAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapte
 	headers.Add("Content-Type", "application/json;charset=utf-8")
 	headers.Add("Accept", "application/json")
 
+	if request.Device != nil {
+		addHeaderIfNonEmpty(headers, "User-Agent", request.Device.UA)
+		addHeaderIfNonEmpty(headers, "X-Forwarded-For", request.Device.IP)
+		addHeaderIfNonEmpty(headers, "Accept-Language", request.Device.Language)
+		if request.Device.DNT != nil {
+			addHeaderIfNonEmpty(headers, "DNT", strconv.Itoa(int(*request.Device.DNT)))
+		}
+	}
+
 	return []*adapters.RequestData{{
 		Method:  "POST",
 		Uri:     uri,
@@ -288,9 +298,6 @@ func getBannerRequest(req *openrtb.BidRequest) (BeachfrontBannerRequest, []error
 	return beachfrontReq, errs, imps
 }
 
-/*
-Prepare the request that has been received from Prebid.js, translating it to the beachfront format
-*/
 func getVideoRequest(req *openrtb.BidRequest) (BeachfrontVideoRequest, []error, int) {
 	var videoImpsIndex = 0
 	var beachfrontReq = NewBeachfrontVideoRequest()
@@ -324,7 +331,7 @@ func getVideoRequest(req *openrtb.BidRequest) (BeachfrontVideoRequest, []error,
 		The req could contain banner,audio,native and video imps when It arrives here. I am only
 		interested in video
 
-		The beach front video endpoint is only capable of returning a single nurl and price, wrapped in
+		The beachfront video endpoint is only capable of returning a single nurl and price, wrapped in
 		an openrtb format, so even though I'm building a request here that will include multiple video
 		impressions, only a single URL will be returned. Hopefully the beachfront endpoint can be modified
 		in the future to return multiple video ads
@@ -394,12 +401,33 @@ func getVideoRequest(req *openrtb.BidRequest) (BeachfrontVideoRequest, []error,
 func (a *BeachfrontAdapter) MakeBids(internalRequest *openrtb.BidRequest, externalRequest *adapters.RequestData, response *adapters.ResponseData) (*adapters.BidderResponse, []error) {
 	var bids []openrtb.Bid
 	var bidtype = getBidType(internalRequest)
+
+	/*
+		Beachfront is now sending an empty array and 200 as their "no results" response. This should catch that.
+	*/
+
+	if response.StatusCode == http.StatusOK && len(response.Body) <= 2 {
+		return nil, nil
+	}
+
+	if response.StatusCode == http.StatusNoContent {
+		return nil, nil
+	}
+
+	if response.StatusCode == http.StatusBadRequest {
+		return nil, []error{&errortypes.BadInput{
+			Message: fmt.Sprintf("Unexpected status code: %d. Run with request.debug = 1 for more info", response.StatusCode),
+		}}
+	}
+
+	if response.StatusCode != http.StatusOK {
+		return nil, []error{fmt.Errorf("Unexpected status code: %d. Run with request.debug = 1 for more info", response.StatusCode)}
+	}
+
 	bids, errs := postprocess(response, externalRequest, internalRequest.ID, bidtype)
 
 	if len(errs) != 0 {
-		return nil, append(errs, &errortypes.BadServerResponse{
-			Message: "Failed to process the beachfront response",
-		})
+		return nil, errs
 	}
 
 	bidResponse := adapters.NewBidderResponseWithBidsCapacity(BidCapacity)
@@ -499,6 +527,13 @@ func extractVideoCrid(nurl string) string {
 	return strings.TrimSuffix(chunky[2], ":")
 }
 
+// Thank you, brightroll.
+func addHeaderIfNonEmpty(headers http.Header, headerName string, headerValue string) {
+	if len(headerValue) > 0 {
+		headers.Add(headerName, headerValue)
+	}
+}
+
 func NewBeachfrontBidder() *BeachfrontAdapter {
 	return &BeachfrontAdapter{}
 }
diff --git a/adapters/beachfront/beachfronttest/exemplary/minimal-banner.json b/adapters/beachfront/beachfronttest/exemplary/minimal-banner.json
@@ -56,7 +56,7 @@
           "dnt": 0,
           "ua": "",
           "adapterName": "BF_PREBID_S2S",
-          "adapterVersion": "0.2.2",
+          "adapterVersion": "0.3.0",
           "user": {
           }
         }

diff --git a/adapters/beachfront/beachfronttest/exemplary/simple-video.json b/adapters/beachfront/beachfronttest/exemplary/simple-video.json
@@ -80,6 +80,7 @@
         }
         },
       "mockResponse": {
+        "status" : 200,
         "body": {
           "id":"61b87329-8790-47b7-90dd-c53ae7ce1723",
           "seatBid":[

diff --git a/adapters/beachfront/beachfronttest/supplemental/minimal-banner-empty_array-200.json b/adapters/beachfront/beachfronttest/supplemental/minimal-banner-empty_array-200.json
@@ -0,0 +1,72 @@
+{
+  "mockBidRequest": {
+    "id": "some_test_ad",
+    "site": {
+      "page": "https://test.opposingviews.com/i/society/republican-sen-collins-may-change-vote-tax-bill?cb=1234534"
+    },
+    "imp": [
+      {
+        "bidfloor": 0.02,
+        "banner": {
+          "format": [
+            {
+              "w": 300,
+              "h": 250
+            }
+          ]
+        },
+        "ext": {
+          "bidder": {
+            "bidfloor": 0.02,
+            "appId": "00000000-1111-2222-3333-11111111111"
+          }
+        }
+      }
+    ]
+  },
+
+  "httpCalls": [
+    {
+      "expectedRequest": {
+        "uri": "https://display.bfmio.com/prebid_display",
+        "body": {
+          "slots": [
+            {
+              "slot": "",
+              "id": "00000000-1111-2222-3333-11111111111",
+              "bidfloor": 0.02,
+              "sizes": [
+                {
+                  "w": 300,
+                  "h": 250
+                }
+              ]
+            }
+          ],
+          "domain": "test.opposingviews.com",
+          "page": "https://test.opposingviews.com/i/society/republican-sen-collins-may-change-vote-tax-bill?cb=1234534",
+          "referrer": "",
+          "search": "",
+          "secure": 0,
+          "requestId": "some_test_ad",
+          "isMobile": 0,
+          "ip": "",
+          "deviceModel": "",
+          "deviceOs": "",
+          "dnt": 0,
+          "ua": "",
+          "adapterName": "BF_PREBID_S2S",
+          "adapterVersion": "0.3.0",
+          "user": {
+          }
+        }
+      },
+      "mockResponse": {
+        "status": 200,
+        "body": []
+      }
+    }
+  ],
+
+  "expectedBids": []
+}
diff --git a/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-banner.json b/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-banner.json
@@ -57,7 +57,7 @@
           "dnt": 0,
           "ua": "",
           "adapterName": "BF_PREBID_S2S",
-          "adapterVersion": "0.2.2",
+          "adapterVersion": "0.3.0",
           "user": {
           }
         }

diff --git a/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-video.json b/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-video.json
@@ -63,6 +63,7 @@
         }
       },
       "mockResponse": {
+        "status": 200,
         "body": {
           "id":"61b87329-8790-47b7-90dd-c53ae7ce1723",
           "seatBid":[

diff --git a/adapters/beachfront/beachfronttest/supplemental/minimal-site-banner.json b/adapters/beachfront/beachfronttest/supplemental/minimal-site-banner.json
@@ -56,7 +56,7 @@
             "dnt": 0,
             "ua": "",
             "adapterName": "BF_PREBID_S2S",
-            "adapterVersion": "0.2.2",
+            "adapterVersion": "0.3.0",
             "user": {
             }
           }

diff --git a/adapters/beachfront/beachfronttest/supplemental/multi-banner.json b/adapters/beachfront/beachfronttest/supplemental/multi-banner.json
@@ -96,7 +96,7 @@
           "dnt": 1,
           "ua": "Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16",
           "adapterName": "BF_PREBID_S2S",
-          "adapterVersion": "0.2.2",
+          "adapterVersion": "0.3.0",
           "user": {
             "buyeruid": "some-buyer",
             "id": "some-user"

diff --git a/adapters/beachfront/beachfronttest/supplemental/multi-mix.json b/adapters/beachfront/beachfronttest/supplemental/multi-mix.json
@@ -154,6 +154,7 @@
         }
         },
       "mockResponse": {
+        "status": 200,
         "body": {
           "id":"61b87329-8790-47b7-90dd-c53ae7ce1723",
           "seatBid":[

diff --git a/adapters/beachfront/beachfronttest/supplemental/multi-video.json b/adapters/beachfront/beachfronttest/supplemental/multi-video.json
@@ -106,6 +106,7 @@
         }
         },
       "mockResponse": {
+        "status": 200,
         "body": {
           "id":"61b87329-8790-47b7-90dd-c53ae7ce1723",
           "seatBid":[

diff --git a/adapters/improvedigital/improvedigital.go b/adapters/improvedigital/improvedigital.go
@@ -8,6 +8,7 @@ import (
 	"github.com/mxmCherry/openrtb"
 	"github.com/prebid/prebid-server/adapters"
 	"github.com/prebid/prebid-server/errortypes"
+	"github.com/prebid/prebid-server/openrtb_ext"
 )
 
 type ImprovedigitalAdapter struct {
@@ -58,15 +59,36 @@ func (a *ImprovedigitalAdapter) MakeBids(internalRequest *openrtb.BidRequest, ex
 		return nil, []error{err}
 	}
 
-	bidResponse := adapters.NewBidderResponseWithBidsCapacity(1)
+	if len(bidResp.SeatBid) == 0 {
+		return nil, nil
+	}
+
+	if len(bidResp.SeatBid) > 1 {
+		return nil, []error{&errortypes.BadServerResponse{
+			Message: fmt.Sprintf("Unexpected SeatBid! Must be only one but have: %d", len(bidResp.SeatBid)),
+		}}
+	}
+
+	seatBid := bidResp.SeatBid[0]
+
+	if len(seatBid.Bid) == 0 {
+		return nil, nil
+	}
+
+	bidResponse := adapters.NewBidderResponseWithBidsCapacity(len(seatBid.Bid))
 
-	for _, sb := range bidResp.SeatBid {
-		for i := range sb.Bid {
-			bidResponse.Bids = append(bidResponse.Bids, &adapters.TypedBid{
-				Bid:     &sb.Bid[i],
-				BidType: "banner",
-			})
+	for i := range seatBid.Bid {
+		bid := seatBid.Bid[i]
+
+		bidType, err := getMediaTypeForImp(bid.ImpID, internalRequest.Imp)
+		if err != nil {
+			return nil, []error{err}
 		}
+
+		bidResponse.Bids = append(bidResponse.Bids, &adapters.TypedBid{
+			Bid:     &bid,
+			BidType: bidType,
+		})
 	}
 	return bidResponse, nil
 
@@ -78,3 +100,26 @@ func NewImprovedigitalBidder(endpoint string) *ImprovedigitalAdapter {
 		endpoint: endpoint,
 	}
 }
+
+func getMediaTypeForImp(impID string, imps []openrtb.Imp) (openrtb_ext.BidType, error) {
+	for _, imp := range imps {
+		if imp.ID == impID {
+			if imp.Banner != nil {
+				return openrtb_ext.BidTypeBanner, nil
+			}
+
+			if imp.Video != nil {
+				return openrtb_ext.BidTypeVideo, nil
+			}
+
+			return "", &errortypes.BadServerResponse{
+				Message: fmt.Sprintf("Unknown impression type for ID: \"%s\"", impID),
+			}
+		}
+	}
+
+	// This shouldnt happen. Lets handle it just incase by returning an error.
+	return "", &errortypes.BadServerResponse{
+		Message: fmt.Sprintf("Failed to find impression for ID: \"%s\"", impID),
+	}
+}