ToTCString() runtime error: index out of range [708] with length 708

[alexandre-lim]

Hello, thank you for the library! I'm having an error when calling the function ToTCString(). The error is

panic: runtime error: index out of range [708] with length 708

goroutine 1 [running]:
github.com/SirDataFR/iabtcfv2.(*Bits).writeNumber(...)
        /go/pkg/mod/github.com/!sir!data!f!r/iabtcfv2@v1.1.3/bits.go:58
github.com/SirDataFR/iabtcfv2.(*Bits).writeInt(...)
        /go/pkg/mod/github.com/!sir!data!f!r/iabtcfv2@v1.1.3/bits.go:49
github.com/SirDataFR/iabtcfv2.(*CoreString).Encode(0xc0000bfdf8)
        /go/pkg/mod/github.com/!sir!data!f!r/iabtcfv2@v1.1.3/segment_core_string.go:358 +0x1066
github.com/SirDataFR/iabtcfv2.(*TCData).ToTCString(0xc0000bfca8)
        /go/pkg/mod/github.com/!sir!data!f!r/iabtcfv2@v1.1.3/tcdata.go:73 +0x3a
main.main()
        /workspace/main.go:115 +0x7b0
exit status 2

Here are some screenshots where the issue happen when going through debugging:

iabtcfv2@v1.1.3/segment_core_string.go

abtcfv2@v1.1.3/bits.go

b.bytes length is 707 but byteIndex value is 708

I made a codesandbox which reproduce this error: https://codesandbox.io/p/sandbox/stupefied-rumple-2mjqqk
The vendor variable comes from https://vendor-list.consensu.org/v2/vendor-list.json. I had an old version of the vendor-list.json which worked fine and this error appeared after trying to use the latest version

[RemiSirdata]

Hi @alexandre-lim ,

Thank you for reporting this well described issue, we're look into as soon as possible.

Rémi

[RemiSirdata]

Hi @alexandre-lim,

I wanted to be sure the vendor list is ok, I changed your script to download the vendor list instead using local data, no more errors with valid consent string.
I'll let @tla-sirdata look deeply when he can.

Regards,
Rémi

CPx73UAPx73UAFzABBFRDWCsAP_AAH_AAAqIgoNf_X__b3_v-_7___t0eY1f9_7__-0zjhfdt-8N3f_X_L8X_2M7vF36tr4KuR4ku3bBIUdtHPncTVmx6olVrzPsbk2cr7NKJ_Pkmnsbe2dYGH9_n9_z_ZKZ7___f__7_______________________________________________________________________-________4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQTUAJMNC4gC7AkZCbaMIoEAIwrCQqgUAFEAkLRAYQurgp2VwE-sBEAIEUABwQAhgBRkACAAACAJCIAJAjgQCAQCAQAAgAVCAQAEbAAKACwEAgAFAdCxTigCUCwgyISIhTAhCkSCgnsqEEoP1BXCEMssAKDR_xUICBQAxWBEJCxehwBICXCSQLdUb4ACEAKAUUoViCT0wADgkbLUHggAA

package main

import (
	"encoding/json"
	"github.com/SirDataFR/iabtcfv2"
	"log"
	"net/http"
	"time"
)

type VendorList struct {
	VendorListVersion     int               `json:"vendorListVersion"`
	Vendors               map[string]Vendor `json:"vendors"`
	VendorsConsent        map[int]bool
	VendorsLITransparency map[int]bool
	MaxVendorId           int
	MaxVendorIdLI         int
}

type Vendor struct {
	Id             int    `json:"id"`
	Name           string `json:"name"`
	PolicyUrl      string `json:"policyUrl"`
	LegIntPurposes []int  `json:"legIntPurposes"`
}

func GetNowWithoutTime() (date time.Time) {
	now := time.Now()
	y, m, d := now.Date()
	date = time.Date(y, m, d, 0, 0, 0, 0, time.UTC)
	return
}

func main() {
	resp, err := http.Get("https://vendor-list.consensu.org/v2/vendor-list.json")
	if err != nil {
		log.Fatalf("fail to get vendor list: %s", err.Error())
	}
	if resp.StatusCode != http.StatusOK {
		log.Fatalf("fail to get vendor list: invalid status code %d", resp.StatusCode)
	}
	var list VendorList
	if err := json.NewDecoder(resp.Body).Decode(&list); err != nil {
		log.Fatalf("fail to decode vendor list: %s", err.Error())
	}

	maxVendorId := 0
	maxVendorIdLI := 0
	vendorConsent := make(map[int]bool)
	vendorsLITransparency := make(map[int]bool)
	for _, vendor := range list.Vendors {
		vendorConsent[vendor.Id] = true
		if vendor.Id > maxVendorId {
			maxVendorId = vendor.Id
		}

		if len(vendor.LegIntPurposes) > 0 {
			vendorsLITransparency[vendor.Id] = true
			if vendor.Id > maxVendorIdLI {
				maxVendorIdLI = vendor.Id
			}
		}
	}
	list.MaxVendorId = maxVendorId
	list.MaxVendorIdLI = maxVendorIdLI
	list.VendorsConsent = vendorConsent
	list.VendorsLITransparency = vendorsLITransparency

	tcDataAcceptAll := &iabtcfv2.TCData{
		CoreString: &iabtcfv2.CoreString{
			Version:              2,
			Created:              GetNowWithoutTime(),
			LastUpdated:          GetNowWithoutTime(),
			CmpId:                371,
			CmpVersion:           1,
			ConsentScreen:        1,
			ConsentLanguage:      "FR",
			VendorListVersion:    list.VendorListVersion,
			TcfPolicyVersion:     2,
			IsServiceSpecific:    true,
			UseNonStandardStacks: false,
			SpecialFeatureOptIns: map[int]bool{
				1: true,
				2: true,
			},
			PurposesConsent: map[int]bool{
				1:  true,
				2:  true,
				3:  true,
				4:  true,
				5:  true,
				6:  true,
				7:  true,
				8:  true,
				9:  true,
				10: true,
			},
			PurposesLITransparency: map[int]bool{
				2:  true,
				3:  true,
				4:  true,
				5:  true,
				6:  true,
				7:  true,
				8:  true,
				9:  true,
				10: true,
			},
			PurposeOneTreatment:   false,
			PublisherCC:           "FR",
			MaxVendorId:           list.MaxVendorId,
			MaxVendorIdLI:         list.MaxVendorIdLI,
			VendorsConsent:        list.VendorsConsent,
			VendorsLITransparency: list.VendorsLITransparency,
		},
	}

	log.Printf("output: %s", tcDataAcceptAll.ToTCString())
}

[alexandre-lim]

Hi @RemiSirdata

Thank you for looking into the issue. The JSON from the vendor list was okay; I have a command that takes the content from https://vendor-list.consensu.org/v2/vendor-list.json and pastes it into a JSON file. But I relaunched the command, which now works with the recently updated content 🤔 It seems the earlier update created this bug.

It's probably an edge case, but it would be very helpful if you have time to take a deeper look into it 👍 Thanks!

Best Regards,
Alexandre

[tla-sirdata]

Hi @alexandre-lim

Thank you for reporting the issue.

After looking deeper, there was a problem with the bit allocation related to the range sections and bitfield sections while encoding the TC String.

It is not directly related to the GVL but depending of the vendor list (and so the maxVendorId fields), the TC String can be encoded using range sections or bitfield sections.
In your case the code fails with the GVL v213 but works with the GVL v214 as the maxVendorIdLI has changed between the versions.

The issue has been fixed and a release has been published here : https://github.com/SirDataFR/iabtcfv2/releases/tag/v1.2.0

Regards,
Thomas

[alexandre-lim]

Hi @tla-sirdata

Thank you for the explanation and the fast fix! 🙏