WIP: fix(gcp_stackdriver_metrics): fixes invalid format for gcp metrics (#14890)

[ansel1]

Needs real-world test...there is no integration test for this. I'm hoping the PR will trigger a CI build and publish a docker image I can test with.

[bits-bot]

All committers have signed the CLA.

[netlify]

✅ Deploy Preview for vrl-playground canceled.

Name	Link
🔨 Latest commit	f196eb5
🔍 Latest deploy log	https://app.netlify.com/sites/vrl-playground/deploys/63ceb71d0602050009819616

[netlify]

✅ Deploy Preview for vector-project ready!

Name	Link
🔨 Latest commit	f196eb5
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/63ceb71d5352a500090ee1a0
😎 Deploy Preview	https://deploy-preview-16079--vector-project.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[jszwedko]

Thanks for this @ansel1 !

Needs real-world test...there is no integration test for this. I'm hoping the PR will trigger a CI build and publish a docker image I can test with.

Unfortunately CI won't publish docker images. I think you'll need to build and run locally to test this change. It's also unfortunate that there are no integration tests for this sink just yet.

[ansel1]

I've been trying to build it locally all weekend, no luck. I have an Apple silicon mac, and an Intel mac. On the Apple silicon, I get errors with make environment about x86 stdlibs missings, I assume because it's basing the docker build image on the wrong platform.

On Intel mac, I got further, but I can't figure out the right set of make commands to build the docker image. The make release-docker complains about missing the targets/artifacts folder, and the 'package' targets (which look like they populate the artifacts folder) fail with

cross build \
	--release \
	--target x86_64-unknown-linux-musl \
	--no-default-features \
	--features target-x86_64-unknown-linux-musl
info: downloading component 'rust-src'
info: installing component 'rust-src'
Error:
   0: `docker inspect ''` failed with exit status: 1

Stderr:
   Error: No such object:

Stdout:
   []
make: *** [Makefile:255: target/x86_64-unknown-linux-musl/release/vector] Error 1

[ansel1]

Also tried doing a simple make build, then running targets/release/vector, with the following config:

[sources.vector_metrics_source]
type = "internal_metrics"

[sinks.gcp]
type = "gcp_stackdriver_metrics"
inputs = [ "vector_metrics_source" ]
project_id = "myproject"

[sinks.gcp.resource]
type = "myresource"
projectId = "myproject"
zone = "use-central1-a"

Vector fails with this error:

2023-01-23T18:38:17.504023Z ERROR vector::cli: Configuration error. error=missing field `labels` for key `sinks.gcp`

labels is not a valid field for gcp_stackdriver_metrics, and I don't get that error running the released vector. Really confused here.

[jszwedko]

Ah, gotcha, yeah building docker images locally is a bit fraught right now: it requires multiple steps and really only works on Linux machines. We hope to add a way to do one-off builds in CI in the nearish future to make this process easier.

The error you see about missing labels is due to the change you introduced here. Since they are no longer flattened the deserialization requires a labels field to be provided.

[ansel1]

Yeah, I figured that out. I was finally able to figure out how to build a binary and test it locally, and confirmed that unflattening the labels makes it work with gcp. But as you said, it changes the format of the configuration.

I opened a new PR ( #16089 ). Asked there whether the preference is to change the config schema, or make more code changes to flatten labels when reading config, but not flatten when creating request bodies.