Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bigquery #687

Merged
merged 4 commits into from
Apr 6, 2023
Merged

Bigquery #687

merged 4 commits into from
Apr 6, 2023

Conversation

AvineshTripathi
Copy link
Collaborator

@AvineshTripathi AvineshTripathi commented Apr 1, 2023

fixes #665

Changes Made

Added BigQuery table support

Note: the code is not been tested with the real time tables in GCP however according to the documentation/go sdk for gcp it should work

Checklist

  • Code follows the contributing guidelines
  • Changes have been thoroughly tested
  • Documentation has been updated, if necessary
  • Any dependencies have been added to the project, if necessary

@eneskaya eneskaya changed the base branch from master to develop April 3, 2023 08:24
@mlabouardy mlabouardy requested a review from eneskaya April 3, 2023 11:48
@mlabouardy mlabouardy added this to the v3.0.10 milestone Apr 3, 2023
@mlabouardy mlabouardy added the gcp label Apr 3, 2023
Name: tableMetadata.Name,
FetchedAt: time.Now(),
Tags: tags,
Link: fmt.Sprintf("https://console.cloud.google.com/compute/instancesDetail/zones/%s/instances/%s?project=%s", tableMetadata.Location, tableMetadata.Name, client.GCPClient.Credentials.ProjectID),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This URL looks not right for a BQ Table, for example this would be the URL to a table:

https://console.cloud.google.com/bigquery?referrer=search&authuser=1&project=[PROJECTID]&ws=!1m5!1m4!4m3!1s[WORSKPACE]!2s[DATASET]!3s[TABLENAME]

So you'd need to fill in [PROJECTID], [WORKSPACE], [DATASET] and [TABLENAME] from the data you grab

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I cannot set up the BQ table I am not able to sure how to get the workspace field can you help me with that please

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm ok, I would have to check how to do this myself. I'll get back at you

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, the workspace isn't needed, you just need to provide the following:
https://console.cloud.google.com/bigquery?project={PROJECT_NAME}&page=dataset&p={PROJECT_NAME}&d={DATASET}

Copy link
Collaborator Author

@AvineshTripathi AvineshTripathi Apr 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Thanks @mlabouardy can you confirm if PROJECT_NAME is actually PROJECT_NAME field or is it PROJECT_ID and also DATASET here is the ID?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's client.GCPClient.Credentials.ProjectID for PROJECT_NAME from @mlabouardy snippet

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you review it now @eneskaya thank you!

@AvineshTripathi AvineshTripathi requested a review from eneskaya April 3, 2023 14:41
@mlabouardy mlabouardy modified the milestones: v3.0.10, v3.0.11 Apr 4, 2023
Copy link
Contributor

@eneskaya eneskaya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine, please rebase your fork with the upstream and resolve the conflicts so we can merge 👍

Signed-off-by: AvineshTripathi <avineshtripathi1@gmail.com>
Signed-off-by: AvineshTripathi <avineshtripathi1@gmail.com>
Signed-off-by: AvineshTripathi <avineshtripathi1@gmail.com>
@mlabouardy mlabouardy merged commit 8ab58b0 into tailwarden:develop Apr 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support GCP BigQuery tables
3 participants