GCP timeout with querypack

[rpujet]

packs:

• uid: inventory-gcp
  name: GCP ConMon Inventory
  tags:
  website.com/platform: gcp,cloud
  website.com/category: inventory
  docs:
  desc: |
  This pack is meant to gather information for our GCP platform that c an be used for ConMon
  filters:
  • asset.platform == "gcp"
    querys:
  • uid: gcp-compute-instances
    title: Gather GCP Computer Instances
    query: gcp.project.computeService.instances

I'm getting the following error:
! could not scan asset error="execution timed out after 10s" asset="GCP project project-name
when running the above querypack, but when I try to run the query directly with the run -c options it works correctly. I also have something similar for AWS, but AWS works, so not sure what's going with this GCP querypack. Any ideas?

[chris-rock]

Thank you @rpujet for asking that question. Do you see any report output of queries?

[rpujet]

@chris-rock This is the entire output I see:

→ no Mondoo configuration file provided. using defaults
! Scanning with local bundles will switch into --incognito mode by default. Your results will not be sent upstream.
→ discover related assets for 1 asset(s)
→ resolved assets resolved-assets=1
→ connecting to asset GCP project development-0 (api)

░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0% GCP project development-0
! could not scan asset error="execution timed out after 10s" asset="GCP project development-0"

[rpujet]

I was trying different queries and now I'm getting the following:

! could not scan asset error="rpc error: code = InvalidArgument desc = asset does not match any of the activated query packs" asset="GCP project development-0"

this is also the result of the original query that was having timeouts. Everything still works as intended if I run the query manually through the CLI.

[imilchev]

Could you run the same command with --log-level debug? That might give us some more information on what is happening. Another thing would be if you could provide us with the amount of instances you have in your project

[rpujet]

when I added --log-level debug I noticed references to a project that wasn't my default and removed the related file in my gcloud's config directory. That resulted in manual scans not working either. The below output says I don't have access/permissions to the project, but I have confirmed I do via the gcloud CLI tool

→ no Mondoo configuration file provided. using defaults
DBG parsing asset asset-type=1 provider=GCP
! Scanning with local bundles will switch into --incognito mode by default. Your results will not be sent upstream.
→ discover related assets for 1 asset(s)
DBG run resolver resolver="GCP Resolver" resolver-id=gcp
DBG local> run command gcloud config config-helper --format json
DBG fallback to default google sdk authentication
→ resolved assets resolved-assets=0
x could not resolve asset error="could not find or have no access to project development-0" asset=
FTL failed to run scan error="failed to resolve multiple assets"

[imilchev]

It was the case that the actual error remained hidden in exactly the situation in which you are. As of the latest release (7.15.0 got released about 3h ago) that is fixed. If you update cnquery/cnspec to the latest version and re-run the command, you should be able to see the actual error behind the failure

[rpujet]

Is there any way to increase the timeout? I was trying to find where it is set but haven't had any luck. I just want to rule this out. I compared the below output to the output from the successful run (opposed to scan) and there weren't many differences.

→ no Mondoo configuration file provided. using defaults
DBG parsing asset asset-type=1 provider=GCP
! Scanning with local bundles will switch into --incognito mode by default. Your results will not be sent upstream.
→ discover related assets for 1 asset(s)
DBG run resolver resolver="GCP Resolver" resolver-id=gcp
DBG local> run command gcloud config config-helper --format json
DBG establish motor connection
DBG connection> load gcp provider
DBG fallback to default google sdk authentication
DBG fallback to default google sdk authentication
→ resolved assets resolved-assets=1
DBG connecting to asset GCP project development-0 (api)
DBG establish connection to asset connection=gcp insecure=false
DBG establish motor connection
DBG connection> load gcp provider
DBG fallback to default google sdk authentication
DBG established connection
DBG store query pack mrn=//local.cnquery.io/run/local-execution/querypack/inventory-gcp owner= req-id=global uid=
DBG assets> ensure asset mrn=//explorer.api.mondoo.com/assets/2LCKNRO6lAn55UnHcqDxrklS1Nb
DBG client> request bundle for asset asset=//explorer.api.mondoo.com/assets/2LCKNRO6lAn55UnHcqDxrklS1Nb
DBG client> got bundle
DBG client> got filters asset=//explorer.api.mondoo.com/assets/2LCKNRO6lAn55UnHcqDxrklS1Nb
DBG client> shell update filters asset=//explorer.api.mondoo.com/assets/2LCKNRO6lAn55UnHcqDxrklS1Nb
[
  {
    "code_id": "d03WaM+G94M=",
    "mrn": "//local.cnquery.io/run/local-execution/querypack/inventory-gcp/assetfilter/d03WaM+G94M=",
    "query": "asset.platform == \"gcp\""
  }
]
DBG client> got resolved bundle for asset asset=//explorer.api.mondoo.com/assets/2LCKNRO6lAn55UnHcqDxrklS1Nb
DBG scan complete asset=//explorer.api.mondoo.com/assets/2LCKNRO6lAn55UnHcqDxrklS1Nb
DBG could not scan asset error="execution timed out after 10s" asset="GCP project ujet-development-0"
Summary (1 assets)
==================

Target:     GCP project ujet-development-0
✕ Error for asset GCP project ujet-development-0: execution timed out after 10s

[imilchev]

Do you know what query is causing this? Normally things shouldn't be as slow. I can take a look into the code if I know what you are checking for

[rpujet]

just: gcp.projects.computeService.instances

[imilchev]

I tried to replicate the problem with the pack that you have provided above. It's a bit difficult since Github messed up with the YAML formatting. I noticed you had several typos and issues with the general structure of your pack. I adjusted it:

packs:
  - uid: inventory-gcp
    name: GCP ConMon Inventory
    tags:
      website.com/platform: gcp,cloud
      website.com/category: inventory
    docs:
      desc: |
        This pack is meant to gather information for our GCP platform that c an be used for ConMon
    filters:
      asset.platform == "gcp"
    queries:
      - uid: gcp-compute-instances
        title: Gather GCP Computer Instances
        query: gcp.project.computeService.instances # <-- this is equivalent to gcp.project.compute.instances so you can use that as well

When I run this in my GCP project it seems to return the correct results without a timeout. Please have a try with that example pack and let us know if you still see any timeouts.

P.S.: I am very interested in the problem you are encountering since I cannot seem to reproduce it. If it keeps happening perhaps it's easier to hop on Slack/Zoom and sort it out there

[rpujet]

This worked, and explains why it worked for AWS and not the GCP portion. I spelled it correctly in one spot but not the other.
Thank you!