Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: bugs around aborting scan #1583

Merged
merged 3 commits into from
Apr 19, 2024
Merged

fix: bugs around aborting scan #1583

merged 3 commits into from
Apr 19, 2024

Conversation

akijakya
Copy link
Contributor

Description

Fixed some bugs while testing, mainly around when a scan is being aborted:

  • allow to transition from any status to aborted - I think this makes sense as sometimes aborting the scan is needed something is broken and some assetScans couldn't get to the ReadyToScan phase
  • fix a race condition in the orchestrator - related log:
    WARNING: DATA RACE
    Write at 0x00c001bb8c40 by goroutine 9201:
      github.com/openclarity/vmclarity/orchestrator/watcher/scan.(*Watcher).reconcileAborted.func1()
          /build/orchestrator/watcher/scan/watcher.go:491 +0x238
    
    Previous write at 0x00c001bb8c40 by goroutine 9200:
      github.com/openclarity/vmclarity/orchestrator/watcher/scan.(*Watcher).reconcileAborted.func1()
          /build/orchestrator/watcher/scan/watcher.go:491 +0x238
    
    Goroutine 9201 (running) created at:
      github.com/openclarity/vmclarity/orchestrator/watcher/scan.(*Watcher).reconcileAborted()
          /build/orchestrator/watcher/scan/watcher.go:481 +0xbc0
      github.com/openclarity/vmclarity/orchestrator/watcher/scan.(*Watcher).Reconcile()
          /build/orchestrator/watcher/scan/watcher.go:167 +0x710
      github.com/openclarity/vmclarity/orchestrator/watcher/scan.(*Watcher).Reconcile-fm()
          <autogenerated>:1 +0x58
      github.com/openclarity/vmclarity/orchestrator/common.(*Reconciler[go.shape.struct { ScanID string }]).Start.func1()
          /build/orchestrator/common/reconciler.go:70 +0x19c
    
    Goroutine 9200 (finished) created at:
      github.com/openclarity/vmclarity/orchestrator/watcher/scan.(*Watcher).reconcileAborted()
          /build/orchestrator/watcher/scan/watcher.go:481 +0xbc0
      github.com/openclarity/vmclarity/orchestrator/watcher/scan.(*Watcher).Reconcile()
          /build/orchestrator/watcher/scan/watcher.go:167 +0x710
      github.com/openclarity/vmclarity/orchestrator/watcher/scan.(*Watcher).Reconcile-fm()
          <autogenerated>:1 +0x58
      github.com/openclarity/vmclarity/orchestrator/common.(*Reconciler[go.shape.struct { ScanID string }]).Start.func1()
          /build/orchestrator/common/reconciler.go:70 +0x19c
    ==================
  • fix a panic that also occurred in the orchestrator causing the pod to be continuously restarted on Kubernetes - related log:
    time="2024-04-17T07:44:58Z" level=info msg="Starting healthz server. listenAddr=:8082"
    time="2024-04-17T07:44:58Z" level=info msg="Starting Orchestrator server"
    time="2024-04-17T07:45:12Z" level=info msg="Reconciling item" AssetScanID=61aec253-61ef-44c2-87c6-39dfddd9ccfa controller=AssetScanProcessor
    time="2024-04-17T07:45:12Z" level=info msg="Found 0 existing vulnerabilities findings for this scan" controller=AssetScanProcessor
    time="2024-04-17T07:45:12Z" level=info msg="Found 0 existing package findings for this scan" controller=AssetScanProcessor
    time="2024-04-17T07:45:13Z" level=info msg="Found 0 existing exploit findings for this scan" controller=AssetScanProcessor
    time="2024-04-17T07:45:13Z" level=info msg="Found 0 existing secret findings for this scan" controller=AssetScanProcessor
    time="2024-04-17T07:45:13Z" level=info msg="Found 0 existing malware findings for this scan" controller=AssetScanProcessor
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x9466c0]
    
    goroutine 100 [running]:
    github.com/openclarity/vmclarity/scanner/findingkey.GenerateRootkitKey(...)
        /build/scanner/findingkey/rootkit.go:45
    github.com/openclarity/vmclarity/orchestrator/processor/assetscan.(*AssetScanProcessor).getExistingRootkitFindingsForScan(0x400199bc68, {0x823acb0, 0x40001c61c0}, {0x4002645038, 0x400180bec0, 0x4003b88a60, 0x0, 0x4003b88a70, 0x4002645080, 0x40026450e0, ...})
        /build/orchestrator/processor/assetscan/rootkits.go:48 +0x280
    github.com/openclarity/vmclarity/orchestrator/processor/assetscan.(*AssetScanProcessor).reconcileResultRootkitsToFindings(0x400199bc68, {0x823acb0, 0x40001c61c0}, {0x4002645038, 0x400180bec0, 0x4003b88a60, 0x0, 0x4003b88a70, 0x4002645080, 0x40026450e0, ...})
        /build/orchestrator/processor/assetscan/rootkits.go:73 +0x134
    github.com/openclarity/vmclarity/orchestrator/processor/assetscan.(*AssetScanProcessor).Reconcile(0x400199bc68, {0x823acb0, 0x40001c61c0}, {{0x400171c4b0?, 0x0?}})
        /build/orchestrator/processor/assetscan/processor.go:102 +0x69c
    github.com/openclarity/vmclarity/orchestrator/common.(*Reconciler[...]).Start.func1()
        /build/orchestrator/common/reconciler.go:70 +0x11c
    created by github.com/openclarity/vmclarity/orchestrator/common.(*Reconciler[...]).Start in goroutine 1
        /build/orchestrator/common/reconciler.go:56 +0x90
    Stream closed EOF for vmclarity/vmclarity-orchestrator-7bdfdb4f74-cwhtp (orchestrator)

Type of Change

[X] Bug Fix
[ ] New Feature
[ ] Breaking Change
[ ] Refactor
[ ] Documentation
[ ] Other (please describe)

Checklist

  • I have read the contributing guidelines
  • Existing issues have been referenced (where applicable)
  • I have verified this change is not present in other open pull requests
  • Functionality is documented
  • All code style checks pass
  • New code contribution is covered by automated tests
  • All new and existing tests pass

Signed-off-by: András Jáky <ajaky@cisco.com>
Signed-off-by: András Jáky <ajaky@cisco.com>
@akijakya akijakya added bug Something isn't working component:api Issues related to API component:orchestrator labels Apr 17, 2024
@akijakya akijakya requested a review from a team as a code owner April 17, 2024 15:10
@akijakya akijakya changed the title fix: small abort scan bugs fix: bugs around aborting scan Apr 17, 2024

This comment has been minimized.

paralta
paralta previously approved these changes Apr 17, 2024
Copy link
Contributor

@paralta paralta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat 🚀 nice catch on these bugs!

Signed-off-by: András Jáky <ajaky@cisco.com>
Copy link

Hey!

Your images are ready:

  • ghcr.io/openclarity/vmclarity-apiserver-dev:pr1583-333fda8e4374bb281f585d36c9047d72b71b58b3
  • ghcr.io/openclarity/vmclarity-cli-dev:pr1583-333fda8e4374bb281f585d36c9047d72b71b58b3
  • ghcr.io/openclarity/vmclarity-cr-discovery-server-dev:pr1583-333fda8e4374bb281f585d36c9047d72b71b58b3
  • ghcr.io/openclarity/vmclarity-orchestrator-dev:pr1583-333fda8e4374bb281f585d36c9047d72b71b58b3
  • ghcr.io/openclarity/vmclarity-ui-dev:pr1583-333fda8e4374bb281f585d36c9047d72b71b58b3
  • ghcr.io/openclarity/vmclarity-ui-backend-dev:pr1583-333fda8e4374bb281f585d36c9047d72b71b58b3

@akijakya akijakya added this pull request to the merge queue Apr 19, 2024
Merged via the queue into main with commit 59d045d Apr 19, 2024
34 checks passed
@akijakya akijakya deleted the abort-scan-fixes branch April 19, 2024 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working component:api Issues related to API component:orchestrator
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants