Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move cvdmirror #1383

Merged
merged 30 commits into from
Sep 24, 2024
Merged

Move cvdmirror #1383

merged 30 commits into from
Sep 24, 2024

Conversation

skifahrer
Copy link
Contributor

@skifahrer skifahrer commented Sep 11, 2024

This PR introduces the final missing piece to the scanning infrastructure—a mirror of CVD files. The mirror is updated regularly to maintain the latest CVD (ClamAV Virus Database) and distribute it to clamcv instances.

Why Do We Need It?

The CVD distributor limits the number of connections per IP to twice every 24 hours. If you exceed this limit, you get blocked and are unable to run ClamAV.

Key Changes:

  1. CVD Mirror Integration:

    • A mirror for CVD files is being added to ensure the newest virus database is available for distribution to clamav instances.
    • The mirror is updated regularly to maintain up-to-date virus signatures.
  2. Updated Pipeline Precedence:
    The scanning pipeline precedence is now as follows:

    1. cvdmirror: Deployed first and holds the latest CVD files.
    2. clamav: Downloads the CVD database from cvdmirror.
    3. nest-clamav-scanner: Relies on clamav for scanning, requiring clamav to be running.
  3. Exclusion of forms-shared:

    • forms-shared no longer needs to be built during scanner infrastructure deployment since it’s not required.
    • This change reduces build time and speeds up the deployment process.

@skifahrer skifahrer marked this pull request as ready for review September 12, 2024 07:50
@mpinter mpinter requested review from vidriduch and removed request for mpinter September 16, 2024 08:11
@mpinter
Copy link
Contributor

mpinter commented Sep 16, 2024

forwarding the review to @vidriduch

@@ -21,7 +21,6 @@ jobs:

conditions:
name: Check for cluster conditions
needs: build-forms-shared
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now forms shared are not build on every run, but only on those instances where are needed

- id: prod-clamav
name: Check for prod clamav
if: startsWith(github.ref, 'refs/tags/prod-clamav') || steps.prod.outputs.condition == 'true'
if: startsWith(github.ref, 'refs/tags/prod-clamav') || steps.staging.outputs.condition == 'true'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why has prod-clamav staging condition?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it shouldn't be there. It's an old typo that I fixed in ClamAV branch afterwards, but this branch was created before the fix, so the typo remains. It`s now fixed.


# Set permissions
RUN chmod +x ./health.sh
RUN chmod +x ./entrypoint.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing this can be executed in just one RUN instruction

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

improved

@github-actions github-actions bot added the pr: needs work 🛠️ Changes requested before another review label Sep 18, 2024
@skifahrer skifahrer removed the pr: needs work 🛠️ Changes requested before another review label Sep 19, 2024
else
echo "ping failed"
exit 1
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(optional) Missing end lines in bunch of files

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

name: dev cvdmirror
needs: conditions
if: needs.conditions.outputs.dev-cvdmirror == 'true'
uses: bratislava/github-actions/.github/workflows/deploy-with-bratiska-cli-inhouse.yml@beta
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not totally sure if this should be @beta couse of DEV or it needs to be changed before release to @stable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually leave dev deployments with the beta tag, which I use for testing future updates on pipelines. It’s good to keep it distinct from stable.

@github-actions github-actions bot added pr: fix & ship 🚢 No additional review needed before merge - some work may be required, if specified in last review and removed pr: needs review 🙏 labels Sep 23, 2024
@skifahrer skifahrer merged commit 85d0f2e into master Sep 24, 2024
3 checks passed
@skifahrer skifahrer deleted the move-cvdmirror branch September 24, 2024 13:28
Copy link

Test build pipeline info 🚀

Changes in the code and tag info:

➡️ Changes in forms-shared: false

➡️ Changes in next: false

➡️ Changes in strapi: false

➡️ Changes in nest-forms-backend: false

➡️ Changes in nest-clamav-scanner: false

We are going to build 🚢

Copy link

❗ We have zero builds based on conditions. Exiting pipeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr: fix & ship 🚢 No additional review needed before merge - some work may be required, if specified in last review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants