add scripts to build and package starter projects for offline registries

[mike-hoang]

Please specify the area for this PR

/area registry

What does does this PR do / why we need it:

Updates dl_starter_projects.sh

• Adds the option to download all starter projects when no args are passed in

Adds update_devfiles_offline.sh

• Overwrites the devfiles in offline registry builds to point to downloaded starter projects

Which issue(s) this PR fixes:

Fixes devfile/api#1203

PR acceptance criteria:

• Test Coverage
  • Are your changes sufficiently tested, and are any applicable test cases added or updated to cover your changes?

Documentation (WIP)

• Does the REST API doc need to be updated with your changes?
• Does the registry library doc need to be updated with your changes?

How to test changes / Special notes to the reviewer:
To test locally:

1. Update the STACK_DIR to STACKS_DIR=tests/registry/stacks
2. Run the scripts; bash /registry-support/build-tools/dl_starter_projects.sh, bash /registry-support/build-tools/update_devfiles_offline.sh

To test with the registry build script:

• Run the build script from: add dockerfile to support offline registry builds registry#197
• bash build.sh offline

Verify that resources are inside the stack:

• docker run --rm -it --entrypoint bash devfile-index
• cd /registry/stacks/

[openshift-ci]

@mike-hoang: The label(s) area/registry cannot be applied, because the repository doesn't have them.

In response to this:

Please specify the area for this PR

/area registry

What does does this PR do / why we need it:

Updates dl_starter_projects.sh

• Adds the option to download all starter projects when no args are passed in

Adds update_devfiles_offline.sh

• Overwrites the devfiles in offline registry builds to point to downloaded starter projects

Which issue(s) this PR fixes:

Fixes devfile/api#1203

PR acceptance criteria:

• Test Coverage
  • Are your changes sufficiently tested, and are any applicable test cases added or updated to cover your changes?

Documentation (WIP)

• Does the REST API doc need to be updated with your changes?
• Does the registry library doc need to be updated with your changes?

How to test changes / Special notes to the reviewer:
To test locally:

1. Update the STACK_DIR to STACKS_DIR=tests/registry/stacks
2. Run the scripts; bash /registry-support/build-tools/dl_starter_projects.sh, bash /registry-support/build-tools/update_devfiles_offline.sh

To test with the registry build script:
TODO: link to dependant registry PR when opened

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[codecov]

Codecov Report

Patch and project coverage have no change.

Comparison is base (605081d) 35.93% compared to head (23e5a49) 35.93%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #179   +/-   ##
=======================================
  Coverage   35.93%   35.93%           
=======================================
  Files           7        7           
  Lines        1347     1347           
=======================================
  Hits          484      484           
  Misses        816      816           
  Partials       47       47           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[thepetk]

Nice work! Especially the update_devfiles_offline.sh 💯 I've left some minor comments. I'll try to build and test it more thoroughly!

[thepetk]

I was able to build the registry image locally and test it with minikube. More detailed I can see that for the offline build we are adding:

starterProjects:
    - name: go-starter
      zip:
        location: go-starter-offline.zip

Also I've checked inside the image and we are including the necessary zip file with the starterproject.

As we have discussed offline, a next step should be to update https://devfile.io/docs/2.2.0/installation-of-in-cluster-offline-devfile-registry

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mike-hoang, thepetk
Once this PR has been reviewed and has the lgtm label, please assign elsony for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment