Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exporter changes for run image extension #1055

Merged
merged 77 commits into from
Apr 17, 2023
Merged

Exporter changes for run image extension #1055

merged 77 commits into from
Apr 17, 2023

Commits on Feb 14, 2023

  1. Make a single constructor for lifecycle inputs 

    - The logic to update the default path for TOML files was repeated across phases
- In general it is safe to provide default values for inputs that might not be relevant to the current phase,
  as these will be ignored when constructing a new service for the phase;
  e.g., platform.LifecycleInputs.OrderPath will be ignored when constructing a lifecycle.Exporter
- As more inputs are shared across phases (e.g., analyzed.toml is now an input to the detect phase),
  duplicating the logic for providing default values is becoming more cumbersome

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    f22c4f9 View commit details
    Browse the repository at this point in the history

  2. Read values from environment 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    433bf3c View commit details
    Browse the repository at this point in the history

  3. Buildpack API: run.Dockerfiles are allowed instructions on versions >… 

    …= 0.10

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    f40616e View commit details
    Browse the repository at this point in the history

  4. Platform API: the detector accepts a new -run flag 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    29a94a6 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2023

  1. Move responsibility for validating Dockerfiles into the buildpack pac… 

    …kage

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    569f985 View commit details
    Browse the repository at this point in the history

  2. When verifying Dockerfiles, return the new base image name if necessary 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    c72a10e View commit details
    Browse the repository at this point in the history

  3. When determining the new runtime base image, use criteria outlined in… 

    … the platform spec

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    4c5a564 View commit details
    Browse the repository at this point in the history

  4. Platform API: the schema of analyzed.toml is updated to include run-i… 

    …mage.extend = <true or false, default false>

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    01ab4b6 View commit details
    Browse the repository at this point in the history

  5. TESTME: Update analyzed.toml with new run image if needed 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    9ba40e3 View commit details
    Browse the repository at this point in the history

  6. If extensions are used to switch the runtime base image, the detector… 

    … should fail if the selected base image is not found in run.toml.

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    eea3182 View commit details
    Browse the repository at this point in the history

  7. Add fixture to test re-writing of analyzed.toml 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    647a3db View commit details
    Browse the repository at this point in the history

  8. Move updating analyzed.toml into lifecycle package for easier testing 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    9157796 View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2023

  1. Platform API: the restorer will update analyzed.toml with: 

    - digest ref for run image
- target data for run image

Additionally the restorer will download the run image manifest & config when extend is true

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 17, 2023
    Configuration menu
    Copy the full SHA
    62411e8 View commit details
    Browse the repository at this point in the history

  2. Update acceptance/extender_test.go 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano authored Feb 17, 2023
    Configuration menu
    Copy the full SHA
    362e755 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2023

  1. The extender accepts new -extended and -kind flags 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    6cd5a89 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2023

  1. Bring back selective package 

    Because we change the media types to be oci types (vs docker types) this changes the digest of the image

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 22, 2023
    Configuration menu
    Copy the full SHA
    97470cd View commit details
    Browse the repository at this point in the history

  2. Add acceptance test for run image extension 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 22, 2023
    Configuration menu
    Copy the full SHA
    9971560 View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2023

  1. WIP 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 23, 2023
    Configuration menu
    Copy the full SHA
    8013390 View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2023

  1. Refactor 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 24, 2023
    Configuration menu
    Copy the full SHA
    d546169 View commit details
    Browse the repository at this point in the history

  2. WIP: break me up into separate commits 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 24, 2023
    Configuration menu
    Copy the full SHA
    3a55e4e View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2023

  1. Validate extend config during buildpack/generate 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    c3f74aa View commit details
    Browse the repository at this point in the history

  2. Address some TODOs and refactor 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    ee09eb7 View commit details
    Browse the repository at this point in the history

  3. Set io.buildpacks.rebasable 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    50c23a7 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2023

  1. Merge branch 'main' into runext/restore-997

    @natalieparellano
    natalieparellano committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    15c4c06 View commit details
    Browse the repository at this point in the history

  2. Fix merge and restore selective package 

    imgutil/layout/sparse modifies the image media types which we don't want

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    8917ca0 View commit details
    Browse the repository at this point in the history

  3. Update analyzed.toml with digest reference or target data if needed 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    ea13f10 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2023

  1. Merge branch 'main' into runext/detect-996

    @natalieparellano
    natalieparellano committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    d0ce558 View commit details
    Browse the repository at this point in the history

  2. Fix acceptance 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    9ba92d1 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into runext/restore-997

    @natalieparellano
    natalieparellano committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    6487081 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'runext/detect-996' into runext/restore-997

    @natalieparellano
    natalieparellano committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    76c7284 View commit details
    Browse the repository at this point in the history

  5. Don't redefine -layers 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    b8f029f View commit details
    Browse the repository at this point in the history

  6. Merge branch 'runext/detect-996' into runext/restore-997

    @natalieparellano
    natalieparellano committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    2d78993 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'runext/restore-997' into runext/extend-998

    @natalieparellano
    natalieparellano committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    7a37dfb View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2023

  1. WIP: add acceptance test 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    c76b604 View commit details
    Browse the repository at this point in the history

  2. WIP: first assertion passes 

    FIXME: extended layers should be small

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    d895813 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2023

  1. WIP 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    693b575 View commit details
    Browse the repository at this point in the history

  2. WIP: Second assertion passes 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    49e41cb View commit details
    Browse the repository at this point in the history

  3. Add new assertions and rename variables for clarity; fix exporter units 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 7, 2023
    Configuration menu
    Copy the full SHA
    e1cbf60 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2023

  1. Merge branch 'main' into runext/extend-998

    @natalieparellano
    natalieparellano committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    4bf7786 View commit details
    Browse the repository at this point in the history

  2. Save extended run image under <extended>/run 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    523ce2e View commit details
    Browse the repository at this point in the history

  3. Check for nil config and make the loop simpler 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    63d5178 View commit details
    Browse the repository at this point in the history

  4. Fix ineffectual assignment to err 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    e7f0bc6 View commit details
    Browse the repository at this point in the history

  5. Cleanup extender tests 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    21f9560 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'runext/extend-998' into runext/export-999

    @natalieparellano
    natalieparellano committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    7761161 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2023

  1. Fix typo 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    74eddd9 View commit details
    Browse the repository at this point in the history

  2. Add positive assertion 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    a43a606 View commit details
    Browse the repository at this point in the history

  3. Make constants for asserted log lines 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    007da92 View commit details
    Browse the repository at this point in the history

  4. Update comment 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    924351f View commit details
    Browse the repository at this point in the history

  5. Merge branch 'runext/extend-998' into runext/export-999

    @natalieparellano
    natalieparellano committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    a67f3c1 View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2023

  1. Acceptance test passes 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    47c05dd View commit details
    Browse the repository at this point in the history

  2. Fix some TODOs 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    979518e View commit details
    Browse the repository at this point in the history

  3. Update imgutil to point to branch commit 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    0313b88 View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2023

  1. Add support to extending daemon images 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    c3922cd View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2023

  1. WIP 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    dfdd162 View commit details
    Browse the repository at this point in the history

  2. Fix image ref when exporting to daemon 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    5f5942a View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2023

  1. More fixes 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    15a3727 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2023

  1. Improve script to test on fork 

    Clarify names of cosign secrets and remove Codecov

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    b08e51c View commit details
    Browse the repository at this point in the history

  2. Cleanup extender 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    1cbf231 View commit details
    Browse the repository at this point in the history

  3. Update comments 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    fad8bde View commit details
    Browse the repository at this point in the history

  4. Small fixes, units pass 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    aa2453e View commit details
    Browse the repository at this point in the history

  5. Add generator units 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    df6ab56 View commit details
    Browse the repository at this point in the history

  6. Fixes 

    - We shouldn't try to use extension layers if they exist in <layers>/extended unless experimental features are enabled
- We only know the size of the compressed layer, so we can't use io.CopyN (but we don't need to guard against decompression bomb anyway since we control the layers)

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    6d8b300 View commit details
    Browse the repository at this point in the history

  7. Fix test 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    588eb30 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into runext/export-999

    @natalieparellano
    natalieparellano committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    75fcfda View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2023

  1. Rename fixture dirs so that we can check out source code on Windows 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    39cbf64 View commit details
    Browse the repository at this point in the history

  2. Bring back Cleanup() 

    We need access to the final image after all Dockerfiles have been applied,
so we need to create the working directory outside the applier loop.

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    4f68f04 View commit details
    Browse the repository at this point in the history

  3. Update imgutil branch 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    6b6d133 View commit details
    Browse the repository at this point in the history

  4. Fix restorer acceptance 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    90851c0 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2023

  1. Fix acceptance test by making it less brittle 

    Instead of hard-coding the run image and run image top layer SHAs,
we can derive their values

Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    00938a2 View commit details
    Browse the repository at this point in the history

  2. Fixups from PR review 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    9e6752c View commit details
    Browse the repository at this point in the history

  3. Fix units 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    224a9a0 View commit details
    Browse the repository at this point in the history

  4. Point imgutil back to main 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    9acea63 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2023

  1. Update generator.go 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano authored Apr 14, 2023
    Configuration menu
    Copy the full SHA
    83686a6 View commit details
    Browse the repository at this point in the history

  2. Update comment 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    f24dc34 View commit details
    Browse the repository at this point in the history

  3. Update exporter.go 

    Co-authored-by: Jesse Brown <jabrown85@gmail.com>
Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano @jabrown85
    natalieparellano and jabrown85 authored Apr 14, 2023
    Configuration menu
    Copy the full SHA
    4da88b1 View commit details
    Browse the repository at this point in the history

  4. Check err type when we fail to get a layer from a sparse image 

    Signed-off-by: Natalie Arellano <narellano@vmware.com>
    @natalieparellano
    natalieparellano committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    4707bcf View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into runext/export-999

    @natalieparellano
    natalieparellano committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    829e188 View commit details
    Browse the repository at this point in the history