Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Container Retention Policy #3474

Closed
wants to merge 3 commits into from
Closed

Conversation

ruffsl
Copy link
Member

@ruffsl ruffsl commented Mar 13, 2023

To try and archive only one image per month over the past year to date.

As we seem to be hoarding a large number of old images, and I'd rather not have to manually deleting them from the web UI.

while only keeping at least 12
to try and archive one image for each month over the year to date
while only keeping at least 1
to try and archive only one image per month
to alias over github repository name
@ruffsl ruffsl added the CI label Mar 13, 2023
@ruffsl
Copy link
Member Author

ruffsl commented Mar 13, 2023

@SteveMacenski , could you please create new PAT for the ros-planning org, granted with the scopes detailed in:

and add to the Repository secrets for Actions, under the variable name GHCR_PAT:

https://github.com/ros-planning/navigation2/blob/653b269259fd82bbb493715806e8c3bda0aaa7a2/.github/workflows/update_ci_image.yaml#L133

If you see something akin to this prompt, you can decide if/how you'd like to approve each fine-grained personal access token that can access the organization.

https://docs.github.com/en/organizations/managing-programmatic-access-to-your-organization/setting-a-personal-access-token-policy-for-your-organization

@ruffsl
Copy link
Member Author

ruffsl commented Mar 13, 2023

Alternately, I'm also thinking of forgoing our current timestamp tagging approach, and perhaps tagging the CI images by the current release version instead:

https://github.com/ros-planning/navigation2/releases

As that seems to have a much lower cadence, and would be semantically more relevant if needing to pull an old image from the archive. Thus each tag would retain the most recently build CI image with a build environment targeting that version. E.g:

  • main, 1.1.6
  • 1.1.5
  • 1.1.3
  • ...

In retrospect, timestamping the image tags now seems a bit overkill, given we haven't had to roll back and pin our CI image for quite a while.

@SteveMacenski
Copy link
Member

@tfoote is there a way to generate a PAT for the organization rather than being hooked into my personal account? I'm a little weary about creating a PAT that can access my account stored in github's servers in case of hacking (which seems to be the theme of the day). Restricting it to Nav2 at the org level would be the safest bet.

The biggest thing of using the release tags is that the main isn't really updated in those situations, only the release branches. I do 0.X.0 updates for new distribution releases when forking it for the new release branch

@ruffsl
Copy link
Member Author

ruffsl commented Mar 13, 2023

is there a way to generate a PAT for the organization rather than being hooked into my personal account?

@SteveMacenski , as an admin, are you able to view the orgs settings page?

https://github.com/organizations/ros-planning/settings/personal-access-tokens-onboarding

This should be where one can generate PATs for the organization, rather than specific to a user account. However, this page is marked as beta for orgs I own.


The biggest thing of using the release tags is that the main isn't really updated in those situations, only the release branches.

I was just thinking of having the CI read the version number from the package file, and tagging the image with the same version number string. So until you bump up the version in the code, the action would just push the generated image to the same tag in the image registry. So even if commits frequently cause the CI image to rebuild, we would still have clear demarcations of the main image's history via the version bumps.

@SteveMacenski
Copy link
Member

Yeah, I can see that page but I don't see any way via it to add a PAT. I'm wondering if its because its not possible or if I don't have the permissions to see that capability. All I see on that page are toggles to allow PATs.

Sure, but then the main branch (where development mostly happens in reality) would get out of date pretty fast and increasingly so over time and only refreshed back to state 1x a year with the current process.

If the rebuilding is in GitHub Actions (free to us), why not just update it in some regular cadence?

@ruffsl
Copy link
Member Author

ruffsl commented Mar 13, 2023

All I see on that page are toggles to allow PATs.

Oh, so when first enrolling and enabling org level PATs, this is what I see fair one of my other orgs:

SmartSelect_20230313_232705_Chrome

With those being still being the most conservative choices over the three enrollment options, any admin can navigate to here to create a fine-grained PAT:

https://github.com/settings/personal-access-tokens/new

Unfortunately, with fine-grained personal access tokens still under beta, that notably allow for setting the organization as the resource owner, it looks like not all the necessary endpoints are mapped to listed permissions under the drop-down menus when creating a fine-grained token.

SmartSelect_20230314_001055_Chrome

However, I can see the necessary permissions where viewing creating a classic token, specifically the read and delete permissions for packages:

SmartSelect_20230314_002113_Chrome

https://github.com/settings/tokens/new

With packages being hosted by the organization, rather than under a specific repo, my guess is that they haven't yet extended the UI to express those kind of associative permissions.

@nuclearsandwich , forgive me for pinging you yet again, but as our GitHub guru, I figured this would be a familiar topic to you.

@SteveMacenski
Copy link
Member

I think it might be useful if you had admin permissions to poke around. You're a known responsible party. @nuclearsandwich @tfoote any objections?

We could also potentially make a nav2bot account and have that have a PAT we use for CI. I'm mostly concerned about my personal account token being used for this in case its stolen at some point in a hack years down the line and that I have access to some things that are not good to be public around that time. If we have a dummy account, it won't have access to anything but ros planning, which is all open source.

@ruffsl
Copy link
Member Author

ruffsl commented Mar 14, 2023

We could also potentially make a nav2bot account and have that have a PAT we use for CI.

We've used a bot before for managing docker related tasks over on osrf/docker_imsges, although I'm not sure we'd want to reuse it across multiple orgs. Principle of least privilege and what not. Why did GitHub make it so hard to use minimally scoped tokens?

https://github.com/osrf-docker-builder

BTW, That bot still need some attention, as I suspect it's PATs have expired, thus it falling dormant. We'd love for it to come back:

@tfoote
Copy link

tfoote commented Mar 15, 2023

Yeah, historically we've used a dedicated bot account which is given minimum access requirements to do the necessary operations.

@SteveMacenski
Copy link
Member

SteveMacenski commented Mar 15, 2023

@tfoote can we give Ruffin appropriate permissions in the org? He's pretty regular in doing odd stuff like this which require elevated privileges.

@ruffsl
Copy link
Member Author

ruffsl commented Mar 20, 2023

Replaced by #3491 after continuing this thought: #3474 (comment) .

@ruffsl ruffsl closed this Mar 20, 2023
@ruffsl ruffsl deleted the container-retention-policy branch March 20, 2023 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants