Collect requirements/idea/features for v2.7.0

[OrlinVasilev]

Hello Community,

as we released v2.5 and we have plan and roadmap for v2.6 it's time to brainstorm the ideas and requirements for the feature list for 2.7!

Please share your thoughts here so we can all work on proper roadmap and achieving it!
Keep in mind that Harbor is community developed and driven project so not everything can be implemented right away :)
If you have an idea or need and eager to work on, we will support you and help you get there!

Thank you being essential part of the community!

Orlix
Harbor Community Manager

[AlexBarth13]

We would appreciate a working API concept where "normal" users and robot accounts/technical users can make automated API calls to Harbor

Edit: added and robot accounts/technical users

Alexander Barth (alexander.barth@mercedes-benz.com) on behalf of Mercedes-Benz Tech Innovation GmbH, Provider Information

[AlexBarth13]

Enable that e.g. projects/repositories can be opened in a new browser tab



Alexander Barth (alexander.barth@mercedes-benz.com) on behalf of Mercedes-Benz Tech Innovation GmbH, Provider Information

[ekarlso]

Would it be poasible to add user federation concept to harbor where you could use something like girhub federated identity to login?

[shawngmc]

Would it make sense to integrate that directly into Harbor or use Keycloak as an OIDC provider for Harbor, and let Keycloak handle the bridging to external ID providers (Google, GitHub, etc.)?

[shawngmc]

A few for now, might have some more ideas later:

• Artifact Vulnerability UI: Button to download the existing JSON report (as from the /projects/{project_name}/repositories/{repository_name}/artifacts/{reference}/additions/vulnerabilities API endpoint)
• Artifact Vulnerability UI: Some sort of HTML export option. The current UI is fine for quick glances, but a nice report to share/comment on/not need to expand every section would be nice.
• More explicit options to configure Harbor from the YML file; for example, the OIDC connection has to be supplied in the UI and I have issues where the client secret is 'lost' when redeploying because the encryption secret has changed.
• Improved documentation of the contents after the prepare/redeploying with them

[AllForNothing]

CVE data exporting feature is on process goharbor/community#174

[AlexBarth13]

Just for convenient reasons:

• Allow to use the links in the documentation without the version number in it and automatically be redirected to the latest version.

Example:
Working: https://goharbor.io/docs/2.4.0/administration/configure-proxy-cache/
Not working: https://goharbor.io/docs/administration/configure-proxy-cache/

Example from Tanzu:
https://docs.pivotal.io/application-service/concepts/overview.html will be automatically redirected to
https://docs.pivotal.io/application-service/2-12/concepts/overview.html



Alexander Barth (alexander.barth@mercedes-benz.com) on behalf of Mercedes-Benz Tech Innovation GmbH, Provider Information

[OrlinVasilev]

I was thinking the same in the past, and that is not related to the release anyway :)
Can you create an issue in the website repo or should I?

[AlexBarth13]

Thanks for pointing this out!
Of course you are totally correct, that this is not related to an release - sorry!
I will create an GitHub Issue for this :)



Alexander Barth (alexander.barth@mercedes-benz.com) on behalf of Mercedes-Benz Tech Innovation GmbH, Provider Information

[zyyw]

It seems that https://goharbor.io/docs will be automatically redirected to the latest version of doc, for example https://goharbor.io/docs/2.5.0, while subpages will not be redirected, for example https://goharbor.io/docs/administration/configure-proxy-cache/ will not be redirected and report 404.

[thangamani-arun]

Hi,

I know harbor has geo-replication feature, but it is currently replicating only the repositories. whatever goes to database like users, projects. quota's etc are not replicated. We use postgres streaming replication to remote site, but the local harbor-core instances requires a WRITABLE DB on a read-only postgres replica pod, which is not possible unless it got promoted to master mode.

It possible to have active-standby across DC-DR sites, so everything gets replicated across sites. In this case, we don't have to rely on DB replication. DB can be updated using harbor replication feature beyond registry replication?

[Vad1mo]

I would like to see some improvements regarding the API. When you work with the API a lot of things only make sense in the frontend's context and hence a designed like this.

There are quite a few issues around already. I plan to create a summarizing issue or start a discussion.

[qnetter]

Please so. The API should serve the UI, but design should be API-first.

[AlexBarth13]

Working webhook integration between Harbor and Mattermost. Currently, Harbor is not able to send a message to Mattermost because the format of the message is wrong (for Mattermost).
Details can be found here: #16632 (comment)



Alexander Barth (alexander.barth@mercedes-benz.com) on behalf of Mercedes-Benz Tech Innovation GmbH, Provider Information

[Vad1mo]

From the UX/DX point of view and considering the bigger role of OCI artifacts in the Cloud Native ecosystem.
We should extend Harbor to display different typed of OCI artifact in different groups (tabs)

There are ton of projects already using OCI registries to store other artifacts then container images, eg. ORAS, Helm OCI, Singularity, ML Data Models and so on.

At the Moment we only have two groups/tabs
Repositories and Helm Charts. The Helm Charts stored in Chartmuseum are displayed in their own group/tab

My proposal for Harbor 2.7 is that we should extend Harbor in a way that the user can decide on a project basis what groups he wants to see. There should be a default list in the project like Containers, Helm Chart and OCI Artifacts but the user should be able to change that according to the image metadata.

The results would be that an ML team could config their project to only see ML Data to TF and the software engineering team would be fine with the defaults.

[AlexBarth13]

I would like to have the option to add a banner where I can add some information (e.g. maintenance information,...). It could look like this:

Alexander Barth (alexander.barth@mercedes-benz.com) on behalf of Mercedes-Benz Tech Innovation GmbH, Provider Information

[shawngmc]

This would be a big help for managing an instance for multiple client teams. I'd love to take it a step further. Add:

• A customizable text box (maybe the ability to add links) to the login page in the large 'blank' area.
• A persistent banner
• An 'alert' banner that could be dismissed and hidden (until next login or the message changes - I could make arugments for both)

[jayolmos]

Hello, it would be nice to have Trivy metrics about CVE in artifacts to integrate them with prometheus + grafana.

[shawngmc]

I'd hope that this wouldn't be Trivy-specific, but something gathered through the Harbor Scanner API. My install uses Anchore and the Harbor Anchore Scanner Adapter.

[OrlinVasilev]

this one looks must have #17125 @dkulchinsky

[dkulchinsky]

Thank you @OrlinVasilev, would very much to like to see this being implemented.

This becomes especially important for scaling Harbor and still being able to efficiently scan for vulnerabilities stored images.

I think this may indirectly depend on #14638 getting fixed, which seems like there's some possible fix inflight 🤞🏼 🙏🏼

[slushysnowman]

Yep this looks like a good call.

[dkulchinsky]

looks like this won't make into 2.7 afterall #17125 (comment) 😞 but hopefully still on track for 2.8 🙏🏼

[PengWuLin]

The image is too big now

With the upgrade of the go version and the addition of functions, the image is also getting larger。
I want to be able to optimize the image size。

image	v2.3.5	v2.5.3
Core	193MB	203MB
JobService	211MB	227MB
Portal	58.9MB	52.6MB
Registry	82.6MB	77.9MB
Registryctl	133MB	136MB
Trivy	133MB	251MB

[PengWuLin]

Is it possible to use the following means

1. Remove debugging information
2. Use UPX tool, UPX is an advanced executable file compressor. UPX will typically reduce the file size of programs and DLLs by around 50%-70%, thus reducing disk space, network load times, download times and other distribution and storage costs.
   go build -ldflags="-s -w" -o server main.go && upx -9 server

[Vad1mo]

UPX have quite a few disadvantages, I would rather shrink the package and image size.

Here are some other methods reduce the size: https://github.com/xaionaro/documentation/blob/master/golang/reduce-binary-size.md

[PengWuLin]

thanks

[shawngmc]

Configurability/standardization of password requirements

Currently:

• Any account password/token has to be at least 8 characters and contain uppercase, lowercase and numbers
• User generated passwords are only 8 characters long, while manual entry allows 8-30 characters
• Robot generated passwords are 32 characters long, while manual entry allows 8-20 characters
• There are no admin options that modify these settings

For the sake of consistency and security, can we please:

• Add an admin setting for minimum password length enforced across both users and robots
• Make generation consistent between users and robots, following the admin-configured minimum length
• Add an admin setting to allow both, allow only manual entry, or allow only generated tokens

[OrlinVasilev]

@sfgets please add your label rework thoughts here :) Thanks !

[akramelnouby]

With docker not being supported on RHEL 8+ hosts, I think it would be nice to be able to setup harbor on podman (manually or by using ansible). I know that it shouldn't be that hard, because I am currently working on an ansible playbook and it works (only notary is having issues). It would be nice to see it as an official way to install harbor on nodes running podman as container management. Please also see my Q&A regarding this topic (#17268 (comment))

[shawngmc]

I added a couple questions/ideas to #17268; please take a look and see if these solutions might work.

[danielzhanghl]

logging format requirement:

right now Harbor use it's own log library, src/lib/log/*

could consider to use open source logging library like "github.com/Sirupsen/logrus"

or, as Harbor use beego, could use its log as well:
https://github.com/beego/beedoc/blob/master/en-US/module/logs.md

then it will be easy to provide more log options, like json format, out put to file/socket endpoint, like

https://docs.docker.com/registry/configuration/#log
log:
accesslog:
disabled: true
level: debug
formatter: text
fields:
service: registry
environment: staging

[slushysnowman]

I'd REALLY like to see Harbor admins be able to disable the ability to set projects 'public' on a per-project level (see #12306)

The really nice thing about Harbor is that we are able to offer stuff in a self-service way to our teams by giving out project admin rights to certain subgroups of users. But the fact that they can accidentally or otherwise click a button and set their project to be fully public, no auth required to pull images, is super scary.

We'd really like the ability to turn it off, and I think the lack of ability to be able to do so is probably a blocker for some organisations looking to adopt Harbor

[slushysnowman]

the 'project' level switch is the minimum I'd like to be able to disable, all the other suggestions in #12306 are also very relevant

[slushysnowman]

This is also an issue we'd really like to see fixed - #14638

We do daily scans with Trivy, but this resets 'last pull' so retention policies based on time since last pull are just broken, meaning we can't do automatic removal of old / unused images. It's a pretty old issue as well, so it'd be great to see it get picked up and fixed.

[dkulchinsky]

big +1 on that and I think it would be critical to enable #16775 (comment) (as I mentioned there as well)

[AlenversFr]

Cleaning the technical tables in postgres or more interesting add some harbor parameters to schedule cleaning actions (cron + how many we keep)

linked here https://github.com/goharbor/harbor/issues/17538

[dkulchinsky]

I think gaps in GC have to be addressed

there's a story from last year #16010 listing the main issues, some low hanging fruit items were addressed.

However from our experience, GC becomes mostly useless in large scale deployments of Harbor.

Specifically, with the following issues that need to be addressed:

1. [GC performance] The performance of v2 manifest deletion is not good in S3 environment #12948
2. GC fails after deleting many tags from same repository #15807
3. GC fails with "invalid checksum digest format" from registry when deleting manifests #15970

some of these issues are 2+ years old, especially #12948 which in our case results in GC being able to clean only ~200 artifacts in 24 hours, while we have >170,000 artifacts in the thrashed queue.

[dkulchinsky]

/cc @Vad1mo

[Vad1mo]

we still see problems popping up with GC.

[lengrongfu]

Optimize the push image processing logic when the quota is full. #17390