Notation design goals

[gokarnm]

Notation and notation-go-lib design goals

Goals

• An integrator should be able to reuse most of the implementation of Notary v2 specs from the lib, without being required to use the CLI
  • At a high level this includes registry authentication, interaction with registry including ORAS APIs, key config, trust store and trust policy, plugin model, Notary V2 signature envelope
• An integrator should be able to generate basic Notary V2 signatures without dependency on other features, like Notation key configuration
• An integrator should be able to verify basic Notary V2 signatures using their own verification policies, without reliance on Notation defined trust store and trust policy language.
• Notation should support multiple signature formats without requiring extensive changes to complete implementation
• Notation sign/verify, including remote signing, should be cable of supporting non-registry scenarios

Proposal

• Split notation-go-lib into two part.
  • notation-core-go implements
    • Generation and verification of Notary v2 signature spec using specific signature formats (JWS, COSE).
    • Signature format agnostic interface and types which can be consumed by higher layers
    • x509 certificate chain validation
  • notation-go
    • Has a dependency on notation-core-go
    • Implements registry auth and registry interactions, including ORAS APIs, key config, trust store, trust policy, plugin interaction
• notation CLI will be a thin layer that depends on notation-go for most functionality
• Versioning
  • Use semantic versioning for each component notation-core-go, notation-go and notation. Each component revises versions independently.
  • Every source release contains latest versions of notation, notation-go-lib and notation-core.
  • Every binary release only containes notation, dependencies are statically linked.

[gokarnm]

@GaryWA-ops I'll hide this comment as this does seem to be directly relevant to this topic. Notation is an open source tool to sign OCI artifacts (like docker images), and not related to real estate or document notarization. Feel free to open a new issue/discussion if needed.

[shizhMSFT]

Since we have two repositories notation-go and notation-core-go, the package name for notation-go is obvious notation. How about notation-core-go? Should it be notationcore or just core? I personally prefer core as it is shorter.

Here's the convention of golang package naming: https://go.dev/doc/effective_go#package-names.

[shizhMSFT]

The intention of this design is to hide notation-core-go from notation. It can be viewed that all packages in notation-core-go are internal packages for notation-go.

[qmuntal]

Which is the problem we are trying to solve by splitting the non-cli logic into notation-core-go and notation-go, instead of having it all in notation-go?

My gut feeling is that it would add complexity and maintainability burden to our ecosystem:

• If we need to fix a crypto bug, we will have to cut a version of three libraries instead of one.
• It will slow down development of new features which require changes in the three repos.
• More repos = more CI infrastructure and code

If we stick to just notation-go, we also accomplish the goals:

• If a third-party library only needs the crypto logic, so no messing up with registries, it will only import the notation-go/crypto/* packages. Go compiler is smart enough to discard from the final binary any unused package and its dependencies.
• If a third-party library only needs the registry logic, so no messing up with crypto stuff, it will only import the notation-go/registry/* packages. As before, Go won't include unused packages and their dependencies in the final binary.

[gokarnm]

@shizhMSFT @rgnote @sajayantony any feedback? I thought we discussed this option before we split the package. Don't remember the downside of it.

[shizhMSFT]

I've mentioned the downside in our discussion just as @qmuntal has mentioned above that we will have version control issues. Especially, we might have a version mismatch for notation-core-go and notation-go.

AFAIU, the major reason of the repo split is that notation-core-go and notation-go have different paces of development. The notation-core-go will not be updated unless there is a security vulnerability, and the notation-go will be updated rapidly for trust policy, config works.

[qmuntal]

I see the benefit of having the security-critical logic confined in well-defined boundaries.

Here is another take which tackle some versioning problems and stills keeps the crypto code in its own repo: Split the logic between notation-go and notation-crypto-go (renamed from notation-core-go), but don't make notation-go depend on notation-crypto-go. This means notation, and possibly other tools, will depend on both instead of just on notation-go.

With this approach, both repos can be released independently, which means:

• We don't need to cut notation-go when there is a new version of notation-go-core
• Development of new features have one integration point less

Additionally, someone could decide not to use notation-crypto-go and use their own crypto stack.

On the other hand, I still don't know which would be the API of notation-core-go, so I can't know if it is possible to design a compatible API between both repos without making them depend on each other.

[shizhMSFT]

Quoting my comment above.

The intention of this design is to hide notation-core-go from notation. It can be viewed that all packages in notation-core-go are internal packages for notation-go.

[gokarnm]

Things to refactor

• X509 certificate chain is validated twice, once in Signer method and also at the end of signing workflow with plugin
• Notation CLI generates signature manifest currently, we want to push this down to notation-go, and Sign method should change accordingly
• Sign/Verify method in notation-go should be config paths aware. I suggests we have CLI construct a config set which consideres env variables and other rules to get a list of base paths for trust store, policy, config.json etc, and pass as a context parameter to Sign/Verify.

[jondonas]

Observations about additional things to refactor:

notation-go

• crypto/cryptoutil/(cert.go/key.go) should be removed because they've already been moved to notation-core-go
• crypto/jwsutil/* contains jws utils (signing workflow) which should be moved to notation-core-go
• crypto/timestamp/* contains timestamping signing utils (signing workflow) which should be moved to notation-core-go
• crypto/signature/jws/(signer.go/spec.go) contains jws utils (signing workflow) which should be moved to notation-core-go