Support Signing

[steiza]

Description

It would be great if sigstore-go could not just verify, but also sign bundles.

There aren't many libraries that support signing bundles today (just sigstore-js?) This would also allow sigstore-go to support the full range of tests in sigstore-conformance.

Goals

• sigstore-go/pkg/sign/ supports signing Sigstore bundles
• cmd/conformance/main.go is updated to support sign-bundle and sign (similar to verify, sign will "wrap" the bundle flow).
• Have a "signers API"
  • Have built-in support for signing with Fulcio and a keypair where you have direct access to the private key
    • I'm leaning towards a "batteries included" Fulcio support in the library itself, but if we decide against it we'll at least need support in the conformance driver
• Have a "witness API"
  • Built-in support for a timestamp authority
  • Built-in support for Rekor: entry types hashedrekord and DSSE
• Produces bundles with v0.3.1 (open to feedback here)
• Signing requires exactly 1 signer and at least 1 witness

Anti-Goals

• If we update cmd/sigstore-go/main.go, it should not compete with cosign
• API will support providing custom signers, but we won't support any KMS / HSM-based signing in sigstore-go itself

References

• Sigstore client specification
• https://github.com/sigstore/sigstore-js/blob/main/packages/sign/README.md
  • and the discussion on Signing Abstractions sigstore-js#499

[steiza]

So here is a very rough example of what pkg/sign/ might look like:

For interface.go:

package sign                                                                    
                                                                                
import (                                                                        
    protobundle "github.com/sigstore/protobuf-specs/gen/pb-go/bundle/v1"        
    protocommon "github.com/sigstore/protobuf-specs/gen/pb-go/common/v1"        
)                                                                               
                                                                                
// this is just a convienient way to bundle the information from the signer to the witness(es)
type SignedEntity struct{                                                       
    dataDigest []byte                                                           
    signature []byte                                                            
    ephemeralPublicKey []byte                                                   
    envelope []byte                                                             
}                                                                               
                                                                                
func Sign(data []byte, signer *Signer, witnesses []Witness, bundleVersion string) (*protobundle.Bundle, error) {
    bundle := Bundle{}
    
    signerResult, err := signer.SignData(data)                                  
    if err != nil {
        return nil, err                                                         
    }   
    
    // Put signerResult into SignedEntity, depending on type                    
    // Put signerResult into bundle, depending on type
    
    for _, witness := range witnesses {                                         
        witnessResult, err := witness.Witness(se)                               
        if err != nil {
            return nil, err                                                     
        }   
        // Put witnessResult into bundle, depending on type                     
    }   
    
    return bundle, nil                                                          
}

For signer.go:

package sign                                                                    
                                                                                
import (                                                                        
    "crypto"
    "crypto/x509"
)                                                                               
                                                                                
type Signer struct{}
    
type SignerResult struct{                                                       
    // Populate these fields if you are signing with a key                      
    signature []byte                                                            
    publicKeyHint []byte                                                        
                                                                                
    // Populate these fields if you are signing with Fulcio 
    ephemeralPublicKey []byte
    signingCertificate *x509.Certificate
}   
    
func (s *Signer) SignData(data) (*SignerResult, error) {                        
    return nil, errors.New("not implemented")
}   
    
type Fulcio struct {
    baseUrl string
}   
        
type Keypair struct { 
    crypto.PublicKey
    crypto.PrivateKey
    publicKeyHint []byte
}   
    
func (f *Fulcio) SignData(data) (*SignerResult, error) {}                       
func (k *Keypair) SignData(data) (*SignerResult, error) {}

For witness.go:

package sign                                                                    
                                                                                
import (                                                                        
    "errors"
    
    protocommon "github.com/sigstore/protobuf-specs/gen/pb-go/common/v1"        
    protorekor "github.com/sigstore/protobuf-specs/gen/pb-go/rekor/v1"          
)
    
type Witness struct{}
    
type WitnessResult struct {                                                     
    *protocommon.RFC3161SignedTimestamp                                         
    *protorekor.TransparencyLogEntry                                            
}   
    
func (w *Witness) Witness(se *SignedEntity) (*WitnessResult, error) {
    return nil, errors.New("not implemented")
}   

type TimestampAuthority struct{
    baseUrl string
}   

type Rekor struct {
    baseUrl string
    entryType string
}
    
func (ta *TimestampAuthority) Witness(se *SignedEntity) (*WitnessResult, error) {
    // uses se.signature
}   
    
func (r *Rekor) Witness(se *SignedEntity) (*WitnessResult, error) {             
    // hashedrekord uses se.signature and se.dataDigest 

    // dsse uses se.envelope and se.ephemeralPublicKey
}

[haydentherapper]

I'll leave comments on the PR for the interface!

One quick comment is that "witness" needs a different name to avoid conflicting with the concept of witnessing from the tlog ecosystem. I also want to not conflate timestamping and auditability. It just so happens that the log can be used for both currently, but we would like to move towards timestamping being a requirement independent of the log so that you never have to trust the log.

+1 to "batteries included" Fulcio and private key support.

0.3.1 bundle support

As long as other clients support 0.3.x bundles, SGTM.

A few more goals I think we need to tackle:

• Support ambient credentials, i.e. the "providers" interface from Cosign, so that credentials can be fetched from the environment for automated signing
• Support interactive signing for developer-signed artifacts
• Only output bundles - I'm open to debate on this, and this will be a blocker for Cosign adoption until we have bundle support there, but I would rather not deal with detached verification metadata.

[haydentherapper]

Signing requires exactly 1 signer and at least 1 witness

I don't think we should enforce the latter because in the case of private key signing in a private environment, neither logging nor a timestamp is needed. As a CLI, not logging should be discouraged, but as an API, I don't think it should be opinionated.

[steiza]

@vishal-chdhry asked me today "what's left on signing?" so I thought I'd write it down here.

• How should we verify the content we receive, like the Fulcio certificate?
  • I'm thinking of adding the concept of a trusted_root.json to the signer, so it can verify responses it gets from not just Fulcio, but also Rekor and Timestamps
  • The conformance tests and example signing command line tool would make use of the TUF client to get a trusted_root.json
• Clients currently support timeouts but not retries; we will certainly want retries for any production scenario
• We should review the signing interface (cc @phillmv and @codysoyland)
• How much testing is enough?
  • We are adding unit testing and conformance testing to signing. There isn't much testing of Fulcio / Rekor / Timestamp clients, but much of that functionality lives in external libraries.
  • Should we add end-to-end tests with local Fulcio / Rekor / Timestamp providers?

[cmurphy]

I'm new to sigstore-go and was curious about a few things here:

One quick comment is that "witness" needs a different name to avoid conflicting with the concept of witnessing from the tlog ecosystem

Can confirm, I was confused by this on first read. It looks like this term is already used in sigstore-js https://github.com/sigstore/sigstore-js/blob/main/packages/sign/README.md#witness so the train might have left the station on that one. For my education, could you clarify what a witness is supposed to be used for in this context? Does it have an equivalent concept in cosign?

Support ambient credentials, i.e. the "providers" interface from Cosign

With sigstore-go being mainly an API library, I wonder if it's really necessary to have built-in provider support the way cosign does - couldn't a Signer be implemented for a provider in its own library, which the user could import and pass to Sign when they call it?

Support interactive signing for developer-signed artifacts

Is it typical for other client libraries to have an interactive mode? What is the use case for using sigstore-go in interactive mode versus just using the cosign CLI?

Should we add end-to-end tests with local Fulcio / Rekor / Timestamp providers?

I think it's not a bad idea to at least add a basic smoke test for integration with the services, that way the skeleton is there and someone who wants to add a riskier feature has a starting point for more complex testing.

[steiza]

Hi @cmurphy, and welcome to sigstore-go!

could you clarify what a witness is supposed to be used for in this context?

I think the sigstore-js packages/sign/README.md#witness says it pretty well:

Each Witness receives the artifact signature and the public key and returns an VerificationMaterial which represents some sort of counter-signature for the artifact's signature. The returned VerificationMaterial may contain either Rekor transparency log entries or RFC3161 timestamps.

So basically it's a Rekor entry or a RFC3161 timestamp (or possibly both). Inside GitHub, we use the term "witness" as a shorthand for "a Rekor entry if you're working with the Sigstore Public Good Instance (like in the case of npm build provenance) or a RFC3161 Timestamp if you're using GitHub's internal Sigstore instance (like in the case of Artifact Attestations in a private repository)". This shorthand has definitely been useful inside GitHub, but I don't think it has caught on in the wider Sigstore community.

For sigstore-go we decided to use pkg/sign/transparency.go and pkg/sign/timestamping.go instead of combining them in the proposed pkg/sign/witness.go.

Does it have an equivalent concept in cosign?

I knew cosign supported Rekor, but I had to double check on the RFC3161 Timestamp, which it looks like it does support with --timestamp-server-url.

With sigstore-go being mainly an API library, I wonder if it's really necessary to have built-in provider support the way cosign does

Exactly; we ended up deciding that Signer would take an IDToken, but it was up to the caller of the sigstore-go API library to figure out how to obtain that IDToken.

What is the use case for using sigstore-go in interactive mode versus just using the cosign CLI?

Yeah, once we decided Signer would take an IDToken the interactive mode was scoped out of sigstore-go as well. Users of the sigstore-go library (like cosign, possibly) are welcome to make an interactive mode, obtain the IDToken, and then call into sigstore-go.

I think it's not a bad idea to at least add a basic smoke test for integration with the services, that way the skeleton is there and someone who wants to add a riskier feature has a starting point for more complex testing.

Agreed!

[haydentherapper]

I knew cosign supported Rekor, but I had to double check on the RFC3161 Timestamp, which it looks like it does support with --timestamp-server-url.

Yes, Cosign supports RFC3161 timestamp authorities. Those timestamps can be persisted in OCI or as detached metadata.

With sigstore-go being mainly an API library, I wonder if it's really necessary to have built-in provider support the way cosign does
Exactly; we ended up deciding that Signer would take an IDToken, but it was up to the caller of the sigstore-go API library to figure out how to obtain that IDToken.

To add some more context, what's missing is a "batteries included" configuration. I agree that interactive signing doesn't need to be a part of sigstore-go, though the question is, where should that logic live? How do we avoid integrators needing to reimplement the logic in Go to fetch identity tokens per platform? I would like this implemented somewhere in the Sigstore org, to minimize adoption burden and maximize the number of supported platforms.

We've got a few options:

• sigstore-go - A downside being increasing the dependency tree and muddying the separation between API and CLI
• sigstore/sigstore - A catch-all utility repo for Go clients and services, albeit it's poorly named. The OIDC/OAuth logic already resides there. This feels like the best place if not sigstore-go.
• Cosign - I don't think this is the right place, because integrators would have to pull in Cosign's dependency tree mess. Additionally it makes other Go Sigstore applications like Gitsign depend on Cosign, which is the wrong dep graph in my opinion, as Cosign should be a CLI and an API for container signing only.
• Another library dedicated to signing setup - There's overlap with sigstore/sigstore, though this could be less general purpose.

My two cents, if not sigstore-go, let's move the "provider" logic from Cosign into sigstore/sigstore and demonstrate with an example application how to compose sigstore-go and sigstore/sigstore for signing blobs either interactively or on an automated platform. Does this seem reasonable?

"What's left for signing?"

One other feature would be support for the ClientTrustConfig, which lets clients specify with a single configuration file the endpoints they will contact for the CA, logs and TSAs. This has been implemented by sigstore-python - sigstore/sigstore-python#1010. I'll create an issue to track this. (Edit: #185)

[cmurphy]

I would like this implemented somewhere in the Sigstore org, to minimize adoption burden and maximize the number of supported platforms. We've got a few options:

What about either of:

1. a new repo sigstore/sigstore-providers
   • avoids adding a bunch of new dependencies to sigstore/sigstore
   • avoids making sigstore/sigstore even more of a "utils" dumping ground
2. individual repos for each provider, e.g. sigstore/sigstore-spiffe, sigstore/sigstore-google
   • each repo only has the idp provider dependencies that it needs, so pulling in one provider to an application doesn't pull in the kitchen sink
   • (con) the sigstore org becomes more cluttered

[haydentherapper]

What about either of:

1. a new repo sigstore/sigstore-providers

   • avoids adding a bunch of new dependencies to sigstore/sigstore
   • avoids making sigstore/sigstore even more of a "utils" dumping ground

2. individual repos for each provider, e.g. sigstore/sigstore-spiffe, sigstore/sigstore-google

   • each repo only has the idp provider dependencies that it needs, so pulling in one provider to an application doesn't pull in the kitchen sink
   • (con) the sigstore org becomes more cluttered

Sorry, missed responding! Given the complexity of adding repos into the org and finding active maintainers for each, having a single repo with a set of providers seems like a good approach.

We can revisit this as part of Cosign refactoring.

[spencerschrock]

Exactly; we ended up deciding that Signer would take an IDToken, but it was up to the caller of the sigstore-go API library to figure out how to obtain that IDToken.

Figuring out how to extract the IDToken is where I'm at now, as ossf/scorecard-action considers dropping sigstore/cosign in favor of sigstore/sigstore-go.

a new repo sigstore/sigstore-providers

If we switched today, I would likely copy Cosign's GitHub provider, but using a library would be nice.

[haydentherapper]

Yea, you'll need Cosign's GitHub provider if you're fetching the token as part of a Go binary, or if you're running in an action, getting the token following https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-cloud-providers#adding-permissions-settings and passing that to the Scorecard binary.

cc @ramonpetgrave64 since we had chatted about this