Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow read data from stdin on Windows #1104

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,7 @@ Encrypting using Hashicorp Vault

We assume you have an instance (or more) of Vault running and you have privileged access to it. For instructions on how to deploy a secure instance of Vault, refer to Hashicorp's official documentation.

To easily deploy Vault locally: (DO NOT DO THIS FOR PRODUCTION!!!)
To easily deploy Vault locally: (DO NOT DO THIS FOR PRODUCTION!!!)

.. code:: bash

Expand All @@ -324,11 +324,11 @@ To easily deploy Vault locally: (DO NOT DO THIS FOR PRODUCTION!!!)
.. code:: bash

$ # Substitute this with the address Vault is running on
$ export VAULT_ADDR=http://127.0.0.1:8200
$ export VAULT_ADDR=http://127.0.0.1:8200

$ # this may not be necessary in case you previously used `vault login` for production use
$ export VAULT_TOKEN=toor
$ export VAULT_TOKEN=toor

$ # to check if Vault started and is configured correctly
$ vault status
Key Value
Expand Down Expand Up @@ -1139,7 +1139,7 @@ When operating on stdin, use the ``--input-type`` and ``--output-type`` flags as

.. code:: bash

$ cat myfile.json | sops --input-type json --output-type json -d /dev/stdin
$ cat myfile.json | sops --input-type json --output-type json -d -

YAML anchors
~~~~~~~~~~~~
Expand Down Expand Up @@ -1424,8 +1424,8 @@ will encrypt the values under the ``data`` and ``stringData`` keys in a YAML fil
containing kubernetes secrets. It will not encrypt other values that help you to
navigate the file, like ``metadata`` which contains the secrets' names.

Conversely, you can opt in to only left certain keys without encrypting by using the
``--unencrypted-regex`` option, which will leave the values unencrypted of those keys
Conversely, you can opt in to only left certain keys without encrypting by using the
``--unencrypted-regex`` option, which will leave the values unencrypted of those keys
that match the supplied regular expression. For example, this command:

.. code:: bash
Expand Down
20 changes: 18 additions & 2 deletions cmd/sops/common/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -127,9 +127,9 @@ func EncryptTree(opts EncryptTreeOpts) error {

// LoadEncryptedFile loads an encrypted SOPS file, returning a SOPS tree
func LoadEncryptedFile(loader sops.EncryptedFileLoader, inputPath string) (*sops.Tree, error) {
fileBytes, err := ioutil.ReadFile(inputPath)
fileBytes, err := ReadFile(inputPath)
if err != nil {
return nil, NewExitError(fmt.Sprintf("Error reading file: %s", err), codes.CouldNotReadInputFile)
return nil, err
}
path, err := filepath.Abs(inputPath)
if err != nil {
Expand Down Expand Up @@ -441,3 +441,19 @@ func PrettyPrintDiffs(diffs []Diff) {
}
}
}

func ReadFile(path string) (content []byte, err error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this tries to do two things which really should be separate things.

In my opinion, handling of files should really be just handling of files, with os.Stdin not classifying as such. Part because from the perspective of SOPS as an SDK, decrypt.Decrypt magically reading from os.Stdin would be kind of surprising.

Having said this, I would deal with - as a special path case within commands, and e.g. create another wrapper around DataWithFormat for an io.Reader.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

files should really be just handling of files

I don't know. On Unix, stdin is just an other file descriptor and could be handle as file, too. (like /dev/stdin)

Living the unix philosophy, there should not be a difference between /mnt/file and /proc/self/fd/1.

From technical point of view, it may should fine.

Creating an other wrapper, it possible. But the other wrapper would be the exact same copy/paste code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this the same copy/paste code?

func Reader(r io.Reader, format string) (cleartext []byte, err error) {
        encryptedData, err := io.ReadAll(r)
	if err != nil {
		return nil, err
	}
	return DataWithFormat(encryptedData, FormatFromString(format))
}

Copy link
Member

@hiddeco hiddeco Jul 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB: I am fine with a function called ReadContent (or something something) existing (and utilized) within cmd/sops, but I wouldn't want this to be the default for decrypt.

if path == "-" {
content, err = ioutil.ReadAll(os.Stdin)
if err != nil {
return nil, fmt.Errorf("Failed to from stdin: %w", err)
}
} else {
content, err = ioutil.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("Failed to read %q: %w", path, err)
}
}

return content, nil
}
5 changes: 2 additions & 3 deletions cmd/sops/encrypt.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package main

import (
"io/ioutil"
"path/filepath"

"fmt"
Expand Down Expand Up @@ -57,9 +56,9 @@ func ensureNoMetadata(opts encryptOpts, branch sops.TreeBranch) error {

func encrypt(opts encryptOpts) (encryptedFile []byte, err error) {
// Load the file
fileBytes, err := ioutil.ReadFile(opts.InputPath)
fileBytes, err := common.ReadFile(opts.InputPath)
if err != nil {
return nil, common.NewExitError(fmt.Sprintf("Error reading file: %s", err), codes.CouldNotReadInputFile)
return nil, err
}
branches, err := opts.InputStore.LoadPlainFile(fileBytes)
if err != nil {
Expand Down
24 changes: 14 additions & 10 deletions cmd/sops/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -710,17 +710,21 @@ func main() {
if c.Bool("in-place") && c.String("output") != "" {
return common.NewExitError("Error: cannot operate on both --output and --in-place", codes.ErrorConflictingParameters)
}
fileName, err := filepath.Abs(c.Args()[0])
if err != nil {
return toExitError(err)
}
if _, err := os.Stat(fileName); os.IsNotExist(err) {
if c.String("add-kms") != "" || c.String("add-pgp") != "" || c.String("add-gcp-kms") != "" || c.String("add-hc-vault-transit") != "" || c.String("add-azure-kv") != "" || c.String("add-age") != "" ||
c.String("rm-kms") != "" || c.String("rm-pgp") != "" || c.String("rm-gcp-kms") != "" || c.String("rm-hc-vault-transit") != "" || c.String("rm-azure-kv") != "" || c.String("rm-age") != "" {
return common.NewExitError("Error: cannot add or remove keys on non-existent files, use `--kms` and `--pgp` instead.", codes.CannotChangeKeysFromNonExistentFile)

fileName := c.Args()[0]
if fileName != "-" {
fileName, err := filepath.Abs(fileName)
jkroepke marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return toExitError(err)
}
if c.Bool("encrypt") || c.Bool("decrypt") || c.Bool("rotate") {
return common.NewExitError("Error: cannot operate on non-existent file", codes.NoFileSpecified)
if _, err := os.Stat(fileName); os.IsNotExist(err) {
if c.String("add-kms") != "" || c.String("add-pgp") != "" || c.String("add-gcp-kms") != "" || c.String("add-hc-vault-transit") != "" || c.String("add-azure-kv") != "" || c.String("add-age") != "" ||
c.String("rm-kms") != "" || c.String("rm-pgp") != "" || c.String("rm-gcp-kms") != "" || c.String("rm-hc-vault-transit") != "" || c.String("rm-azure-kv") != "" || c.String("rm-age") != "" {
return common.NewExitError("Error: cannot add or remove keys on non-existent files, use `--kms` and `--pgp` instead.", codes.CannotChangeKeysFromNonExistentFile)
}
if c.Bool("encrypt") || c.Bool("decrypt") || c.Bool("rotate") {
return common.NewExitError("Error: cannot operate on non-existent file", codes.NoFileSpecified)
}
}
}

Expand Down
5 changes: 2 additions & 3 deletions decrypt/decrypt.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ package decrypt // import "go.mozilla.org/sops/v3/decrypt"

import (
"fmt"
"io/ioutil"
"time"

"go.mozilla.org/sops/v3/aes"
Expand All @@ -18,9 +17,9 @@ import (
// file and returns its cleartext data in an []byte
func File(path, format string) (cleartext []byte, err error) {
// Read the file into an []byte
encryptedData, err := ioutil.ReadFile(path)
encryptedData, err := common.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("Failed to read %q: %w", path, err)
return nil, err
}

// uses same logic as cli.
Expand Down
30 changes: 30 additions & 0 deletions decrypt/example_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package decrypt

import (
"encoding/json"
"os"

"go.mozilla.org/sops/v3/logging"

Expand Down Expand Up @@ -53,3 +54,32 @@ func ExampleDecryptFile() {
}
log.Printf("%+v", cfg)
}

func StdinDecryptFile() {
var (
confPath string = "-"
cfg configuration
err error
)

oldStdin := os.Stdin
defer func() { os.Stdin = oldStdin }()

os.Stdin, _ = os.Open("./example.json")

confData, err := File(confPath, "json")
if err != nil {
log.Fatalf("cleartext configuration marshalling failed with error: %v", err)
}
err = json.Unmarshal(confData, &cfg)
if err != nil {
log.Fatalf("cleartext configuration unmarshalling failed with error: %v", err)
}
if cfg.FirstName != "John" ||
cfg.LastName != "Smith" ||
cfg.Age != 25.4 ||
cfg.PhoneNumbers[1].Number != "646 555-4567" {
log.Fatalf("configuration does not contain expected values: %+v", cfg)
}
log.Printf("%+v", cfg)
}