Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: restrict secretFile path #669

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
1e7aa37
fix: configuration API returns wrong app_id
MichaelDeSteven Jun 4, 2022
5797639
Merge branch 'main' into main
seeflood Jun 7, 2022
66a2287
Merge branch 'mosn:main' into main
MichaelDeSteven Jun 10, 2022
d6b0518
fix: configuration API returns wrong store_name
MichaelDeSteven Jun 11, 2022
d94e9ce
Merge branch 'main' into main
seeflood Jun 13, 2022
5c332da
Merge branch 'main' into main
seeflood Jun 14, 2022
0b4d4a7
Merge branch 'main' into main
seeflood Jun 14, 2022
463d37d
Update etcdv3.go
MichaelDeSteven Jun 15, 2022
2e53758
Update etcdv3_test.go
MichaelDeSteven Jun 15, 2022
533e32d
Update types.go
MichaelDeSteven Jun 15, 2022
96fa1fc
Update api_configuration.go
MichaelDeSteven Jun 15, 2022
3154441
Update types.go
MichaelDeSteven Jun 15, 2022
c46a944
Update etcdv3.go
MichaelDeSteven Jun 15, 2022
05f846c
Merge branch 'main' into main
Xunzhuo Jun 17, 2022
a35bc0b
fix: configuration API returns wrong app_id
MichaelDeSteven Jun 19, 2022
307bc23
Merge branch 'main' of https://github.com/MichaelDeSteven/layotto int…
MichaelDeSteven Jun 19, 2022
71f0837
fix: apollo configuration API returns wrong store_name
MichaelDeSteven Jun 19, 2022
23eadde
delete unused const
MichaelDeSteven Jun 19, 2022
6c6ba21
delete: redundant assign line
MichaelDeSteven Jun 20, 2022
88baba2
fix: unit test of etcdv3 subscribe
MichaelDeSteven Jun 20, 2022
df3d73f
delete: subscribereq redundant storename field
MichaelDeSteven Jun 20, 2022
00bd590
Merge branch 'main' into main
MichaelDeSteven Jun 20, 2022
1c82c02
Merge branch 'main' into main
wenxuwan Jun 21, 2022
adeac11
Merge branch 'mosn:main' into main
MichaelDeSteven Jun 21, 2022
8e1f908
refactor: replace current apollo sdk with the official one
MichaelDeSteven Jun 21, 2022
f0a7b96
refactor: replace current apollo sdk with the official one
MichaelDeSteven Jun 21, 2022
44b544e
refactor: replace current apollo sdk with the official one
MichaelDeSteven Jun 21, 2022
a77ceda
Merge branch 'main' into main
MichaelDeSteven Jun 22, 2022
5b38153
refactor: replace current apollo sdk with the official one
MichaelDeSteven Jun 22, 2022
610e407
update: go 1.17 module
MichaelDeSteven Jun 22, 2022
951ee9a
refactor: replace current apollo sdk with the official one
MichaelDeSteven Jun 22, 2022
d71cc65
refactor: replace current apollo sdk with the official one
MichaelDeSteven Jun 22, 2022
f738737
refactor: replace current apollo sdk with the official one
MichaelDeSteven Jun 22, 2022
015440b
Merge branch 'mosn:main' into main
MichaelDeSteven Jun 23, 2022
c700d99
fix: restrict secretFile path
MichaelDeSteven Jun 24, 2022
c49ea61
Merge branch 'main' into fix_secret_file_restriction
MichaelDeSteven Jun 25, 2022
a908f15
test: add GetPrefixConfigFilePath unit test
MichaelDeSteven Jun 28, 2022
23f4b8c
Merge branch 'fix_secret_file_restriction' of https://github.com/Mich…
MichaelDeSteven Jun 28, 2022
1894405
Merge branch 'main' into fix_secret_file_restriction
MichaelDeSteven Jun 28, 2022
0b4f988
test: add GetPrefixConfigFilePath unit test
MichaelDeSteven Jun 28, 2022
c55d3fa
Merge branch 'main' into fix_secret_file_restriction
seeflood Jun 30, 2022
79e4284
Merge branch 'main' into fix_secret_file_restriction
seeflood Jul 1, 2022
2630016
change: rename config_secret_local_file.json to secret_local_file.json
MichaelDeSteven Jul 3, 2022
25eef9e
Merge branch 'fix_secret_file_restriction' of https://github.com/Mich…
MichaelDeSteven Jul 3, 2022
cecdc31
add: secret wrapper component
MichaelDeSteven Jul 3, 2022
79ce642
add: secret wrapper component
MichaelDeSteven Jul 3, 2022
da8a794
add: secret wrapper component
MichaelDeSteven Jul 3, 2022
adbadbd
delete: unused field
MichaelDeSteven Jul 3, 2022
965ff5b
Merge branch 'main' into fix_secret_file_restriction
Xunzhuo Jul 4, 2022
c31d897
wrap component
seeflood Jul 5, 2022
ec4afdb
Merge pull request #2 from seeflood/wrap_component
MichaelDeSteven Jul 5, 2022
8b41d2a
delete: unused component
MichaelDeSteven Jul 5, 2022
26d5d4b
fix: startup parameter and path separator
MichaelDeSteven Jul 5, 2022
70deeb7
fix: startup parameter
MichaelDeSteven Jul 5, 2022
f9e7247
Merge branch 'main' into fix_secret_file_restriction
seeflood Jul 6, 2022
033ea40
start without `-c`
seeflood Jul 7, 2022
02de6e8
Merge pull request #3 from seeflood/test_when_no_config
MichaelDeSteven Jul 7, 2022
e8e1103
Merge branch 'main' into fix_secret_file_restriction
MichaelDeSteven Jul 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions cmd/layotto/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import (

"mosn.io/layotto/pkg/grpc/default_api"
secretstores_loader "mosn.io/layotto/pkg/runtime/secretstores"
secretstores_local "mosn.io/layotto/pkg/runtime/secretstores/local"

"mosn.io/layotto/components/file/local"
"mosn.io/layotto/components/file/s3/alicloud"
Expand Down Expand Up @@ -440,12 +441,13 @@ func NewRuntimeGrpcServer(data json.RawMessage, opts ...grpc.ServerOption) (mgrp
return gcp_secretmanager.NewSecreteManager(loggerForDaprComp)
}),
secretstores_loader.NewFactory("local.file", func() secretstores.SecretStore {
return secretstore_file.NewLocalSecretStore(loggerForDaprComp)
return secretstores_local.Wrap(secretstore_file.NewLocalSecretStore(loggerForDaprComp))
}),
secretstores_loader.NewFactory("local.env", func() secretstores.SecretStore {
return secretstore_env.NewEnvSecretStore(loggerForDaprComp)
}),
))
),
)
return server, err
}

Expand Down
4 changes: 3 additions & 1 deletion cmd/layotto_multiple_api/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ import (
"strconv"
"time"

secretstores_local "mosn.io/layotto/pkg/runtime/secretstores/local"

"github.com/dapr/components-contrib/secretstores"
"github.com/dapr/components-contrib/secretstores/aws/parameterstore"
"github.com/dapr/components-contrib/secretstores/aws/secretmanager"
Expand Down Expand Up @@ -429,7 +431,7 @@ func NewRuntimeGrpcServer(data json.RawMessage, opts ...grpc.ServerOption) (mgrp
return gcp_secretmanager.NewSecreteManager(loggerForDaprComp)
}),
secretstores_loader.NewFactory("local.file", func() secretstores.SecretStore {
return secretstore_file.NewLocalSecretStore(loggerForDaprComp)
return secretstores_local.Wrap(secretstore_file.NewLocalSecretStore(loggerForDaprComp))
}),
secretstores_loader.NewFactory("local.env", func() secretstores.SecretStore {
return secretstore_env.NewEnvSecretStore(loggerForDaprComp)
Expand Down
3 changes: 2 additions & 1 deletion configs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,9 @@
},
"secret_store": {
"secret_demo": {
"type": "local.env",
"type": "local.file",
"metadata": {
"secretsFile": "secret_local_file.json"
}
}
},
Expand Down
2 changes: 1 addition & 1 deletion configs/config_standalone.json
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
"secret_demo": {
"type": "local.file",
"metadata": {
"secretsFile": "../../configs/secret/config_secret_local_file.json"
"secretsFile": "secret_local_file.json"
}
},
"secret_demo1": {
Expand Down
2 changes: 1 addition & 1 deletion docs/en/component_specs/secret/common.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Configuration examples of local file keys, local environment variables, and k8s
"secret_demo": {
"type": "local.file",
"metadata": {
"secretsFile": "../../configs/config_secret_local_file.json"
"secretsFile": "secret_local_file.json"
}
},
"secret_demo1": {
Expand Down
2 changes: 1 addition & 1 deletion docs/en/operation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ You can run Layotto using the official Docker images.Currently include:
It does not contain a `config.json` configuration file in the image, you can mount your own configuration file into the `/runtime/configs/` directory of the image. For example.

```shell
docker run -v "$(pwd)/configs/config.json:/runtime/configs/config.json" -d -p 34904:34904 --name layotto layotto/layotto start
docker run -v "$(pwd)/configs/:/runtime/configs/" -d -p 34904:34904 --name layotto layotto/layotto start
```

Of course, you can also run Layotto and other systems (such as Redis) at the same time via docker-compose. Refer to the [Quick start](en/start/state/start?id=step-1-deploy-redis-and-layotto)
Expand Down
2 changes: 1 addition & 1 deletion docs/en/start/secret/start.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# use Secret API to obtain secret
# use Secret API to query secrets
## What is Secret API
The secret API is used to obtain secret from file, env, k8s, etc

Expand Down
2 changes: 1 addition & 1 deletion docs/zh/component_specs/secret/common.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ json配置文件有如下结构:
"secret_demo": {
"type": "local.file",
"metadata": {
"secretsFile": "../../configs/config_secret_local_file.json"
"secretsFile": "secret_local_file.json"
}
},
"secret_demo1": {
Expand Down
10 changes: 9 additions & 1 deletion docs/zh/operation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,15 @@ Layotto 提供了官方 Docker 镜像,包括:
镜像内不包含 `config.json` 配置文件,您可以将自己的配置文件挂载进镜像的`/runtime/configs/config.json`目录, 然后启动镜像。例如:

```shell
docker run -v "$(pwd)/configs/config.json:/runtime/configs/config.json" -d -p 34904:34904 --name layotto layotto/layotto start
docker run -v "$(pwd)/configs/:/runtime/configs/" -d -p 34904:34904 --name layotto layotto/layotto start
```

可以运行 demo 测试效果:

```shell
cd demo/sequencer/common/
go build -o client
./client -s "sequencer_demo"
```

您也可以通过 docker-compose 同时启动 Layotto 和 其他系统(比如 Redis),参考[快速开始](zh/start/state/start?id=step-1-%e5%90%af%e5%8a%a8-redis-%e5%92%8c-layotto)
Expand Down
1 change: 1 addition & 0 deletions etc/script/test-quickstart.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ quickstarts_in_default="docs/en/start/configuration/start.md
docs/zh/start/wasm/start.md
docs/en/start/secret/start.md
docs/zh/start/secret/start.md
docs/zh/operation/README.md
"

# In advance mod, we test these docs with golang 1.17
Expand Down
65 changes: 65 additions & 0 deletions pkg/runtime/secretstores/local/wrapper.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
* Copyright 2021 Layotto Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package secret

import (
"os"
"strings"

"github.com/dapr/components-contrib/secretstores"
)

type wrapper struct {
prefix string
comp secretstores.SecretStore
}

func (w *wrapper) Init(metadata secretstores.Metadata) error {
if metadata.Properties != nil {
metadata.Properties["secretsFile"] = w.prefix + metadata.Properties["secretsFile"]
}
return w.comp.Init(metadata)
}

func (w *wrapper) GetSecret(req secretstores.GetSecretRequest) (secretstores.GetSecretResponse, error) {
return w.comp.GetSecret(req)
}

func (w *wrapper) BulkGetSecret(req secretstores.BulkGetSecretRequest) (secretstores.BulkGetSecretResponse, error) {
return w.comp.BulkGetSecret(req)
}

func Wrap(component secretstores.SecretStore) secretstores.SecretStore {
return &wrapper{
comp: component,
prefix: getPrefixConfigFilePath(),
}
}

func getPrefixConfigFilePath() string {
prefix := ""
for i, str := range os.Args {
if str == "-c" || str == "-config" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FIXME said that we should get the startup parameters in main.go and you can take

stm := stagemanager.InitStageManager(c, c.String("config"), app)
as an example.
Checking -c or -config here is not enough, since users can just ./layotto start and layotto will use the default configuration path

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FIXME said that we should get the startup parameters in main.go and you can take

stm := stagemanager.InitStageManager(c, c.String("config"), app)

as an example.
Checking -c or -config here is not enough, since users can just ./layotto start and layotto will use the default configuration path

Is there means I can get configuration path by used c.String("config") in main.go to replace if str == "-c" || str == "-config"?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! you can get the path by c.String("config") in main.go and then put it in a common place in the memory

We need to review the code and find a suitable place to store the path

Copy link
Contributor Author

@MichaelDeSteven MichaelDeSteven Jul 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how to store var in memory? Can you give me some refs?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The easiest way is to store it in the main.go as a global variable. But this "global variable" way is not "elegant".
I suggest you read the startup code and try to pass the var to the runtime server without using global variable.
If you really can't find a way, then use global variable

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichaelDeSteven Hi ,are u still working on it?
Do you need help? I can help with this

Copy link
Contributor Author

@MichaelDeSteven MichaelDeSteven Jul 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry that I cant find the method of store var in startup code. can you give me some tips?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. let me take a look

strs := strings.Split(os.Args[i+1], string(os.PathSeparator))
for _, s := range strs[:len(strs)-1] {
prefix = prefix + s + string(os.PathSeparator)
}
break
}
}
return prefix
}