Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modify efs-utils.conf in-place instead of overwriting #988

Closed
wants to merge 1 commit into from
Closed

Modify efs-utils.conf in-place instead of overwriting #988

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ require (
github.com/onsi/ginkgo v1.14.0
github.com/onsi/gomega v1.10.1
google.golang.org/grpc v1.47.0
gopkg.in/ini.v1 v1.51.0
k8s.io/api v0.22.16
k8s.io/apimachinery v0.24.3
k8s.io/client-go v1.5.2
Expand Down
5 changes: 5 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,7 @@ github.com/googleapis/gnostic v0.5.5/go.mod h1:7+EbHbldMins07ALC74bsA81Ovc97Dwqy
github.com/googleapis/go-type-adapters v1.0.0/go.mod h1:zHW75FOG2aur7gAO2B+MLby+cLsWGBF62rFAi7WjWO4=
github.com/gophercloud/gophercloud v0.1.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gopherjs/gopherjs v0.0.0-20200217142428-fce0ec30dd00 h1:l5lAOZEym3oK3SQ2HBHWsJUfbNBiTXJDeW2QDxw9AQ0=
github.com/gopherjs/gopherjs v0.0.0-20200217142428-fce0ec30dd00/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
Expand Down Expand Up @@ -413,6 +414,7 @@ github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnr
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
Expand Down Expand Up @@ -577,7 +579,9 @@ github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrf
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
github.com/smartystreets/assertions v1.1.0 h1:MkTeG1DMwsrdH7QtLXy5W+fUxWq+vmb6cLmyJ7aRtF0=
github.com/smartystreets/assertions v1.1.0/go.mod h1:tcbTF8ujkAEcZ8TElKY+i30BzYlVhC/LOxJk7iOWnoo=
github.com/smartystreets/goconvey v1.6.4 h1:fv0U8FUIMPNf1L9lnHLvLhgicrIVChEkdzIKYqbNC9s=
github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0=
Expand Down Expand Up @@ -1229,6 +1233,7 @@ gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMy
gopkg.in/gcfg.v1 v1.2.0/go.mod h1:yesOnuUOFQAhST5vPY4nbZsb/huCgGGXlipJsBn0b3o=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/ini.v1 v1.51.0 h1:AQvPpx3LzTDM0AjnIRlVFwFFGC+npRopjZxLJj6gdno=
gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k=
gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
Expand Down
115 changes: 15 additions & 100 deletions pkg/driver/efs_watch_dog.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,105 +21,12 @@ import (
"os/exec"
"path/filepath"
"sync"
"text/template"

"gopkg.in/ini.v1"
"k8s.io/klog"
)

// https://github.com/aws/efs-utils/blob/v1.30.2/dist/efs-utils.conf
const (
efsUtilsConfigTemplate = `
#
# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
#
# Licensed under the MIT License. See the LICENSE accompanying this file
# for the specific language governing permissions and limitations under
# the License.
#

[DEFAULT]
logging_level = INFO
logging_max_bytes = 1048576
logging_file_count = 10
# mode for /var/run/efs and subdirectories in octal
state_file_dir_mode = 750

[mount]
dns_name_format = {az}.{fs_id}.efs.{region}.{dns_name_suffix}
dns_name_suffix = amazonaws.com
#The region of the file system when mounting from on-premises or cross region.
{{if .Region -}}
region = {{.Region -}}
{{else -}}
#region = us-east-1
{{- end}}
stunnel_debug_enabled = false
#Uncomment the below option to save all stunnel logs for a file system to the same file
#stunnel_logs_file = /var/log/amazon/efs/{fs_id}.stunnel.log
stunnel_cafile = /etc/amazon/efs/efs-utils.crt

# Validate the certificate hostname on mount. This option is not supported by certain stunnel versions.
stunnel_check_cert_hostname = true

# Use OCSP to check certificate validity. This option is not supported by certain stunnel versions.
stunnel_check_cert_validity = false

# Define the port range that the TLS tunnel will choose from
port_range_lower_bound = 20049
port_range_upper_bound = 20449

# Optimize read_ahead_kb for Linux 5.4+
optimize_readahead = true

# By default, we enable the feature to fallback to mount with mount target ip address when dns name cannot be resolved
fall_back_to_mount_target_ip_address_enabled = true

# By default, we use IMDSv2 to get the instance metadata, set this to true if you want to disable IMDSv2 usage
disable_fetch_ec2_metadata_token = false


[mount.cn-north-1]
dns_name_suffix = amazonaws.com.cn


[mount.cn-northwest-1]
dns_name_suffix = amazonaws.com.cn


[mount.us-iso-east-1]
dns_name_suffix = c2s.ic.gov
stunnel_cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

[mount.us-isob-east-1]
dns_name_suffix = sc2s.sgov.gov
stunnel_cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

[mount-watchdog]
enabled = true
poll_interval_sec = 1
unmount_count_for_consistency = 5
unmount_grace_period_sec = 30

# Set client auth/access point certificate renewal rate. Minimum value is 1 minute.
tls_cert_renewal_interval_min = 60

# Periodically check the health of stunnel to make sure the connection is fully established
stunnel_health_check_enabled = true
stunnel_health_check_interval_min = 5
stunnel_health_check_command_timeout_sec = 30

[client-info]
source={{.EfsClientSource}}

[cloudwatch-log]
# enabled = true
log_group_name = /aws/efs/utils

# Possible values are : 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653
# Comment this config to prevent log deletion
retention_in_days = 14
`

efsUtilsConfigFileName = "efs-utils.conf"
)

Expand Down Expand Up @@ -242,16 +149,24 @@ func copyFile(src, dst string) error {
}

func (w *execWatchdog) updateConfig(efsClientSource string) error {
efsCfgTemplate := template.Must(template.New("efs-utils-config").Parse(efsUtilsConfigTemplate))
f, err := os.Create(filepath.Join(w.efsUtilsCfgPath, efsUtilsConfigFileName))
filePath := filepath.Join(w.efsUtilsCfgPath, efsUtilsConfigFileName)
cfg, err := ini.Load(filePath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't this throw an error if the file doesn't exist?

if err != nil {
return fmt.Errorf("cannot create config file %s for efs-utils. Error: %v", w.efsUtilsCfgPath, err)
return fmt.Errorf("cannot load config file %s for efs-utils. Error: %v", w.efsUtilsCfgPath, err)
}
defer f.Close()

// used on Fargate, IMDS queries suffice otherwise
region := os.Getenv("AWS_DEFAULT_REGION")
efsCfg := efsUtilsConfig{EfsClientSource: efsClientSource, Region: region}
if err = efsCfgTemplate.Execute(f, efsCfg); err != nil {
if region != "" {
cfg.Section("mount").Key("region").SetValue(region)
}

cfg.Section("client-info").Key("source").SetValue(efsClientSource)

ini.PrettyFormat = false
ini.PrettyEqual = true
ini.DefaultHeader = true
if err = cfg.SaveTo(filePath); err != nil {
return fmt.Errorf("cannot update config %s for efs-utils. Error: %v", w.efsUtilsCfgPath, err)
}
return nil
Expand Down
18 changes: 12 additions & 6 deletions pkg/driver/efs_watch_dog_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,14 @@ import (
"io/ioutil"
"os"
"path/filepath"
"strings"
"testing"
"time"
)

const (
expectedEfsUtilsConfig = `
// https://github.com/aws/efs-utils/blob/v1.30.2/dist/efs-utils.conf
efsUtilsDefaultConfig = `
#
# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
#
Expand Down Expand Up @@ -98,9 +100,6 @@ stunnel_health_check_enabled = true
stunnel_health_check_interval_min = 5
stunnel_health_check_command_timeout_sec = 30

[client-info]
source=k8s

[cloudwatch-log]
# enabled = true
log_group_name = /aws/efs/utils
Expand All @@ -118,6 +117,8 @@ func TestExecWatchdog(t *testing.T) {
defer os.RemoveAll(configDirName)
defer os.RemoveAll(staticFileDirName)

createFile(t, staticFileDirName, configFileName, efsUtilsDefaultConfig)

w := newExecWatchdog(configDirName, staticFileDirName, "sleep", "300")
if err := w.start(); err != nil {
t.Fatalf("Failed to start %v", err)
Expand Down Expand Up @@ -159,6 +160,8 @@ func TestSetupWithEmptyConfigDirectory(t *testing.T) {
fileBContent := "dummyB"
createFile(t, staticFileDirName, fileBName, fileBContent)

createFile(t, staticFileDirName, configFileName, efsUtilsDefaultConfig)

w := newExecWatchdog(configDirName, staticFileDirName, "sleep", "300").(*execWatchdog)
efsClient := "k8s"
configFilePath := filepath.Join(configDirName, configFileName)
Expand Down Expand Up @@ -192,6 +195,8 @@ func TestSetupWithNonEmptyConfigDirectory(t *testing.T) {
differentContent := "differentDummy"
createFile(t, configDirName, fileBName, differentContent)

createFile(t, configDirName, configFileName, efsUtilsDefaultConfig)

w := newExecWatchdog(configDirName, staticFileDirName, "sleep", "300").(*execWatchdog)
efsClient := "k8s"
configFilePath := filepath.Join(configDirName, configFileName)
Expand Down Expand Up @@ -260,8 +265,9 @@ func verifyConfigFile(t *testing.T, configFilePath string) {
configFileContent, err := ioutil.ReadFile(configFilePath)
checkError(t, err)
actualConfig := string(configFileContent)
if actualConfig != expectedEfsUtilsConfig {
t.Fatalf("Unexpected efs-utils config content: want %s\nactual:%s", expectedEfsUtilsConfig, actualConfig)

if !strings.Contains(actualConfig, "region =") && !strings.Contains(actualConfig, "[client-info]\nsource = k8s") {
t.Fatalf("Unexpected efs-utils config content: missing added keys. actual:%s", actualConfig)
}
}

Expand Down
6 changes: 6 additions & 0 deletions vendor/gopkg.in/ini.v1/.gitignore
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions vendor/gopkg.in/ini.v1/.travis.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading