Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mod: use upstream go-yaml #26

Merged
merged 4 commits into from
Feb 23, 2024
Merged

mod: use upstream go-yaml #26

merged 4 commits into from
Feb 23, 2024

Conversation

tormath1
Copy link
Contributor

@tormath1 tormath1 commented Jan 30, 2024

github.com/coreos/yaml is not even maintained, let's switch to the upstream.

This drops:

  • conversion from hyphen to underscore (e.g write-files-> write_files)
  • usage of wrong values for a given key (e.g reboot-strategy: [1, 2, 3])

Related to: flatcar/Flatcar#1271

In place of 'coreos/yaml'

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
@tormath1 tormath1 self-assigned this Jan 30, 2024
@tormath1 tormath1 changed the title [wip] mod: use upstream go-yaml mod: use upstream go-yaml Feb 22, 2024
@tormath1 tormath1 marked this pull request as ready for review February 22, 2024 13:06
@tormath1 tormath1 requested a review from a team February 22, 2024 13:06
Copy link
Member

@krnowak krnowak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The normalization is the icky part. I'm not sure if this won't cause some nasty bugs.

config/config.go Outdated
Comment on lines 74 to 76
// Normalize transform the "-" in "_" for the keys
// in the YAML configuration.
// e.g reboot-strategy -> reboot_strategy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Normalize transform the "-" in "_" for the keys
// in the YAML configuration.
// e.g reboot-strategy -> reboot_strategy
// Normalize transforms dashes into underlines for the keys
// in the YAML configuration.
// e.g reboot-strategy -> reboot_strategy

config/config.go Outdated
// Normalize transform the "-" in "_" for the keys
// in the YAML configuration.
// e.g reboot-strategy -> reboot_strategy
func Normalize(r io.Reader) string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, this is a rather risky function. Will it mess some inline bash script like:

- some_inline_stuff: |
  echo 'foo-bar: stuff'

https://play.golang.com/p/u6ogWtIHI9Q

This kind of transform is probably done safe only by actually parsing YAML.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to agree here. If we mutate the source yaml, we're asking for trouble. It would be nice if the yaml package supported multiple tags for the same field, but sadly it does not.

Copy link
Contributor Author

@tormath1 tormath1 Feb 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with all of you. I tried to provide a "best effort" approach to continue maintaining the current behavior but honestly I'd be in favor of dropping this "compatibility" (e.g reboot-strategy == reboot_strategy). I understand it was quite convenient back in the days, but today it's a nonsense to me: with a clear spec, such a behavior would not be needed. I never seen Kubernetes YAML manifest or other well known YAML configuration doing such a thing.

This best effort is just to provide backward compatibility with folks still using coreos-cloudinit with "bad" keys.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I'd drop this backward compatibility then, because continuing to support invalid keys adds too much to maintenance overhead. Otherwise we risk mangling unrelated stuff in config file. If you decide to drop it then we also will need to announce it. I think a changelog entry when bumping the commit ID in scripts repo would be sufficient.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Then I dropped the compatibility.

Comment on lines 97 to 120
v := v.(map[interface{}]interface{})
for k, cv := range v {
cn := node{name: fmt.Sprintf("%s", k)}
c, ok := findKey(cn.name, c)
if ok {
cn.line = c.lineNumber
cpv := v
v, ok := cpv.(map[interface{}]interface{})
if ok {
for k, cv := range v {
cn := node{name: fmt.Sprintf("%s", k)}
c, ok := findKey(cn.name, c)
if ok {
cn.line = c.lineNumber
}
toNode(cv, c, &cn)
n.children = append(n.children, cn)
}
toNode(cv, c, &cn)
n.children = append(n.children, cn)
} else {
sv := cpv.(map[string]interface{})
for k, cv := range sv {
cn := node{name: fmt.Sprintf("%s", k)}
c, ok := findKey(cn.name, c)
if ok {
cn.line = c.lineNumber
}
toNode(cv, c, &cn)
n.children = append(n.children, cn)
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably could avoid this duplication by putting the common code into an inline function (excuse my rusty go, haven't typed that in a long time):

handleKV := func(k,v interface{}) {
	cn := node{name: fmt.Sprintf("%s", k)}
	c, ok := findKey(cn.name, c)
	if ok {
		cn.line = c.lineNumber
	}
	toNode(cv, c, &cn)
	n.children = append(n.children, cn)
}
cpv := v
v, ok := cpv.(map[interface{}]interface{})
if ok {
	for k, cv := range v {
		handleKV(k, cv)
	}
} else {
	sv := cpv.(map[string]interface{})
	for k, cv := range sv {
		handleKV(k, cv)
	}
}

Probably could be made even shorter by using Go's generics, but I have no experience with those.

yaml.UnmarshalMappingKeyTransform = func(nameIn string) (nameOut string) {
return nameIn
}
cfg = []byte(config.Normalize(bytes.NewReader(cfg)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, this normalizes the yaml config now, whereas the old code had a no-op transform at this point. Not sure if it matters, though.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
@tormath1 tormath1 force-pushed the tormath1/yaml branch 2 times, most recently from dbc3713 to 88f93b2 Compare February 23, 2024 09:52
some of these tests were relying on the fact that the unmarshal of the
configuration does not panic but it's against yaml lib definition: if
the lib can not unmarshal, it will just panic.

we also drop tests regarding hyphen -> underscore conversion: this support is removed.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
@tormath1 tormath1 merged commit a7bc5f0 into flatcar-master Feb 23, 2024
1 check passed
@tormath1 tormath1 deleted the tormath1/yaml branch February 23, 2024 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants