Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[STRMCMP-610] Fix flink config rendering #75

Merged
merged 2 commits into from
Aug 19, 2019
Merged

[STRMCMP-610] Fix flink config rendering #75

merged 2 commits into from
Aug 19, 2019

Conversation

mwylde
Copy link
Contributor

@mwylde mwylde commented Aug 15, 2019

Flink's config format looks like YAML, but is not really YAML. The parser is extremely simple, and basically operates just by splitting each line on : and treating the first part as a key and the second as the value.

Currently we are using gopkg.in/yaml.v2 to render the config. However, this breaks in the case of long strings with spaces (like

-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=39000 -XX:+UseG1GC

). In that case, yaml.v2 will convert the space into a line break (go-yaml/yaml#166), producing output like

env.java.opts.jobmanager: -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=39000
  -XX:+UseG1GC

This is valid YAML, but cannot be handled by flink's simple parser.

This PR fixes the issue by doing the printing ourselves. As flink only supports simple key-value pairs in its config, this is pretty straightforward and more correct than using a real YAML library.

This is an incompatible change for apps that have a long, space-separated config

@mwylde mwylde merged commit 8fed5e3 into master Aug 19, 2019
@mwylde mwylde deleted the micah_config_fix branch August 19, 2019 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants