Marshalling long, single-line strings results in (correct) line breaks

[tyangliu]

When long, single-line strings (i.e. does not contain "\n") are marshalled, the yaml emitter seems to force line breaks at 80 characters, but does so without prefixing the string with the multi-line notation (e.g. ">" or "|"). This results in unusable yaml output like in the example below.

Example:

data := yaml.MapSlice{{"test", "abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd"}}

res, _ := yaml.Marshal(data)
fmt.Println(string(res))

Output:

test: abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd
  abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd

As a workaround, simply appending "\n" to the end of a string that's known to be lengthy works fine, although not ideal since the yaml output now exceeds the 80 char width. Mutiline ">" prefixed outputs would be preferable.

Output with trailing \n:

test: |
  abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd aabdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd

Appreciate if someone could confirm whether this is an issue or just incorrect usage. If a fix is needed, I'd be happy to work on a PR. Thanks!

[deitch]

Been banging my head on this for the last while. Is there any way to force it not to break? It should be perfectly legitimate to say, "give me a really long line".

[wayt]

I'm facing the same issue. A tag would be very nice.

[rogpeppe]

@tyangliu

This results in unusable yaml output like in the example below.

What do you mean by "unusable yaml output" there? By my reading of the specification, that YAML looks valid. See section 8.1.3 in http://yaml.org/spec/1.2/spec.html, which says "Folding allows long lines to be broken anywhere a single space character separates two non-space characters.". That means that the example is perfectly valid YAML and the map value holds exactly the string you produced.

The Go YAML marshaler uses heuristics to try to produce decent looking output for any possible string. There are so many possible output style choices in YAML, that I'm not sure it's a good idea to start down the slippery slope of allowing string style choices to be specified with struct tags.

That said, suggestions are welcome - any such struct tag would need to be applicable to all possible strings.

[wayt]

I understand this is valid according to the specification, but not compatible in all implementations (Perl for instance).
A fallback option would be appreciated.

[rogpeppe]

I understand this is valid according to the specification, but not compatible in all implementations

How hard should go-yaml try to produce YAML that is compatible with other broken YAML implementations? As the YAML spec is insanely complex, I wouldn't be surprised if every single implementation was broken in some way.

If we decide that we want to be compatible with perl's broken implementation, the right answer I think would be to avoid breaking lines like this at all. Providing a tag so that the behaviour can be changed in individual cases doesn't seem like the right fix (what if some other field happens to have a long value with spaces in?).

tbh, if you're marshalling YAML for data interchange, I'd suggest not using YAML but using JSON instead - it's simple enough that most implrmentations get it right, and it's also compatible (mostly) with YAML.

[alexandrevicenzi]

My guess to solve this problem is to expose YAML Encoder and Decoder class. Go encoding/json exposes its encoder and decoder, this allows setting custom parameters.

In apic.go there's a function called yaml_emitter_set_width with allows to disable long line breaks if set to -1, but this is an internal function that is never called.

Something like this code would be nice:

e := yaml.NewEncoder(wr)
e.SetLineWidth(-1)
err := e.Encode(myObj)

Using a tag would be hard I think, there's no easy way to pass the tag info to the yaml_emitter_t.

[rogpeppe]

@alexandrevicenzi Now that we have an Encoder type, I think having a SetLineWidth method on it seems very reasonable. I think I'd treat any line width <= 0 as disabling line breaks. Happy to accept a PR for that.

[alexandrevicenzi]

@rogpeppe this Encoder type is only available in devel branch? If so, I'll send a PR.

[vivook]

Is it any closer to fix? This is breaking my parsing yaml -> go template by marshal/unmarshal yaml because it puts in new lines breaking go template

[rohith-vallabhaneni]

By any chance can I know if this feature is implemented already, I'm getting |+ at the start of the yaml output which is effecting my parsing. I could ignore it my perforning some data maipulations but i'm writing many yaml files to stdout out of which only one file is throwing this issue :(

Could anyone suggest me any other work around, thanks!

Please find the block of output that is effecting me

metrics:

• type: Resource
  object: null
  pods: null
  resource:
  name: cpu
  targetaverageutilization: 70
  targetaveragevalue: null
  --- |+
  apiVersion: v1

[alexandrevicenzi]

@rohith-vallabhaneni there's an open PR (#455), you can apply this patch locally and wait until is merged and released.

[laverya]

@rohith-vallabhaneni that patch isn't quite as simple as 'switch to the patch and everything works', I'm afraid - you'll need to edit your calling code to set the desired line length too, as in this comment: #166 (comment)

[bt]

Any updates on this issue? It's breaking my application config files as I have some strings that are long and it's being spanned across 2 lines.

[kivutar]

I think that line length should default to -1. This value of 80 is arbitrary. This behavior is surprising.

[laverya]

Honestly, that's probably just as good of a solution as adding a function to set line length for my purposes. Better backwards compatibility too

[ChanderG]

Facing this issue as well. Using the workaround recommended by tyangliu for now.

Edit: Turns out that adding newlines to fields breaks other things (k8s in our case). Using a locally patched version of yaml with yaml_emitter_set_width -1 in the newEncoder function itself.

[niemeyer]

@laverya That's what we'll do indeed. 👍

[laverya]

@niemeyer would you mind if I opened a PR for that? (setting default line length to -1, I mean)

[matteomazza91]

I think this could be closed now.
for whoever is still looking into this today, v2 has now the ability to disable line wrapping. See additional details in the doc: https://pkg.go.dev/gopkg.in/yaml.v2#FutureLineWrap