[API Proposal]: Add the ability to specify line endings when serializing Json

[aodpi]

Background and motivation

Currently when serializing Json you have the option to write it Indented. But the new line character seems to be picked up based on the operating system through Environment.NewLine. This is fair but I've encountered a situation when a client wanted Line endings to be Line Feed character ('\n') instead of Carriage Return and Line Feed (\r\n). Proposal is to add the option to specify the line ending character to be used when serializing json with indentation.

When using popular package Newtonsoft.Json you have the ability to pass a StreamWriter instance which will be used to write json, with an additional ability to specify the NewLine Character when instantiating a StreamWriter

API Proposal

Extend JsonSerializerOptions and give users ability what line terminator to use when writing Indented json so we would have:

namespace System.Text.Json;

public partial class JsonSerializerOptions
{
    public bool WriteIndented { get; set; }
    public char IndentChar { get; set; }
    public int IndentSize { get; set; }
+   public string NewLine { get; set; } = Environment.NewLine; // only "\n" or "\r\n" are permitted values.

public partial struct JsonWriterOptions
{
    public bool Indented { get; set; }
    public char IndentChar { get; set; }
    public int IndentSize { get; set; }
+   public string NewLine { get; set; } = Environment.NewLine; // only "\n" or "\r\n" are permitted values.
}

By default the NewLine property will have the line terminator based on the current OS as it's treated at the moment.

API Usage

User would initialize JsonSerializerOptions by specifying the line terminator

var jsonOptions = new JsonSerializerOptions
{
    WriteIndented = true,
    NewLine = "\n"
};

User would pass these options when serializing json as follows:

using System.Text.Json;

JsonSerializer.Serialize(stream, value, jsonOptions);

The result would be a json string serialized with specified line terminator written to the specified stream.

Alternative Designs

No response

Risks

No identified risks or breaking changes.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

Currently when serializing Json you have the option to write it Indented. But the new line character seems to be picked up based on the operating system through Environment.NewLine. This is fair but I've encountered a situation when a client wanted Line endings to be Line Feed character ('\n') instead of Carriage Return and Line Feed (\r\n). Proposal is to add the option to specify the line ending character to be used when serializing json with indentation.

When using popular package Newtonsoft.Json you have the ability to pass a StreamWriter instance which will be used to write json, with an additional ability to specify the NewLine Character when instantiating a StreamWriter

API Proposal

Extend JsonSerializerOptions and give users ability what line terminator to use when writing Indented json so we would have:

using System.Text.Json;

var jsonOptions = new JsonSerializerOptions
{
    WriteIndented = true,
    NewLine = "\n"
}

By default the NewLine property will have the line terminator based on the current OS as it's treated at the moment.

API Usage

User would initialize JsonSerializerOptions by specifying the line terminator

var jsonOptions = new JsonSerializerOptions
{
    WriteIndented = true,
    NewLine = "\n"
};

User would pass these options when serializing json as follows:

using System.Text.Json;

JsonSerializer.Serialize(stream, value, jsonOptions);

The result would be a json string serialized with specified line terminator written to the specified stream.

Alternative Designs

No response

Risks

No identified risks or breaking changes.

Author:	aodpi
Assignees:	-
Labels:	api-suggestion, area-System.Text.Json, untriaged
Milestone:	-

[eiriktsarpalis]

Sounds reasonable, we could consider this. Related to #63882.

[gregsdennis]

I think having an enumeration for this instead of an open-ended string would be better.

JsonSerializationOptions.NewLine = LineTermination.CarriageReturn;

[eiriktsarpalis]

FWIW this would also need an equivalent option in JsonWriterOptions which JsonSerializerOptions forwards to.

[Ayazcan3333]

using System.Text.Json;

var jsonOptions = new JsonSerializerOptions
{
WriteIndented = true,
NewLine = "\n"
}

[Ayazcan3333]

using System.Text.Json;

var jsonOptions = new JsonSerializerOptions
{
WriteIndented = true,
NewLine = "\n"
}

[msoler8785]

Yes please add this, because I did a code review today and I saw the following and I just went home for the rest of the day:

JsonSerializer.Serialize(settings).Replace(Environment.NewLine, "\r\n\t")

[aodpi]

Sounds reasonable, we could consider this. Related to #63882.

I don't think it's related, that issue describes indentation size and indent characters. This is more about line endings in the result json string.

[rogerfar]

I ran into this issue because we're doing unit tests where we compare raw json to a resulting json:

        const String json1 =
            """
            [{
              "name": "test"
            }]
            """;

        const String json2 =
            """
            [{
              "name": "test"
            }]
            """;
			
Assert.Equal(json1, json2); // works

var json3 = JsonSerializer.Deserialize<>(JsonSerializer.Serialize(json2));
Assert.Equal(json1, resultJson); // works on Windows, but not on the linux build server

Expected: "{\r\n  "name": "test",\r\n...
Actual:   "{\n  "name": "test",\n...

[eiriktsarpalis]

@rogerfar This type of comparison would always be susceptible to whitespace issues. I would recommend creating a JsonNode instance for each JSON document and then compare using the newly added JsonNode.DeepEquals method.

[martincostello]

+1 for this feature - this week I've been working on a tool what makes targeted edits to users' JSON files to update specific nodes that meet specific criteria.

I want these changes to be as minimally invasive as possible, so as retaining the file's original file endings (assuming they're consistent). With .NET 8 today, without resorting to custom code (unless there's a better way I'm not aware of), I have to live with the compromise of the line endings being whatever the operating system uses when the edits are saved and hoping they're the same as the original.

I could then re-read/edit the file post JSON processing to fix-up the line endings again, but that's a bit bleurgh. I might do that anyway to as it's simple enough as I already detect line endings for other types of edits, but the ability to override it like you can with TextWriter.NewLine would be most welcome.

I'd be happy to take a stab at it for .NET 9 if it goes through API review.

[eiriktsarpalis]

API Proposal

namespace System.Text.Json;

public partial class JsonSerializerOptions
{
    public bool WriteIndented { get; set; }
    public char IndentChar { get; set; }
    public int IndentSize { get; set; }
+   public string NewLine { get; set; } = Environment.NewLine; // only "\n" or "\r\n" are permitted values.

public partial struct JsonWriterOptions
{
    public bool Indented { get; set; }
    public char IndentChar { get; set; }
    public int IndentSize { get; set; }
+   public string NewLine { get; set; } = Environment.NewLine; // only "\n" or "\r\n" are permitted values.
}

I've marked it ready for API review.

[baronfel]

Adding a +1 to this - this is very helpful for calculating things like OCI Digests. In many container scenarios, json is returned with both indentation and \n newlines only, and since the digest calculations use the raw json as input, STJ inserting \r\n newlines makes my calculations incorrect by construction - even if I opt into indented writing. Specifically I discovered this while implementing dotnet/sdk#39160 - adding container label metadata for the 'base' image's digest, which container tooling will use to give you good provenance tooling/tracing.

[bartonjs]

Video

• JsonSourceGenerationOptionsAttribute was missing, but should have parity with JsonSerializerOptions, so it was added.

namespace System.Text.Json
{
    public partial class JsonSerializerOptions
    {
        public string NewLine { get; set; } = Environment.NewLine; // only "\n" or "\r\n" are permitted values.
    }

    public partial struct JsonWriterOptions
    {
        public string NewLine { get; set; } = Environment.NewLine; // only "\n" or "\r\n" are permitted values.
    }
}

namespace System.Text.Json.Serialization
{
    public partial class JsonSourceGenerationOptionsAttribute
    {
        public string? NewLine { get; set; }
    }
}

[martincostello]

@eiriktsarpalis I'll look at starting on this later today.

[martincostello]

@bartonjs Looks like API review bot didn't include the timestamp in the YouTube link 😿

[bartonjs]

@martincostello The bot/tool embeds the time that the previous issue got marked in a way that says we're moving on (approved, needs more info, closed), which is usually conveniently just a few seconds before we start the issue after it. This was the first issue of the meeting, so no previous issue, so 0h0m0s is the best it can come up with.

If you have a better algorithm that doesn't require me or Immo to need to remember to do paperwork or push buttons while driving the meeting, we'd be happy to hear it 😄.

[martincostello]

Ah, I just heard the intro music and assumed it had gone awry, rather than that it was the first thing discussed 😄