OData V4 client is not working with "deep insert" aka parent / child insert

[ImGonaRot]

I know batching is working but the OData client is not working when inserting a new parent and new child at the same time.

Ex.
`Parent p = new Parent();
// set parent properties

Child c = new Child();
// set child properties
c.Parent = p;

p.Children.Add(c);
context.AddToParents(c);

context.SaveChanges(SaveChangesOptions.BatchWithSingleChangeset);
`

This does not work.

[ThomasBarnekow]

What is the status on this? I've tested this with the Microsoft.OData.Client (7.4.0-beta2) and it still doesn't work for me.

When using Postman to post a request containing parent and child objects, deeply inserting works nicely. With the OData client, the children are not included in the request.

[ImGonaRot]

This does not look to be fixed BUT here is a "partial" class of the OData client "Container" class that does the trick.

public partial class Container
{
    public Container()
        : this(new Uri(ConfigurationManager.AppSettings["ServiceUrl"]))
    {
        //this.Timeout = 120; // 2 min
        this.IgnoreResourceNotFoundException = true; // ignore 404 Not Found -- this can be used to get null from the odata service
        this.SendingRequest2 += Container_SendingRequest2; // wire into the request
        this.ReceivingResponse += Container_ReceivingResponse; // wire into the response
    }

    private void Container_ReceivingResponse(object sender, Microsoft.OData.Client.ReceivingResponseEventArgs e)
    {
    }

    private void Container_SendingRequest2(object sender, Microsoft.OData.Client.SendingRequest2EventArgs eventArgs)
    {
        // enable HTTP compression for json
        if (!eventArgs.IsBatchPart) // The request message is not HttpWebRequestMessage in batch part.
        {
            HttpWebRequest request = ((HttpWebRequestMessage)eventArgs.RequestMessage).HttpWebRequest;

            request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
        }

        // future use for data annotations
        eventArgs.RequestMessage.SetHeader("Prefer", "odata.include-annotations=\"*\"");            
    }

    /// <summary>
    /// Deep insert parent and child into OData.
    /// </summary>
    /// <param name="parentEntityPluralName"></param>
    /// <param name="entity"></param>
    public TEntity InsertEntity<TEntity>(string parentEntityPluralName, TEntity entity) where TEntity : BaseEntityType
    {
        // need to serialize the entity so that we can send parent and child together
        string serializedEntity = Newtonsoft.Json.JsonConvert.SerializeObject(entity, 
            new Newtonsoft.Json.JsonSerializerSettings() { ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore });

        // create a handler for the httpclient
        using (System.Net.Http.HttpClientHandler httpHandler = new System.Net.Http.HttpClientHandler())
        {
            // create the httpclient and add the handler
            using (System.Net.Http.HttpClient httpClient = new System.Net.Http.HttpClient(httpHandler))
            {
                // setup the headers
                httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Prefer", @"odata.include-annotations=""*""");
                httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "application/json;odata.metadata=minimal");

                // setup the content to send
                using (System.Net.Http.StringContent odataContent = new System.Net.Http.StringContent(serializedEntity))
                {
                    // setup the content type to json
                    odataContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");

                    // post the data to the odata service
                    using (System.Net.Http.HttpResponseMessage response = httpClient.PostAsync(this.BaseUri.ToString() + parentEntityPluralName, odataContent).Result)
                    {
                        // get back any errors or content
                        string content = response.Content.ReadAsStringAsync().Result;

                        // show error if service failed
                        if (response.IsSuccessStatusCode == false)
                        {
                            throw new Exception(content);
                        }

                        // try to convert the object back from the service call
                        return Newtonsoft.Json.JsonConvert.DeserializeObject<TEntity>(content);
                    }
                }
            }
        }
    }
}

Then just call it like the following
DbContext.InsertEntity(nameof(DbContext.PluralModelName), parentAndChildObject)

[ThomasBarnekow]

@ImGonaRot Thanks for your quick reply. I was thinking about hand-coding something like this myself. However, the question is still whether something that is required by the standard could make it into the library sooner or later.

I am looking at OData and this library because I wanted to stop writing and testing my own code for these things.

[ThomasBarnekow]

I've now created a simple, working example, using RestSharp (106.2.0) and Newtonsoft.Json (10.0.3) in conjunction with the entity classes created by the OData Connected Service (Version 0.3.0) extension.

Metadata (Simplified)

<?xml version="1.0" encoding="UTF-8"?>
<edmx:Edmx Version="4.0" xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx">
  <edmx:DataServices>
    <Schema Namespace="MyNamespace.Models" xmlns="http://docs.oasis-open.org/odata/ns/edm">
      <EntityType Name="Event">
        <Key>
          <PropertyRef Name="EventId" />
        </Key>
        <Property Name="EventId" Type="Edm.Int64" Nullable="false" />
        <Property Name="EventName" Type="Edm.String" Nullable="false" MaxLength="450" />
        <Property Name="CorrelationId" Type="Edm.String" Nullable="false" MaxLength="450" />
        <NavigationProperty Name="EventAttributes" 
                            Type="Collection(MyNamespace.Models.EventAttribute)" 
                            ContainsTarget="true" />
      </EntityType>
      <EntityType Name="EventAttribute">
        <Key>
          <PropertyRef Name="EventAttributeId" />
        </Key>
        <Property Name="EventAttributeId" Type="Edm.Int64" Nullable="false" />
        <Property Name="AttributeName" Type="Edm.String" Nullable="false" MaxLength="100" />
        <Property Name="AttributeValue" Type="Edm.String" Nullable="false" />
        <Property Name="EventId" Type="Edm.Int64" />
        <NavigationProperty Name="Event" Type="MyNamespace.Models.Event">
          <ReferentialConstraint Property="EventId" ReferencedProperty="EventId" />
        </NavigationProperty>
      </EntityType>
    </Schema>
    <Schema Namespace="MyNamespace" xmlns="http://docs.oasis-open.org/odata/ns/edm">
      <Action Name="PostEvent">
        <Parameter Name="deviceKey" Type="Edm.String" Unicode="false" />
        <Parameter Name="event" Type="MyNamespace.Models.Event" />
        <ReturnType Type="Edm.Boolean" Nullable="false" />
      </Action>
      <EntityContainer Name="MyContainer">
        <ActionImport Name="PostEvent" Action="MyNamespace.PostEvent" />
      </EntityContainer>
    </Schema>
  </edmx:DataServices>
</edmx:Edmx>

The real EntityContainer defines other entity sets. However, the Event entity type is contained and therefore not directly accessible from the service root. The same is true for the EventAttribute entities. Those are contained in events and it makes sense to use deep inserts to create them together with the containing event.

RestSharp-based Container

    public class RestContainer
    {
        private readonly RestClient _restClient;

        public RestContainer(Uri serviceRoot)
        {
            _restClient = new RestClient(serviceRoot);
        }

        public bool PostEvent(string deviceKey, Event @event)
        {
            var request = new RestRequest("PostEvent", Method.POST);
            request.AddJsonBody(new { deviceKey, @event });

            IRestResponse restResponse = _restClient.Execute(request);
            if (!restResponse.IsSuccessful) throw new Exception(restResponse.ErrorMessage);

            return JsonConvert
                .DeserializeObject<ODataResponse<bool>>(restResponse.Content)
                .Value;
        }
    }

    public class ODataResponse<T>
    {
        [JsonProperty("@odata.context")]
        public string ODataContext { get; set; }

        public T Value { get; set; }
    }

I've omitted the PostEventAsync method for brevity.

Using the RestSharp-based Container

    public static class Program
    {
        public static void Main()
        {
            // Create Event entity with contained EventAttribute entities.
            var @event = new Event
            {
                EventName = "[Event Name]",
                CorrelationId = "[Correlation ID]"
            };

            @event.EventAttributes.Add(new EventAttribute
            {
                AttributeName = "[Name 1]",
                AttributeValue = "[Value 1]"
            });

            @event.EventAttributes.Add(new EventAttribute
            {
                AttributeName = "[Name 2]",
                AttributeValue = "[Value 2]"
            });

            // Create RestSharp-based container.
            var serviceRoot = new Uri("[Enter your service root URI]");
            var restContainer = new RestContainer(serviceRoot);

            // Execute the PostEvent OData action.
            bool value = restContainer.PostEvent("SomeDeviceKey", @event);
        }
    }

Thoughts

It was very easy to create the RestSharp-based container and make it work right away. The event and its two attributes are inserted in the database as expected.

Therefore, the question is how hard it would be to make the OData Client do the same thing. But maybe I am also not using the generated classes correctly. My problem there is that the documentation only covers the most basic examples and it is somewhat hard to figure out more advanced usage scenarios. Whatever I tried did not work.

[jangix]

This does not look to be fixed BUT here is a "partial" class of the OData client "Container" class that does the trick.

public partial class Container
{
    public Container()
        : this(new Uri(ConfigurationManager.AppSettings["ServiceUrl"]))
    {
        //this.Timeout = 120; // 2 min
        this.IgnoreResourceNotFoundException = true; // ignore 404 Not Found -- this can be used to get null from the odata service
        this.SendingRequest2 += Container_SendingRequest2; // wire into the request
        this.ReceivingResponse += Container_ReceivingResponse; // wire into the response
    }

    private void Container_ReceivingResponse(object sender, Microsoft.OData.Client.ReceivingResponseEventArgs e)
    {
    }

    private void Container_SendingRequest2(object sender, Microsoft.OData.Client.SendingRequest2EventArgs eventArgs)
    {
        // enable HTTP compression for json
        if (!eventArgs.IsBatchPart) // The request message is not HttpWebRequestMessage in batch part.
        {
            HttpWebRequest request = ((HttpWebRequestMessage)eventArgs.RequestMessage).HttpWebRequest;

            request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
        }

        // future use for data annotations
        eventArgs.RequestMessage.SetHeader("Prefer", "odata.include-annotations=\"*\"");            
    }

    /// <summary>
    /// Deep insert parent and child into OData.
    /// </summary>
    /// <param name="parentEntityPluralName"></param>
    /// <param name="entity"></param>
    public TEntity InsertEntity<TEntity>(string parentEntityPluralName, TEntity entity) where TEntity : BaseEntityType
    {
        // need to serialize the entity so that we can send parent and child together
        string serializedEntity = Newtonsoft.Json.JsonConvert.SerializeObject(entity, 
            new Newtonsoft.Json.JsonSerializerSettings() { ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore });

        // create a handler for the httpclient
        using (System.Net.Http.HttpClientHandler httpHandler = new System.Net.Http.HttpClientHandler())
        {
            // create the httpclient and add the handler
            using (System.Net.Http.HttpClient httpClient = new System.Net.Http.HttpClient(httpHandler))
            {
                // setup the headers
                httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Prefer", @"odata.include-annotations=""*""");
                httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "application/json;odata.metadata=minimal");

                // setup the content to send
                using (System.Net.Http.StringContent odataContent = new System.Net.Http.StringContent(serializedEntity))
                {
                    // setup the content type to json
                    odataContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");

                    // post the data to the odata service
                    using (System.Net.Http.HttpResponseMessage response = httpClient.PostAsync(this.BaseUri.ToString() + parentEntityPluralName, odataContent).Result)
                    {
                        // get back any errors or content
                        string content = response.Content.ReadAsStringAsync().Result;

                        // show error if service failed
                        if (response.IsSuccessStatusCode == false)
                        {
                            throw new Exception(content);
                        }

                        // try to convert the object back from the service call
                        return Newtonsoft.Json.JsonConvert.DeserializeObject<TEntity>(content);
                    }
                }
            }
        }
    }
}

Then just call it like the following
DbContext.InsertEntity(nameof(DbContext.PluralModelName), parentAndChildObject)

Json serialization doesn't work fine with Edm types.

For example, Edm.Date is serialized:
"StuffDate":{ "Year":2019, "Month":10, "Day":25 }
and it's not like 'Common Verbose JSON Serialization Rules' :
"StuffDate": "2019-10-25"

Does anyone know a way to serialize the entity with EDM types correctly?
It would be very important for me to find a solution.
Unfortunately as the DeepInsert is not implemented by OData.Client.DataServiceContext and since the server with which I have to communicate does not support Batch requests, serializing the HttpRequest is my only possibility to implement the Deep Insert.

[jangix]

I share my serializer settings fixed for EDM types and other fix:

`
using Microsoft.OData.Edm;
using Newtonsoft.Json;

public static string ToJson (this BaseEntityType entity) {
var nameResolver = new DefaultContractResolver { NamingStrategy = new CamelCaseNamingStrategy () };
string serializedEntity = JsonConvert.SerializeObject (entity, new JsonSerializerSettings () {
ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
DefaultValueHandling = DefaultValueHandling.Ignore,
NullValueHandling = NullValueHandling.Ignore,
Formatting = Formatting.Indented,
ContractResolver = nameResolver,
Converters = JsonConvertersCustom.EdmTypes,
});
return serializedEntity;
}

public static class JsonConvertersCustom {
public static JsonConverter[] EdmTypes { get; set; } = {
//Register here your custom json converter
new EdmDateConverter (),
new EdmDateNullableConverter (),
};
}

public class EdmDateConverter : JsonConverter {
public override Date ReadJson (JsonReader reader, Type objectType, Date existingValue, bool hasExistingValue, JsonSerializer serializer)
=> Date.Parse ((string) reader.Value);

public override void WriteJson (JsonWriter writer, Date value, JsonSerializer serializer)
=> writer.WriteValue (value.ToString ());
}

public class EdmDateNullableConverter : JsonConverter<Date?> {
public override Date? ReadJson (JsonReader reader, Type objectType, Date? existingValue, bool hasExistingValue, JsonSerializer serializer)
=> reader.Value as string != null ? Date.Parse ((string) reader.Value) : (Date?) null;

public override void WriteJson (JsonWriter writer, Date? value, JsonSerializer serializer)
=> writer.WriteValue (value.HasValue ? value.Value.ToString () : null);
}
`

[ghost]

I am following the issue here. Is there any update on this?

[marabooy]

@ThomasBarnekow @mukesh-shobhit Looking into the issue.

@ImGonaRot Should the right semantics for deep insert be

Parent p = new Parent();
// set parent properties

Child c = new Child();
// set child properties
 //c.Parent = p; Omit this as the creation of the parent should preserve the relationship from parent->child which should create the inverse relationship on the database

p.Children.Add(c);
context.AddToParents(c);

// context.SaveChanges(); in the odata spec this should work as well as the addition should be done in one transaction
context.SaveChanges(SaveChangesOptions.BatchWithSingleChangeset);

So in flight the JSON would look like

{
 //parent properties
"children":[ { //child one properties} .....{// child n properties}]
}

[mmichtch]

This is exactly what we expecting, but I am following the pattern you suggesting and deep insert does not work. I have:

 <PackageReference Include="Microsoft.OData.Client" Version="7.6.4" />
 <PackageReference Include="Microsoft.OData.Core" Version="7.6.4" />
 <PackageReference Include="Microsoft.OData.Edm" Version="7.6.4" />

and client code generated with:
"OData Connected Service" - Version 0.10.0
With the postman and complete object graph server works as expected (no surprise), it seems that client does not send children of the object.

[dr-consit]

It's been several years now. Is this completely dead in the water?

[fededim]

@marabooy I am experiencing the same issue with the latest Microsoft.AspNetCore.OData package (7.5.6) and ODataConnectedService (0.12.1). It has passed a year since your last reply, are there any updates on this ? Just to understand if this issue will be solved or not, because this jeopardizes the usability of OData since in almost every project you need to insert/update entities with related ones.

[marabooy]

@fededim The issue is currently being worked on so it can be supported across our tooling.

[fededim]

@marabooy Ok, is it possible to have a timeframe ? Just to understand if its resolution goes on for months or on another year.

[marabooy]

@fededim I really can't give you a timeline for the whole delivery but you may see some of the pieces ship with some of our expected releases sometime through the year which will be linked to this work item when they are out of draft status :).

[fededim]

@marabooy I'll hope for the best. Just a note: besides "deep insert" you should also address the "deep update" functionality of OData since they are both important for every project.

[ImGonaRot]

@marabooy As a suggestion, my thought is that there be a SaveChangesOptions.IncludeChildren when calling SaveChanges.
Else all current users calling SaveChanges might have issues with it "auto" saving children when they didn't expect that to happen.

// context.SaveChanges(); in the odata spec this should work as well as the addition should be done in one transaction
context.SaveChanges(SaveChangesOptions.IncludeChildren);

[alkubo]

Hi guys
Any progress on this issue?

[mbauerdev]

Hi guys
I just found the following Pull Request:
Support deep updates #1585

Does anyone know if the change just addresses updates? Any possibility that it works for inserts as well?

[adrien-constant]

This feature would be very useful. When will it be included ? Thanks

[drventure]

Any word on Deep updates and inserts? Looks like there was a PR for Updates at any rate....

[KenitoInc]

Hey everyone,
Deep insert/update for OData Client v4 is a feature currently in design stage, we should be having a beta release sometime in the future.

[rblanca]

any news?. Thanks

[ElizabethOkerio]

There is on-going work to support this and that is being tracked.

[dr-consit]

@ElizabethOkerio Cool, can you link to that? Or is there some other way to subscribe to something, so we can know, when this feature is available?

[ElizabethOkerio]

Here are the PRs on the on-going work.
#2561
#2627
#2653