CosmosDBOutput Unsupported PartitionKey value component '[]' #2112
Comments
|
@ealsur Could you help with this? |
|
@Benjiiim Are you sure the container is partitioned by This is the first time I see these kind of syntax, following the public documentation (https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-cosmosdb-v2-output?tabs=python-v2%2Cin-process%2Cnodejs-v4%2Cextensionv4&pivots=programming-language-csharp#queue-trigger-write-docs-using-iasynccollector) I would expect @kshyju Is this something particular to the Worker packages? |
|
@Benjiiim Can you also post the full Stack Trace of the exception? This message is coming from the Cosmos DB SDK (not the Extension code). The Stack Trace would normally show where is it happening. My hunch is that the SDK is attempting to extract the Partition Key Value from the documents and cannot do so, maybe because of the type of objects. Output bindings work in 2 ways:
You seem to be using the first approach but attempting to save multiple items. |
|
Thanks @ealsur Yes, I'm sure that the container is partitioned by /email. From my understanding, IAsyncCollector<T> was used in In-process model. Isolated model must use T[]. https://learn.microsoft.com/en-us/azure/azure-functions/migrate-dotnet-to-isolated-model?tabs=net8
On https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-cosmosdb-v2-output, you should use the isolated tab and not the in-process tab. I think the issue is coming from the use of dynamic but I have to use it as I don't know the properties of the Json objects. The same output was working with IAsyncCollector when my function was In-process. |
|
Stack trace: |
|
@Benjiiim Yes, this confirm the case. The problem is you are using the return pattern, which is for 1 item as per the documentation, but you are returning an object which is an array. Please use the |
|
Sorry but the doc says that IAsyncCollector should not be used in isolated mode. |
|
I have personally no knowledge how isolated mode works or why IAsyncCollector won't work there. Are you saying that when you use a Strongly Typed Array, it works, but in this example (using I wonder if the problem is the underlying Cosmos DB SDK cannot work with |
|
I don't want to be rude but why not involving someone who knows about the difference between how the CosmosDB output worked with in-process functions and how it is working now with isolated functions? |
I sadly do not know. @kshyju, any ideas who knows how Isolated Functions work in terms of Output Bindings?
The key error is this:
You mean that you have a working code that still returns a List but using different type works? Can you share which Type? |
|
I was able to output multiple documents to the CosmosDBOutput binding as follows. Perhaps it might be helpful to you. I use JsonNode since I only care about a few properties on the document. |
|
Thanks @chocvanstraw for the example, |
|
While newtonsoft is required when interacting with cosmos directly using the cosmos client I’ve found that the function output binding doesn’t seem to be as strict. I haven’t had any issues using system.text.json in this use case On Dec 8, 2023, at 5:07 PM, Matias Quaranta ***@***.***> wrote:
Thanks @chocvanstraw for the example, JsonNode works if you are using System.Text.Json (are you customizing the serializer for the extension?). The default serialization for the Cosmos DB SDK uses Newtonsoft.Json, that is why I suggested to use JObject because the OP is using dynamic data = JsonConvert.DeserializeObject(input); (Newtonsoft.Json) for the example.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
Interesting. Seems like the Isolation Mode might be using System.Text.Json to serializer/communicate between processes? @kshyju is there anyone that works on the Worker/Isolation mode area that can confirm this? |
|
Hey @kshyju @mattchenderson. I've just hit this issue as well. Is there an ETA on this one getting sorted? Or any workarounds for it? Thanks! |
|
@tim-SIOA Is the above workaround working? |
I've just changed to use 'JsonNode' and it seems to work - thanks @chocvanstraw ! |
|
@Benjiiim I was able to look into this and could reproduce the issue with the code you provided. Thank you! The issue here is that your code is mixing two serializers. Your function code uses [{"email":[],"event":[],"id":[]},{"email":[],"event":[],"category":[],"id":[]}]This will eventually cause an error while trying to persist this to cosmos db. You have 3 options to resolve the issue:
[CosmosDBOutput(databaseName: "Test", containerName: "Test",
CreateIfNotExists = true, PartitionKey = "/email",
Connection = "cosmosConnectionString")]
public Object[] Run3([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestData req)
{
var input = @"[
{
""email"": ""two@toto.com"",
""event"": ""processed""
},
{
""email"": ""hello@toto.com"",
""event"": ""deferred"",
""category"": ""facts""
}]";
var data = System.Text.Json.JsonSerializer.Deserialize<JsonNode[]>(input);
foreach (var item in data)
{
item["id"] = System.Guid.NewGuid().ToString();
}
return data;
}
var host = new HostBuilder()
.ConfigureFunctionsWorkerDefaults(builder =>
{
builder.UseNewtonsoftJson();
})
.Build();
host.Run();The
Create a POCO representing your data and use that in your function code. [CosmosDBOutput(databaseName: "Test", containerName: "Test",
CreateIfNotExists = true, PartitionKey = "/email",
Connection = "cosmosConnectionString")]
public MyType[] Run2([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestData req)
{
// Super minimal example returning an array. You may update this as needed.
return new MyType[]
{
new MyType { id = Guid.NewGuid().ToString(), Email = "two@toto.com", Category = "Fin", Event = "Order" }
};
}Hopefully you can adopt one of these solutions to resolve your issue. Let us know if you still run into problems. |
kshyju
added
Needs: Author Feedback
and removed
bug
|
Thanks @kshyju ! |
microsoft-github-policy-service
bot
added
Needs: Attention 👋
and removed
Needs: Author Feedback
labels
|
I have switched my code to use a POCO in the input and output bindings instead of JsonNode. However, the POCO uses Newtonsoft [JsonProperty] attributes on its properties, so the property names serialize properly in Cosmos. However, because the function app uses System.Text.Json as the serializer by default, the Cosmos Output binding fails. Your options are:
If you use option 2 you may not be able to share POCOs between the function app and any projects that use the actual CosmosClient as CosmosClient seems to have a dependency on Newtonsoft for serialization. |
You can if you set your CosmosClient with a custom System.Text.Json serializer. We are shipping support soon (Azure/azure-cosmos-dotnet-v3#4332) that should make it easier. |
|
@kshyju > that makes sense. No reason to include in the Cosmos binding page indeed. Thanks a lot for your time. |
In a .NET 8 Isolated function (migrating from .NET 6 In-Process), I'm trying to use CosmosDBOutput to write multiple documents without knowing their types but I'm getting the following error:
System.Private.CoreLib: Exception while executing function: Functions.Function1. Microsoft.Azure.Cosmos.Client: Unsupported PartitionKey value component '[]'. Numeric, string, bool, null, Undefined are the only supported types.
Here is a repo:
Am I doing anything wrong?
The text was updated successfully, but these errors were encountered: