Add support for $count in navigation properties #2051

KenitoInc · 2021-04-13T13:09:01Z

Issues

This pull request fixes issue #xxx.

Description

Add support for $count in Navigation properties.

expand=$navProp/$count
expand=$navProp/$count($filter=prop gt abc)
expand=$navProp/$count($search=prop)

We are adding a new PathSelectItem named ExpandedCountSelectItem which inherits from ExpandedReferenceSelectItem.
The ExpandedCountSelectItem only supports 2 query options, $filter and $search. See spec http://docs.oasis-open.org/odata/odata/v4.01/cs01/part1-protocol/odata-v4.01-cs01-part1-protocol.html#sec_RequestingtheNumberofItemsinaCollect

Checklist (Uncheck if it is not completed)

Test cases added
Build and test with one-click build and test script passed

Additional work necessary

If documentation update is needed, please add "Docs Needed" label to the issue and provide details about the required document change in the issue.

xuzhg · 2021-04-14T18:05:39Z

src/Microsoft.OData.Core/UriParser/Parsers/SelectExpandTermParser.cs

+
+            if (this.lexer.CurrentToken.Text == UriQueryConstants.CountSegment && isSelect)
+            {
+                // $count in only allowed in $expand. e.g $expand=NavProperty/$count


src/Microsoft.OData.Core/UriParser/Parsers/SelectExpandTermParser.cs

xuzhg · 2021-04-14T18:14:48Z

src/Microsoft.OData.Core/UriParser/SemanticAst/ExpandedCountSelectItem.cs

+    /// <summary>
+    /// This represents one level of expansion for a particular expansion tree.
+    /// </summary>
+    public sealed class ExpandedCountSelectItem : ExpandedReferenceSelectItem


Do you think it's better to derive from ExpandedReferenceSelectItem?

Or shall we have a base NavigationSelectItem, and others derived from it?

The semantics are the same though.
Also, ExpandedNavigationSelectItem inherits from the ExpandedReferenceSelectItem

xuzhg · 2021-04-14T18:15:22Z

src/Microsoft.OData.Core/UriParser/SemanticAst/ExpandedCountSelectItem.cs

+        /// <param name="filterOption">A filter clause for this expand (can be null)</param>
+        /// <param name="searchOption">A search clause for this expand (can be null)</param>
+        /// <exception cref="System.ArgumentNullException">Throws if input pathToNavigationProperty is null.</exception>
+        public ExpandedCountSelectItem(ODataExpandPath pathToNavigationProperty, IEdmNavigationSource navigationSource, FilterClause filterOption, SearchClause searchOption)


Only $filter and $search are allowed within nav/$count?

Yes: From the protocol http://docs.oasis-open.org/odata/odata/v4.01/cs01/part1-protocol/odata-v4.01-cs01-part1-protocol.html#sec_RequestingtheNumberofItemsinaCollect

On success, the response body MUST contain the exact count of items matching the request after applying any $filter or $search system query options, formatted as a simple primitive integer value with media type text/plain. Clients SHOULD NOT combine the system query options $top, $skip, $orderby, $expand, and $format with the path suffix /$count. The result of such a request is undefined.

xuzhg · 2021-04-14T18:17:24Z

Looks good to me.

gathogojr · 2021-04-19T06:25:16Z

@KenitoInc I cloned your branch and performed some tests based on the sample scenarios you provided in the PR description.
Consider this result of expanding Products a navigation property in Category

http://service/Categories?$expand=Products

Result:

{
    "@odata.context": "http://service/$metadata#Categories(Products())",
    "value": [
        {
            "Id": "PG1",
            "Name": "Food",
            "Products": [
                {
                    "Id": "P1",
                    "CategoryId": "PG1",
                    "Name": "Sugar",
                    "Color": "White",
                    "TaxRate": 0.06
                },
                {
                    "Id": "P2",
                    "CategoryId": "PG1",
                    "Name": "Coffee",
                    "Color": "Brown",
                    "TaxRate": 0.06
                }
            ]
        },
        {
            "Id": "PG2",
            "Name": "Non-Food",
            "Products": [
                {
                    "Id": "P3",
                    "CategoryId": "PG2",
                    "Name": "Paper",
                    "Color": "White",
                    "TaxRate": 0.14
                },
                {
                    "Id": "P4",
                    "CategoryId": "PG2",
                    "Name": "Pencil",
                    "Color": "Black",
                    "TaxRate": 0.14
                }
            ]
        }
    ]
}

Test 1:

http://service/Categories?$expand=Products/$count

Result:

{
    "@odata.context": "http://service/$metadata#Categories(Products/$ref())",
    "value": [
        {
            "Id": "PG1",
            "Name": "Food",
            "Products": [
                {
                    "@odata.id": "Products('P1')"
                },
                {
                    "@odata.id": "Products('P2')"
                }
            ]
        },
        {
            "Id": "PG2",
            "Name": "Non-Food",
            "Products": [
                {
                    "@odata.id": "Products('P3')"
                },
                {
                    "@odata.id": "Products('P4')"
                }
            ]
        }
    ]
}

Test 2:

http://service/Categories?$expand=Products/$count($filter=Color eq 'White')

Result:

{
    "@odata.context": "http://service/$metadata#Categories(Products/$ref())",
    "value": [
        {
            "Id": "PG1",
            "Name": "Food",
            "Products": [
                {
                    "@odata.id": "Products('P1')"
                }
            ]
        },
        {
            "Id": "PG2",
            "Name": "Non-Food",
            "Products": [
                {
                    "@odata.id": "Products('P3')"
                }
            ]
        }
    ]
}

Test 3:

http://service/Categories?$expand=Products/$count($search=Black)

Result:

{
    "@odata.context": "http://service/$metadata#Categories(Products/$ref())",
    "value": [
        {
            "Id": "PG1",
            "Name": "Food",
            "Products": [
                {
                    "@odata.id": "Products('P1')"
                },
                {
                    "@odata.id": "Products('P2')"
                }
            ]
        },
        {
            "Id": "PG2",
            "Name": "Non-Food",
            "Products": [
                {
                    "@odata.id": "Products('P3')"
                },
                {
                    "@odata.id": "Products('P4')"
                }
            ]
        }
    ]
}

I don't know if I'm doing something wrong. Can you verify whether you get the expected results against a sample service?

KenitoInc · 2021-04-19T08:05:29Z

@KenitoInc I cloned your branch and performed some tests based on the sample scenarios you provided in the PR description.
Consider this result of expanding Products a navigation property in Category

http://service/Categories?$expand=Products

Result:

{
    "@odata.context": "http://service/$metadata#Categories(Products())",
    "value": [
        {
            "Id": "PG1",
            "Name": "Food",
            "Products": [
                {
                    "Id": "P1",
                    "CategoryId": "PG1",
                    "Name": "Sugar",
                    "Color": "White",
                    "TaxRate": 0.06
                },
                {
                    "Id": "P2",
                    "CategoryId": "PG1",
                    "Name": "Coffee",
                    "Color": "Brown",
                    "TaxRate": 0.06
                }
            ]
        },
        {
            "Id": "PG2",
            "Name": "Non-Food",
            "Products": [
                {
                    "Id": "P3",
                    "CategoryId": "PG2",
                    "Name": "Paper",
                    "Color": "White",
                    "TaxRate": 0.14
                },
                {
                    "Id": "P4",
                    "CategoryId": "PG2",
                    "Name": "Pencil",
                    "Color": "Black",
                    "TaxRate": 0.14
                }
            ]
        }
    ]
}

Test 1:

http://service/Categories?$expand=Products/$count

Result:

{
    "@odata.context": "http://service/$metadata#Categories(Products/$ref())",
    "value": [
        {
            "Id": "PG1",
            "Name": "Food",
            "Products": [
                {
                    "@odata.id": "Products('P1')"
                },
                {
                    "@odata.id": "Products('P2')"
                }
            ]
        },
        {
            "Id": "PG2",
            "Name": "Non-Food",
            "Products": [
                {
                    "@odata.id": "Products('P3')"
                },
                {
                    "@odata.id": "Products('P4')"
                }
            ]
        }
    ]
}

Test 2:

http://service/Categories?$expand=Products/$count($filter=Color eq 'White')

Result:

{
    "@odata.context": "http://service/$metadata#Categories(Products/$ref())",
    "value": [
        {
            "Id": "PG1",
            "Name": "Food",
            "Products": [
                {
                    "@odata.id": "Products('P1')"
                }
            ]
        },
        {
            "Id": "PG2",
            "Name": "Non-Food",
            "Products": [
                {
                    "@odata.id": "Products('P3')"
                }
            ]
        }
    ]
}

Test 3:

http://service/Categories?$expand=Products/$count($search=Black)

Result:

{
    "@odata.context": "http://service/$metadata#Categories(Products/$ref())",
    "value": [
        {
            "Id": "PG1",
            "Name": "Food",
            "Products": [
                {
                    "@odata.id": "Products('P1')"
                },
                {
                    "@odata.id": "Products('P2')"
                }
            ]
        },
        {
            "Id": "PG2",
            "Name": "Non-Food",
            "Products": [
                {
                    "@odata.id": "Products('P3')"
                },
                {
                    "@odata.id": "Products('P4')"
                }
            ]
        }
    ]
}

I don't know if I'm doing something wrong. Can you verify whether you get the expected results against a sample service?

@gathogojr I have added this feature to OData Core. We will support it in WebApi later.

gathogojr · 2021-04-19T08:07:58Z

@gathogojr I have added this feature to OData Core. We will support it in WebApi later.

argh... I jumped the gun without even realizing it!

pull-request-quantifier-deprecated · 2021-04-19T09:03:31Z

This PR has 75 quantified lines of changes. In general, a change size of upto 200 lines is ideal for the best PR experience!

Quantification details

Label      : Small
Size       : +73 -2
Percentile : 30%

Total files changed: 7

Change summary by file extension:
.cs : +71 -2
.txt : +2 -0

Change counts above are quantified counts, based on the PullRequestQuantifier customizations.

Why proper sizing of changes matters

Optimal pull request sizes drive a better predictable PR flow as they strike a
balance between between PR complexity and PR review overhead. PRs within the
optimal size (typical small, or medium sized PRs) mean:

Fast and predictable releases to production:
- Optimal size changes are more likely to be reviewed faster with fewer
  iterations.
- Similarity in low PR complexity drives similar review times.
Review quality is likely higher as complexity is lower:
- Bugs are more likely to be detected.
- Code inconsistencies are more likely to be detetcted.
Knowledge sharing is improved within the participants:
- Small portions can be assimilated better.
Better engineering practices are exercised:
- Solving big problems by dividing them in well contained, smaller problems.
- Exercising separation of concerns within the code changes.

What can I do to optimize my changes

Use the PullRequestQuantifier to quantify your PR accurately
- Create a context profile for your repo using the context generator
- Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out the Excluded section from your prquantifier.yaml context profile.
- Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in your prquantifier.yaml context profile.
- Only use the labels that matter to you, see context specification to customize your prquantifier.yaml context profile.
Change your engineering behaviors
- For PRs that fall outside of the desired spectrum, review the details and check if:
  - Your PR could be split in smaller, self-contained PRs instead
  - Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).

How to interpret the change counts in git diff output

One line was added: +1 -0
One line was deleted: +0 -1
One line was modified: +1 -1 (git diff doesn't know about modified, it will
interpret that line like one addition plus one deletion)
Change percentiles: Change characteristics (addition, deletion, modification)
of this PR in relation to all other PRs within the repository.

Was this comment helpful? 👍 :ok_hand: :thumbsdown: (Email)
Customize PullRequestQuantifier for this repository.

habbes · 2021-04-20T09:05:10Z

src/Microsoft.OData.Core/UriParser/Parsers/SelectExpandTermParser.cs

+            if (this.lexer.CurrentToken.Text.StartsWith("$", StringComparison.Ordinal)
+                && (!allowRef || this.lexer.CurrentToken.Text != UriQueryConstants.RefSegment)
+                && this.lexer.CurrentToken.Text != UriQueryConstants.CountSegment)


Could you explain the logic here?

If Path segment starts with $, it should only have $count or $ref when allowRef is true
e.g
$expand=NavProp/$ref
$expand=NavProp/$count

habbes · 2021-04-20T09:06:53Z

src/Microsoft.OData.Core/UriParser/SemanticAst/ExpandedCountSelectItem.cs

+        /// <summary>
+        /// Create an Expand item using a nav prop, its entity set and a SelectExpandClause
+        /// </summary>
+        /// <param name="pathToNavigationProperty">the path to the navigation property for this expand item, including any type segments</param>
+        /// <param name="navigationSource">the navigation source for this ExpandItem</param>
+        /// <param name="filterOption">A filter clause for this expand (can be null)</param>
+        /// <param name="searchOption">A search clause for this expand (can be null)</param>
+        /// <exception cref="System.ArgumentNullException">Throws if input pathToNavigationProperty is null.</exception>
+        public ExpandedCountSelectItem(ODataExpandPath pathToNavigationProperty, IEdmNavigationSource navigationSource, FilterClause filterOption, SearchClause searchOption)


I think it would be helpful if you could provide an concrete example of a path that this class represents, and how the parts of the path that the different parameters represent.

gathogojr

KenitoInc added 6 commits April 12, 2021 11:21

Add support for dollar count in expand

3765baf

Code cleanup

8f28347

Improve validation in PathSegment Token

48ce526

Deleted unnecessary file

cfa2fb0

Improve comments

88baf1b

Update expand tests

b4c88d3

pull-request-quantifier-deprecated bot added the Small label Apr 13, 2021

KenitoInc changed the title ~~$count improvements~~ Add support for $count in navigation properties Apr 14, 2021

Remove unnecessary tests

ed2516e

KenitoInc requested review from xuzhg, ElizabethOkerio, gathogojr, habbes, marabooy and mikepizzo April 14, 2021 11:11

KenitoInc marked this pull request as ready for review April 14, 2021 11:11

KenitoInc added the Ready for review Use this label if a pull request is ready to be reviewed label Apr 14, 2021

xuzhg approved these changes Apr 14, 2021

View reviewed changes

KenitoInc added 2 commits April 19, 2021 12:00

Update error message

991d56d

Update comment

c62f79f

marabooy approved these changes Apr 20, 2021

View reviewed changes

habbes reviewed Apr 20, 2021

View reviewed changes

habbes approved these changes Apr 20, 2021

View reviewed changes

gathogojr approved these changes Apr 21, 2021

View reviewed changes

KenitoInc merged commit 2486d9f into OData:master Apr 22, 2021

gathogojr mentioned this pull request Apr 10, 2022

Odata count inside Expand [.Net Core] OData/WebApi#1564

Closed

xuzhg mentioned this pull request Aug 2, 2023

Enable $count in navigation properties OData/AspNetCoreOData#1008

Open

xuzhg mentioned this pull request Dec 19, 2024

Enable serialize/deserialize navigation property with count without content #3154

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for $count in navigation properties #2051

Add support for $count in navigation properties #2051

KenitoInc commented Apr 13, 2021 •

edited

Loading

xuzhg Apr 14, 2021

xuzhg Apr 14, 2021

KenitoInc Apr 19, 2021

xuzhg Apr 14, 2021

KenitoInc Apr 19, 2021

xuzhg commented Apr 14, 2021

gathogojr commented Apr 19, 2021 •

edited

Loading

KenitoInc commented Apr 19, 2021

gathogojr commented Apr 19, 2021

pull-request-quantifier-deprecated bot commented Apr 19, 2021

What can I do to optimize my changes

How to interpret the change counts in git diff output

habbes Apr 20, 2021

KenitoInc Apr 22, 2021

habbes Apr 20, 2021

gathogojr left a comment

Add support for $count in navigation properties #2051

Add support for $count in navigation properties #2051

Conversation

KenitoInc commented Apr 13, 2021 • edited Loading

Issues

Description

Checklist (Uncheck if it is not completed)

Additional work necessary

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xuzhg commented Apr 14, 2021

gathogojr commented Apr 19, 2021 • edited Loading

KenitoInc commented Apr 19, 2021

gathogojr commented Apr 19, 2021

pull-request-quantifier-deprecated bot commented Apr 19, 2021

What can I do to optimize my changes

How to interpret the change counts in git diff output

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gathogojr left a comment

Choose a reason for hiding this comment

KenitoInc commented Apr 13, 2021 •

edited

Loading

gathogojr commented Apr 19, 2021 •

edited

Loading