Add support for new filtering ops

[czaloom]

Feature

This PR implements a new filtering schema for Valor and connects the refactored query generator #597 to the API.

This includes support for logical operations (AND, OR, NOT) and the ability to apply filters per table.

API Endpoints

Filters no longer use query parameters in api calls.

GET - gets all instances of a table 
/datasets
/models
/data
/labels

POST - post a filter, return filtered instances of a table
/datasets/filter
/models/filter
/data/filter
/labels/filter

Examples

# No swimmer, small boats
no_swimmer_small_boats = client.get_datums(
    Filter(
        datums=And(
            Label.key == "class",
            Label.value != "swimmer",
        ),
        annotations=And(
            Label.key == "class",
            Label.value == "boat",
            Annotation.bounding_box.area < 50,
        ),
    )
)
assert len(no_swimmer_small_boats) == 1
assert no_swimmer_small_boats[0].uid == "uid1"

# No swimmer, large boats
no_swimmer_large_boats = client.get_datums(
    Filter(
        datums=And(
            Label.key == "class",
            Label.value != "swimmer",
        ),
        annotations=And(
            Label.key == "class",
            Label.value == "boat",
            Annotation.bounding_box.area > 50,
        ),
    )
)
assert len(no_swimmer_large_boats) == 1
assert no_swimmer_large_boats[0].uid == "uid2"

# Swimmer with small boat
swimmer_with_small_boats = client.get_datums(
    Filter(
        datums=And(
            Label.key == "class",
            Label.value == "swimmer",
        ),
        annotations=And(
            Label.key == "class",
            Label.value == "boat",
            Annotation.bounding_box.area < 50,
        ),
    )
)
assert len(swimmer_with_small_boats) == 1
assert swimmer_with_small_boats[0].uid == "uid3"

# Swimmer with large boat
swimmers_and_boats = client.get_datums(
    Filter(
        datums=And(
            Label.key == "class",
            Label.value == "swimmer",
        ),
        annotations=And(
            Label.key == "class",
            Label.value == "boat",
            Annotation.bounding_box.area > 50,
        ),
    )
)
assert len(swimmers_and_boats) == 1
assert swimmers_and_boats[0].uid == "uid4"

[ntlind]

this is much longer and harder to read than the old version. is it necessary to write it this way? I'm sure the answer is "yes", and I know it doesn't matter much since this will all live in the backend, but...

[czaloom]

I couldnt think of a more intuitive name. I could enumerate it into And, Or and Not like the client?

[ntlind]

I just meant that we're going from one class Filter with a few easy arguments to 5(?) classes with more complicated arguments and doubling the lines of code in the process. I don't have any real suggestions here as I'm not deep enough in the code to understand if we need to be this specific with all of these different classes

[czaloom]

5(?) classes

Which ones are you referring to?

Filters are constructed using 2 classes.

• Condition (e.g. ==, !=, >, etc)
• LogicalFunctions (e.g. AND, OR, NOT).

[ntlind]

what's causing there to be a type issue on this line with these changes?

[czaloom]

Removed type ignore. There is red highlighting but it passes pre-commit.

Some of these do fail pre-commit. This is an issue documented in #604.

[ntlind]

I thought this was to be the last filtering PR, so I was expecting issues like this one to be fixed. I'm ok with pushing this into another PR if you think that's best.

[czaloom]

It is the last one to work on filter structure. These type ignores are related to pyright issues that only affect json generation in the client.

too soon