Feature request: Basic AND/OR support for tags filtering.

[SimonLuckenuik]

Hello,

Question about the roadmap: Are there any plans for fancier tags filtering with basic AND/OR support when accessing memory, in order to scope content?

Example:
A) You have a hierarchial structure: an organization, with departments and teams in each department. From there, you want to scope to which documents each part of the organization has access to (somehow manage some basic visibility rules):
1) The Company security guidelines are stored at the Organization level, accessible to everyone.
2) The IT department internal procedures documents are accessible only to members of the department.
3) The Security Team of the IT department is working on a draft document only accessible inside their team.

In that example, when accessing memory, a user should have access to its hierarchical parent.

With the current Tag approach it would be difficult to accomplish considering that all Tags are filtered as ANDs.
Now, assuming that we support basic hierarchial AND/OR (2-3 levels), we could go a long way in supporting new scenarios.

Searching inside the organization only: Organization=A AND Department=None AND Team=None
Searching inside the organization or department: (Organization=A AND Department=None AND Team=None) OR (Organization=A AND Department=IT AND Team=None)
And so on...

Thanks!

[dluc]

hi @SimonLuckenuik

OR logic is already supported by passing a list of filters. The rules inside each filter are composed with AND, and multiple filters are composed with OR.

Example:

var filter1 = MemoryFilters.ByTag("group", "g1").ByTag("user", "u1")

var filter2 = MemoryFilters.ByTag("group", "g1").ByTag("user", "u2")

var filter3 = MemoryFilters.ByTag("group", "g2").ByTag("user", "u3")

var answer = answer = await memory.AskAsync(
    question,
    filters: new List<MemoryFilter> { filter1, filter2, filter3 });

Is interpreted as: WHERE filter1 OR filter2 OR filter3, ie

WHERE
        (tags contains "group:g1" AND tags contains "user:u1")
     OR (tags contains "group:g1" AND tags contains "user:u2")
     OR (tags contains "group:g2" AND tags contains "user:u3")

[SimonLuckenuik]

Excellent, thanks for the detailed answer and quick response!

Would probably be worth documenting it or reviewing the property naming to reflect that behavior. My point of contact with the Kernel Memory was through the MemoryPlugin, it seems that the "filters" is missing in the MemoryPlugin only "filter" is used, preventing that AND/OR logic.

[dluc]

good point about the plugin, we should update that. In terms of documentation, we currently have this https://microsoft.github.io/kernel-memory/security/filters - I guess we could decouple the topic from "security"