GH-5030: fix FedXDataset support for queries with FROM clauses #5031
Conversation
FedX supports interpret the Dataset being defined on the query: if it is of type FedXDataset, the federation is reduced to the members defined by FedXDataset#getEndpoints(). This is a nice means to evaluate a query on a subset of the federation and is generally working fine. When the query contains a FROM clause, however, the externally passed Dataset is wrapped into a FallbackDataset (which internally keeps the FedXDataset as primary one. This change now inspects the FallbackDataset and unwraps it to reduce the members. Note that for execution still the "FallbackDataset" is used.
| @@ -113,4 +113,12 @@ private void appendURI(StringBuilder sb, IRI uri) { | |||
| } | |||
| } | |||
|
|
|||
| public Dataset getPrimary() { | |||
There was a problem hiding this comment.
do you think it is OK to make the primary dataset accessible for use in FedX?
| FedX fed = federationContext.getFederation(); | ||
| members = fed.getMembers(); | ||
| } | ||
| List<Endpoint> members = getMembersFromContext(stats); |
There was a problem hiding this comment.
Note: the logic is now moved into a helper method. Potentially allowing derived extensions to do additional pruning of members in the query context
|
|
||
| // if the query uses a FROM clause, the external dataset | ||
| // is wrapped as primary dataset in FallbackDataset | ||
| if (queryDataset instanceof FallbackDataset) { |
There was a problem hiding this comment.
this is actually the only change in logic: we unwrap the primary dataset from the FallbackDataset
|
@hmottestad can I ask for a review to have this bug fix still in 5.0? It is relevant for the use of FedX in our product, where we have means to reduce the federation for a specific query. Also: can you comment on the failed "dash-license" check? Is this something we need to look after? In my change I have not done anything with dependencies. |
|
Dunno why we are seeing that license issue now. We've had some random issues over the past few months were the license tool suddenly starts flagging an existing dependency without any changes from our side. I've added the three packages to harvet on clearlydefined.io so let's hope that that works. |
GitHub issue resolved: #5030
FedX supports interpret the Dataset being defined on the query: if it is of type FedXDataset, the federation is reduced to the members defined by FedXDataset#getEndpoints().
This is a nice means to evaluate a query on a subset of the federation and is generally working fine.
When the query contains a FROM clause, however, the externally passed Dataset is wrapped into a FallbackDataset (which internally keeps the FedXDataset as primary one.
This change now inspects the FallbackDataset and unwraps it to reduce the members. Note that for execution still the "FallbackDataset" is used.
PR Author Checklist (see the contributor guidelines for more details):
mvn process-resourcesto format from the command line)