Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Toplevel reimplementation with leaf answer callbacks #2527

Merged
merged 7 commits into from
Oct 13, 2024

Conversation

bakaq
Copy link
Contributor

@bakaq bakaq commented Sep 3, 2024

I went and reimplemented a good part of the toplevel on top of a predicate I called run_query_goal/4. It passes all the current tests, and works the same as the old implementation in my manual testing. The idea is to eventually make this same predicate drive the library interface (when I figure out how to call arbitrary Prolog from the Rust side1), other (pseudo) toplevels, and maybe even some other kinds of things, it's very general. It takes 4 arguments: the query term, a list of variable names (in the form given by the variable_names(-VarNames) option of read_term/3), a callback goal expecting 3 arguments and a list of options. This predicate then runs the query and calls the callback with the leaf answer and additional information, and the callback then signals if the query should continue or stop. The possibilities for the first argument (the leaf answer) are:

  • final(false)
  • final(exception(Exception))
  • final(true)
  • final(leaf_answer(Bindings, ResidualGoals, VarNames))
  • pending(true)
  • pending(leaf_answer(Bindings, ResidualGoals, VarNames))

A principal functor of final/1 means that this is the last leaf answer in the complete answer, and a principal functor of pending/1 means that there are more leaf answers. Implementing this made me realize that this pending/final "attribute" of leaf answers is actually very important (it's not quite an iterator, in the Rust sense) and gives me ideas of how to improve the library interface further2. The second argument to the callback is a list with additional information that we can use to implement extensions to this in the future (like reporting inference counts). The third argument needs to be instantiated by the callback to either continue or stop to signal if the query should stop.

Notice that this can fully represent residual goals (!), with projections (!!) and also reifies exceptions (!!!). The only improvement on this I can think of is to map the query variable names onto the exception like in the leaf_answer/3 case, but it seems that variables lose their identity when going through a throw-catch, so I don't know if that is even possible.

I want to further break the toplevel into configurable/reusable pieces in the future, because that will allow a lot of really cool stuff, especially in the Scryer Playground!

Ok, now I gotta sleep, I've been coding this non-stop for 14 hours straight.

Footnotes

  1. I have some ideas! If it works we could move even more of the engine into Prolog very easily.

  2. It also was the first time I've felt a use for a soft cut, which was interesting to stumble on, and the first time I've actually used subsumes_term/2, very nice (this was actually not a place where subsumes_term/2 helped much).

src/toplevel.pl Show resolved Hide resolved
src/toplevel.pl Outdated Show resolved Hide resolved
src/toplevel.pl Outdated Show resolved Hide resolved
@triska
Copy link
Contributor

triska commented Sep 3, 2024

variables lose their identity when going through a throw-catch

You can throw, as exception, a term of the form T-Vs, where Vs are the variables in T. This will yield a copy of T and Vs. If you have the original variables Vs0 somewhere, then unifying Vs with Vs0 will reinstate the original variables in T.

src/toplevel.pl Outdated Show resolved Hide resolved
@triska
Copy link
Contributor

triska commented Sep 3, 2024

Wow, thank you a lot for working on this! I added a few comments directly in the commit, and I hope you find them useful!

@UWN
Copy link

UWN commented Sep 4, 2024

What about a seamless integration of inference limits and/or timeouts?

@jjtolton
Copy link

jjtolton commented Sep 4, 2024

This is amazing work! Once this is done, would you have any interest in doing an informal mini-talk or discussion about your experience reimplementing the toplevel? It's a little beyond my current aptitude but I'm really fascinated by it because no other language I've worked with manipulates its own internals like this. It's like a puppet working its own ($$\tiny{compact null terminated UTF-8}$$) strings! I'd be really interest in your thoughts on it.

@bakaq
Copy link
Contributor Author

bakaq commented Sep 4, 2024

What about a seamless integration of inference limits and/or timeouts?

I assume you mean this as options to run_query_term/3 and not necessarily the toplevel. I thought about that, and I think I should at least leave space for that to be implemented in the future. If I find out how to do it I may even implement this now.

Once this is done, would you have any interest in doing an informal mini-talk or discussion about your experience reimplementing the toplevel?

Definitely. I already worked a whole bunch on low level Prolog 1 with the intent to make high level monotonic Prolog easier and more general to use. It's fascinating and I would really like to share some of the insights I learned along the way.

Footnotes

  1. Low level Prolog (implementing constraint solvers and systems to work with side effects) is a good example on where you either can't or don't want monotonic code, though bare !/0 is still very situational even at this level and it's better to use some more structured non-monotonic abstractions ((->)/2, (\+)/1, once/1, nonvar/1, etc...).

@bakaq
Copy link
Contributor Author

bakaq commented Sep 6, 2024

I renamed some things, removed the old toplevel, expanded the interface with an Options argument and an extra argument to the callback to allow future extensions and documented run_query/3 and run_query_goal/4. I'm thinking of maybe pulling exception(_) out of final(_), because that may be easier to use in practice (exception is almost always error handling).

You can throw, as exception, a term of the form T-Vs, where Vs are the variables in T.

This sounds interesting, but what I wanted here is to be able to trace variable identity for any exception1, to have things like this:

?- throw(a(A, B)).
   % Current behavior
   throw(a(_41,_42)).
?- throw(a(A, B)).
   % Nice behavior I wanted to implement
   throw(a(A, B)).

Is there a fundamental reason why variables need to be copied and lose their identity in exceptions, or why you wouldn't want the "nice behavior" here? Related to this, and I already talked about this before, currently there is an inconsistency between printing of exceptions that are errors (in the sense of having principal functor error/2) and exceptions that aren't:

?- throw(a(A, B)).
   throw(a(_41,_42)).
?- throw(error(A, B)).
   % Strips away the throw/1
   error(_41,_42).

I guess this is for readability, but I think this is maybe the last detail that makes the toplevel not completely follow the philosophy of "the complete answer is logically equivalent to the query"2. If readability is the only argument here, I don't think it matters much. Let's just implement basic comment error pretty printing and readability is solved. In fact I think that I may work on that next, doesn't seem that difficult, although it won't be that useful yet because errors still don't have much information. The vision:

?- throw(a(A, B)).
   throw(a(A,B)).
?- throw(error(instantiation_error,example/2).
   % ERROR: instantiation_error
   % - culprit: example/2
   throw(error(instantiation_error,example/2).

Footnotes

  1. And also doing an unification may trigger some constraint propagation and have some crazy side effects which I think we should avoid as much as possible. But if a constraint implementation does that for unifying two variables then I think it's probably the constraint fault.

  2. Although error/2 is actually a predicate, so it still kinda works.

@bakaq bakaq marked this pull request as ready for review September 6, 2024 19:01
@bakaq
Copy link
Contributor Author

bakaq commented Sep 7, 2024

I added a mechanism to stop the query with another argument to the callback. This is needed for when you press enter in the toplevel and also in the Drop implementation of QueryState. Previously I did a hack using throw/1 and catch/3 with a special atom for control flow, which just seemed wrong.

@triska
Copy link
Contributor

triska commented Sep 7, 2024

what I wanted here is to be able to trace variable identity for any exception

You can use term_variables/2 to obtain all variables that occur in the query.

@triska
Copy link
Contributor

triska commented Sep 7, 2024

This is awesome, thank you a lot for all this work!

In the description of the PR, you mention "a use for a soft cut"; which part of your code would benefit from this? Scryer currently does not provide a soft cut; I see in your code you are able to inspect the WAM to deduce interesting information about choice points, maybe this part can be separated as a construct, at least to improve readability of the current implementation.

@mthom
Copy link
Owner

mthom commented Sep 27, 2024

@bakaq I'll try this out soon and tell if I have any notes. thanks for your patience.

@mthom
Copy link
Owner

mthom commented Sep 30, 2024

@bakaq very nice! there's some trailing whitespace in toplevel.pl but it all looks good

src/toplevel.pl Show resolved Hide resolved
@bakaq
Copy link
Contributor Author

bakaq commented Oct 13, 2024

Is there anything else to be done here? I'm currently trying to use this to implement Machine::run_query() in #2582 with support for residual goals and such.

@mthom
Copy link
Owner

mthom commented Oct 13, 2024

no, was just waiting for the correction in reply to @UWN's comment. merging now

@mthom mthom merged commit c39ea48 into mthom:master Oct 13, 2024
13 checks passed
@bakaq bakaq deleted the leaf_answers_callbacks branch October 13, 2024 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants