Extend abi to support peeking of intermediate witnesses

[joss-aztec]

Problem

Currently it is not possible to conveniently autogenerate a value from a circuit + inputs without that value being designated as public. E.g. In the following circuit the developer would have implement their mutation logic both inside and outside of the circuit in order to infer what the secret next_state should be.

// main.nr
fn main (
    prev_state: State,
    prev_state_hash: pub Field,
    mutation: Mutation,
) -> pub Field {
  constrain prev_state_hash == hash_state(prev_state);
  let next_state = mutate(prev_state, mutation);
  hash_state(next_state)
}

Solution

Provide a way to designate certain intermediate witnesses as part of the abi. e.g.

// main.nr
  // ...
  let next_state = mutate(prev_state, mutation);
  peek next_state;
  // ...

Such that the developer tooling can to do something to the effect of:

// client.ts
const nextState = abi.decodePeekable(solvedWitnesses, 'next_state');

Alternatives considered

Support private returns from main. (But apparently there's a security concern with private returns -- I'm not familiar with the reasoning.)

[joss-aztec]

Events in ethereum are roughly analogous to this abi concept. It might make sense to mimic that instead for the sake of familiarity.

[joss-aztec]

In fact, wrapping the intermediate in some structure akin to an event would probably be necessary for recognising when an intermediate is unused (e.g. the intermediate occurs in a zeroed-out branch of an if statement).

[kevaundray]

What would be the usecase for having the output of the circuit being private instead of making it public?

Is it not possible to make the output of your first circuit public and then pass that in as private input to your second recursive circuit? ie promoting a public input to a private input

---

Can you clarify how events are akin to this ABI? The ABI is solely used to tell users how they should call the Noir program's entry point and what will be returned

---

We probably don't want to recognise when a zeroed out branch is used or unused; on the constraint system layer both of branches are compiled, the place where this would useful is on the witness generation layer but this is orthogonal to this part of the stack because that optimisation can be done during witness generation

[joss-aztec]

Is it not possible to make the output of your first circuit public and then pass that in as private input to your second recursive circuit? ie promoting a public input to a private input

Fair point. I'll remove the additional context. I added the note about recursive proofs to win some extra points, but it backfired and distracted from the original intention. The use case that the Problem+Solution describes is to allow a user to write their app logic just once. Without a means to expose some private autogenerated value from a circuit, the user is forced to manually compute that value outside of the circuit instead, i.e. they have to write their app logic twice.

Can you clarify how events are akin to this ABI? The ABI is solely used to tell users how they should call the Noir program's entry point and what will be returned

By "this abi concept" I was referring to the "peeking" of intermediate values. This concept extends the interface into the program, hence why I considered it an extension to the ABI. If you didn't flag one of these "peeked" intermediate witnesses as relevant to the ABI you wouldn't know how to reference it or know to protect it from being optimised away.

I say this concept ("peeking") might be considered similar to ethereum events, because the purpose of an ethereum event is to simply emit some useful information during the execution. Emitting ethereum events doesn't affect the outcome of execution (except gas costs). On second thoughts, the word 'event' is a poor choice, because in most other environments events usually cause side effects within the same execution.

We probably don't want to recognise when a zeroed out branch is used or unused; on the constraint system layer both of branches are compiled, the place where this would useful is on the witness generation layer but this is orthogonal to this part of the stack because that optimisation can be done during witness generation

Possibly I misused the terms zeroed-out & branch (I'm not thinking of compile time zeroing, I'm instead talking about witness selection). As a developer I wouldn't want to misleadingly inspect some intermediate value that was "unused" for a particular execution -- hence why it might be intuitive to describe a "peeked" intermediate instead as an emitted event (either it was emitted or it wasn't).

[kevaundray]

Pasting conversation from slack, which clarifies this idea:

Take the following noir program:

fn main(x : Field) -> pub Field {
   let y = std::do_something(x);
   let z = std::do_something_else(y);
   z
} 

Now lets say I want to know the value of y , its intermediate private state, so there is no easy way to retrieve this. None if you are just a Noir developer, unless you set it as public.

The idea is to add event syntax which will create a list of Event logs for the Prover which will allow them to fetch intermediate private state. The alternative is to re-implement this std::do_something method outside of the circuit, but this will cause us to have duplicated code.

With events, it now looks something like:

fn main(x : Field) -> pub Field {
   let y = std::do_something(x);
   event("Deposit", y);
   let z = std::do_something_else(y);
   z
} 

The first parameter is the event name and the second parameter is the value you want to print out. The latter will need further thought on how we print out composite objects like structs.

@sean Cheetham informed me that events in solidity are implemented using logs, so this function may just be sugar on top of our log functionality.

[kevaundray]

There are still conversations to bike-shed the name -- one idea from Joe is "trace"

[vezenovm]

Organizing some thoughts from slack:

We can expand our new Log directive in the acvm to have this functionality, as it is essentially already being done. The Log directive currently simply prints user-specified intermediate witness values to console, but if the user would like to access them through file or some other environment (in memory w/ node.js) they should be able to do so. Instead of immediately printing the witness values we can instead collect them as a vector. solve_directives and solve then has the possibility to return a collection of intermediate witness values that a user has specified they wish to access in their program (either through a std::println,std::trace,etc.). For nargo these returned intermediate witnesses will most likely be written to a log file, while the npm packages will simply return the vector for a dev to use as they see fit.

[kevaundray]

Closing as we can now use logs/Oracles to get the same behavior