lib: add evaluation state dump utility type #74
Conversation
efd6
added
enhancement
efd6
force-pushed
the
dumpstate
branch
7 times, most recently
from
3756e64
to
196e057
Compare
efd6
force-pushed
the
dumpstate
branch
3 times, most recently
from
c259e6f
to
1f4b6c2
Compare
example.com is rejecting POST requests.
There was a problem hiding this comment.
The dump output is interesting, and likely quite useful even during development as a pseudo step debugger. For a complex programs, I imagine the size of the EvalState can get large if (?) it is holding copies of the ref.Val at each eval (not sure about this though).
We can proceed with exposing this through mito, and then do some more testing in scenarios involving elastic/integrations.
| prg, err := env.Program(ast) | ||
| var progOpts []cel.ProgramOption | ||
| if details { | ||
| progOpts = []cel.ProgramOption{cel.EvalOptions(cel.OptTrackState)} |
There was a problem hiding this comment.
What are the memory implications of turning this on? Like how big does the EvalState become if say the program is processing a large XML response?
There was a problem hiding this comment.
I would imagine this would be untenably large. The actual size would depend how many allocations were made to perform evaluations. In the worst case this would be the sum of all sizes for each evaluation node. In practice, I would imagine that this is mitigated by sharing between values when there has not been mutation or type conversion that requires a new allocation (e.g. []byte → string or map construction, though in the latter, the only allocation would be the shallow value). I think we need to experiment to get a completish picture.
There was a problem hiding this comment.
mitigated by sharing between values when there has not been mutation
That's what I hoping too, like a copy-on-write paradigm to achieve immutability while being memory efficient.
This is intended to be used for whole-of-program debugging. If it works out here, it may be worth considering having an option in the CEL input to dump the eval state in the case of error. This cannot be on always since it is heavy, retaining the full state of all nodes during the evaluation.
Please take a look.