Eager Execution

[jasmith-hs]

This issue is to serve as the status and main point of info for the Eager Execution project that I'm working on.
Check out #519 for the initial draft PR that I opened for this project. (This PR will stay a draft)

Purpose

To partially render as much of a Jinjava/HubL template as possible to allow for a lightweight Jinjava engine to be used from the cdn worker at serve-time to eliminate as much serve-time dependency on the cos-renderer.

When executing eagerly, if there exist any deferred values in the Jinjava code, then the template will need to be rendered a second time with the values not deferred. The second render should have the same output as if the template was only rendered once, normally, with all values resolved on the context. Only the tags and functions that directly depend on the deferred values will end up getting deferred, and they are marked this way with a new set of objects called the context’s eagerTokens.

Background

TagNodes will become Deferred nodes if the tag.interpretOutput() throws a DeferredValueException. The node, which includes its children, will get put into the context’s deferredNodes. ExpressionNodes will become Deferred nodes if the expression within the tag throws a DeferredValueException when interpreter.resolveELExpression() is called.

Since the node and its children get deferred, it is unknown what kind of operations are therefore deferred. Additionally, there may be logic that could have been executed but got deferred because the parent node was deferred. Without eager execution, it is a requirement that an initial pass that results in deferred nodes, when rendered for a second pass, must be rendered with an engine with the same context.

Eager execution is therefore beneficial for 2 main reasons: The final rendering pass will be faster as there will be less unevaluated tags, functions, filters, and expressions to evaluate. And second, it allows for the final pass to be done by a different jinjava interpreter. (Limited access to functions, filters, etc; limited access to context; sandbox/separation from other services)

PR Backlog

Open main PR's

Merged PR's

(Branch) Execution Mode config option
Chunk Resolver
Introduce Eager Tokens
Tests and EagerTagDecorator skeleton
Run prepareContext method
Use strategy pattern for ExpressionNode
Full EagerTagDecorator Functionality (minus macro functions)
Add eager macro function logic
Eager expression node strategy
Length limiter
EagerIfTag
EagerForTag
EagerSetTag
EagerPrintTag & EagerDoTag
EagerCycleTag
Tests for extends and include tags
EagerImportTag & EagerFromTag

Progress

At the moment, everything in the below list is working and implemented, but I have made a checklist as I plan to introduce all the features in several different PRs (because it's way too much code for a single PR). I will check them off after the functionality and tests are merged in and will try to link the PR next to each item or something similar.

• AutoEscapeTag
• BlockTag Eager execution tests for ExtendsTag and IncludesTag #561
• CallTag
• CycleTag Implement EagerCycleTag #556
• DoTag Implement EagerPrintTag and EagerDoTag #558
• ExtendsTag Eager execution tests for ExtendsTag and IncludesTag #561
• ForTag Implement EagerForTag #554
  • When the tag itself is deferred
  • When there are children depending on vars set in the for tag that are deferred
• FromTag Implement EagerFromTag and EagerImportTag #560
  • Macros
  • Values (currently adds FromTag to deferred nodes)
  • Values As
• IfchangedTag Implement EagerForTag #554
• IfTag Implement EagerIfTag #553
  • The tag itself is deferred
  • When an elseif is deferred
  • Impossible branches are also optimized out
• ImportTag Implement EagerFromTag and EagerImportTag #560
  • Macros
  • Values
• IncludeTag Eager execution tests for ExtendsTag and IncludesTag #561
• MacroTag Add eager deferred macro logic #547
• PrintTag Implement EagerPrintTag and EagerDoTag #558
  • Certain output gets wrapped in raw tags
• RawTag
• SetTag Implement EagerSetTag #557
• UnlessTag Implement EagerIfTag #553
• ExpressionNode (Not a tag but similar) Eager expression node #548
  • Values
  • Macro functions
  • Certain output gets wrapped in raw tags

[jasmith-hs]

cc @Joeoh @boulter @hs-lsong @mattcoley in case you're interested in this project