Bring back --enable-exec for declarative function pipelines.

[aw185176]

Describe your problem

We use Kpt internally, where we have nearly 20 functions. We produce all of the function binaries that we use and we put them into our pipeline images so that Kpt can use them directly.

A typical invocation might look like kpt fn run --enable-exec $DIR/ for us.

We explicitly do not want to use the container runtime because it introduces lots of overhead we don't have to deal with, as well as the performance of the pipeline falling off a cliff relative to the exec runtime.

The decision to neuter this functionality in Kpt v1 was a mistake that ignores enterprise users and favors toy demos / public pipelines.

[droot]

We explicitly do not want to use the container runtime because it introduces lots of overhead we don't have to deal with, as well as the performance of the pipeline falling off a cliff relative to the exec runtime.

Can you share more details about the overheads that container runtime introduces ?

Regarding performance, yes, there is scope for improvements here and we will be addressing it soon.

An unrelated question: What tooling/SDK did you (or your team) use to develop the functions ?

[aw185176]

Can you share more details about the overheads that container runtime introduces ?

You have to pull each container and you need a container run time.

Would you rather have one container with the Docker daemon mounted that then pulls and executes 20 containers, or would you rather have one container with 20 binaries packaged in it that you can run directly? Try a simple example on your own terminal. A pipeline with 10 functions ran via exec can finish in seconds, 10 containers that aren't already pulled takes much longer.

Regarding performance, yes, there is scope for improvements here and we will be addressing it soon.

Performance is fine for exec runtime.

An unrelated question: What tooling/SDK did you (or your team) use to develop the functions ?

We used kyaml/framework directly. Easiest way to get updates the fastest and make changes that we want. The extra layer that Kpt adds on top of it seems mostly unnecessary and adds a layer of indirection

[bgrant0607]

See also #2469 and #2567

[droot]

@aw185176 we added exec support in fn render in beta.11, so closing this issue.