generic proxy support

[wbpcode]

Now, Envoy already supports proxy traffic of many different protocol. But if we want to support a new protocol, we alway need to implement a complete L4 filter and all related L7 filters. But in fact, many protocols have similarities. So, may be we can try to implement a generic proxy and then, by extending different encoders and decoders to extend different protocol support. And all these protocol can reuse the L7 filters and route of the generic proxy.

Perhaps, the generic proxy cannot achieve the best performance, but it can reduce the complexity and workload of extending protocol support in Envoy. In this way, everyone can simply add their own private protocol support to Envoy and the scope of use of Envoy should be further expanded.

Design docs: https://docs.google.com/document/d/1y_ALyjGp_H1_TRrrRxZxnqENISqeZI6eLn71FAFsZPk/edit?usp=sharing

meta protocol API: #18823
meta protocol core framework: #18944

other related issue:
Generic Access Log: #13085
Generic Routing & Rds: #9656

[wbpcode]

Ref MOSN, brpc and aeraki metadata protocol etc.

[rojkov]

It would be interesting to try making these binary encoders/decoders optionally asynchronous, so that they could be accelerated in HW (e.g. FPGA).

[wbpcode]

It would be interesting to try making these binary encoders/decoders optionally asynchronous, so that they could be accelerated in HW (e.g. FPGA).

Sounds great. Although my current design and implementation do not take this into consideration, I can give it a try. 🤔

[htuch]

@wbpcode @zhaohuabing the basic idea here make a lot of sense. I'd like to make sure we have maintainer/community buy-in and support for the proposed design before we go too far down the review path. Are you planning on putting this in contrib/ or in the stable extension tree?

[zhaohuabing]

@htuch The design and implementation have already been proven working in Aeraki MetaProtcol Proxy. Dubbo and Thrift have been ported to MetaProtcol with RDS supported, and the [Aeraki] works alongside Istio (https://github.com/aeraki-framework/aeraki) as the control plane. My long-term plan is also pushing the common control plane to Istio, so there will be an end-to-end solution for most layer-7 protocols. @howardjohn how do you think about it? Before that, we can use standalone Aeraki + Istio as the control plane.

I guess I should put it in contrib extension tree and move it to stable after a few releases when the codes and apis become stable.

The overall design includes both data plane and control plane for a generic proxy: https://istio.io/latest/blog/2021/aeraki/

[wbpcode]

@htuch Thanks. I have discussed this with @rojkov and he is interested in it.

He thinks the extension would be more valuable if encoder/decoder interface could be made more HW friendly. So, we are trying to redesign our interface. As the most important part, we are willing to spend more time improving it.

And for this big feature, the contrib/ would be a good start.

[wbpcode]

In fact, as you can see, there are now two different implementations, Aeraki meta and my generic proxy (there seems to be some kind of competition). Although their overall ideas are not different, the details of the implementation has some differences.
I created this issue about a month ago and have been preparing for my PR. In fact, I also considered using Aeraki meta directly, but because I prefer the interface-first design (It's a bit like brpc's multi-protocol support), I decided to recreate an implementation myself.

[zhaohuabing]

@wbpcode I didn't expect that you'd created the Issue and PR. When I came out with this idea and decided to create MetaProtocol Proxy in Aeraki 3 months ago, my intention is to create an end-to-end solution, including both the control plane and data plane. I haven't brought it earlier to the Envoy community because I would like to prove this idea is workable first.
I'll be more than happy to work with you and the community to port MetaProtocol to Enovy. I suggest we start with MetaProtocol proxy because it's already proven withDubbo and Thrift protocols, and working well in the Istio service mesh. Now the APIs follow the KISS principle to be designed as a minimal set, simple but working, we can improve it as more concrete use cases come in.

[wbpcode]

@zhaohuabing In fact, the reason I chose to create a new implementation is because Aeraka mata is based on dubbo proxy. In the process of maintaining dubbo proxy, I have always wanted to refactor it 🤣 . And refactoring is sometimes more difficult than creating new things.

We can discuss the issue of working together in private. Leave the rest to the community. Regardless of the outcome, all of us will have a generic solution.

[zhaohuabing]

@zhaohuabing In fact, the reason I chose to create a new implementation is because Aeraka mata is based on dubbo proxy. In the process of maintaining dubbo proxy, I have always wanted to refactor it 🤣 . And refactoring is sometimes more difficult than creating new things.

Not exactly, for example, dubbo doesn't support RDS but MetaProtocol does.

We can discuss the issue of working together in private. Leave the rest to the community. Regardless of the outcome, all of us will have a generic solution.

I agree, let's discuss it offline.

[rojkov]

Can you give it some comments to help me take this work to the next step . Because I am not sure that I fully understand what you said HW friendly.

Hardly I can say I'm an expert in HW acceleration, but I think it's easier to assume the actor model for accelerators. Think of it as a concurrent external process, because it can be emulated as such. Practically this means that a worker thread should be capable of receiving a stream from a client, sending chunks of raw data to the actor's input queue then it should either switch to receiving a stream from another client or sleep until the actor notifies the worker about availability of transformed data in its output queue. Only in the latter case the worker can hand over execution control to the next phase (to the generic filters that is).

In synchronous systems encoders/decoders consume data and return their status as either Continue or StopIteration. The former means new data has been produced and is ready to be consumed by the next consumer. The latter means full stop for the stream.

In HW-friendly ones a third case is possible when a decoder returns something like Again or Retry. In this case the worker must schedule a callback to be triggered upon availability of decoded data in the accelerator's output queue. The callback must contain the state of the stream, so when called eventually it hands over execution control farther to the generic filters.

Now, after having it written down, I think it's rather an implementation detail which can be added later if needed. Anyway it'd be good to bear in mind.

Honestly I wasn't aware there's demand for support for private binary protocols in Envoy. But apparently there is. I know the HFT people use HW for encoding/decoding protocols like FAST, but not sure they use Envoy. So, I'm ready to sponsor the extension when you agree on the design with @zhaohuabing. Can you write a design doc, btw?

[wbpcode]

@rojkov Thanks for you detailed explanation 🌷 . I will provide a detailed design doc (included the interface) about the feature in this weekend. Thanks again.

[wbpcode]

@zhaohuabing 🌹 Can you help me to iterate this design document ? I know the Aeraki mata implementation very well, so it is clear to me that Aeraki meta needs to do something similar if it is to be developed further, such as:

• Abstracting the request/response stream.
• Designing the decoder/encoder interface.
• Designing the upstream request management mechanism.

How about we deprecate the old constraints and work together on this complete fresh extension?

We can actually see this part in my current codes, but documenting it should make it better for more people to review the solution.

[mattklein123]

Thanks for the discussion here. I'm very supportive of moving this forward, potentially even in core versus contrib, if we have a solid set of owners that are invested in pushing this forward and maintaining it.

I do think that we should merge the work of @wbpcode and @zhaohuabing into a single design, and I also agree that the next step should be a design document to present to the community. At that point I think we can decide on contrib vs. core, etc. Sound good? Very excited to see this move forward.

[wbpcode]

Hi, Since I'm not sure how specific you want to see the content. So I've directly translated the simple design document I used to guide my development before. I think this weekend, I can have more time to continue to improve this document.
If you have any questions, please feel free to ask me directly to make changes.

Here is the link: https://docs.google.com/document/d/1y_ALyjGp_H1_TRrrRxZxnqENISqeZI6eLn71FAFsZPk/edit?usp=sharing

And, this document is completely open for editing, not just open for comments. So @zhaohuabing (or even anyone who hopes to complete this extension together, I hope that this extension will eventually be designed and implemented in a very outstanding way), if there is anything you think needs to be changed, you can just edit it or add new content to it.

cc @rojkov

[zhaohuabing]

https://docs.google.com/document/d/1y_ALyjGp_H1_TRrrRxZxnqENISqeZI6eLn71FAFsZPk/edit?usp=sharing

Hi @wbpcode I'd like to include the big picture and end-2-end solution including both the data plane and control plane in the design, which could be found here in this blog: https://istio.io/latest/blog/2021/aeraki/. For the Interface, I suggest starting with Aeraki MetaProtocol, which is already proven to work well with Dubbo, Thrift and Istio, and we can iterate this design and implementation as more sophisticated use cases come in.

[wbpcode]

Complete agree with the e2e solution. And we can discuss second part offline.

[zhaohuabing]

Thanks for the discussion here. I'm very supportive of moving this forward, potentially even in core versus contrib, if we have a solid set of owners that are invested in pushing this forward and maintaining it.

I do think that we should merge the work of @wbpcode and @zhaohuabing into a single design, and I also agree that the next step should be a design document to present to the community. At that point I think we can decide on contrib vs. core, etc. Sound good? Very excited to see this move forward.

Hi @mattklein123 Thanks for your support. Now I and @wbpcode are working together to merge the design and implementation. As you suggest, we plan to push it to core.

[wbpcode]

And some short conclusions sync to community:

• New extension, new implementation design. Check this doc.
• Compatible with the existing Aeraki meta proto API as much as possible. (This proto API is indeed general enough)
• Try to move some important functions (such as RDS) of Aeraki meta to new proxy. (Based on the opinions of the community)

[zhaohuabing]

Hi guys, per our discussion here, #18823 has been created to represent the merged effort.