doc: add guide to dev HTNN

examples/dev_your_plugin/cmd/libgolang/main.go

@@ -22,16 +22,18 @@ import (
 	"github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http"
 
 	"mosn.io/htnn/api/pkg/filtermanager"
-	_ "mosn.io/htnn/api/plugins/tests/integration/data_plane" // for utility plugins provided in the test framework
-	_ "mosn.io/htnn/dev_your_plugin/plugins"
-)
+    // For utility plugins provided in the test framework.
+    // don't import this if you want to produce the shared library for product environment.
+	_ "mosn.io/htnn/api/plugins/tests/integration/data_plane"
 
-// If you want to use the built-in plugins, you can import them here:
-// _ "mosn.io/htnn/plugins/plugins"
-//
-// Note that because we only update the module dependency in the release, if you use
-// a non-release version of mosn.io/htnn/plugins module, you may need to manually update the
-// dependency yourself, such as using replace in go.mod
+    // If you want to use the built-in plugins, you can import them here:
+    // _ "mosn.io/htnn/plugins/plugins"
+    //
+    // Note that because we only update the module dependency in the release, if you use
+    // a non-release version of mosn.io/htnn/plugins module, you may need to manually update the
+    // dependency yourself, such as using replace in go.mod
+    _ "mosn.io/htnn/dev_your_plugin/plugins"
+)
 
 func init() {
 	http.RegisterHttpFilterConfigFactoryAndParser("fm", filtermanager.FilterManagerFactory, &filtermanager.FilterManagerConfigParser{})

site/.htmltest.yml

@@ -4,6 +4,7 @@ IgnoreURLs:
   - https://mosn.io/ # fake base url for now
   - http://localhost:1313/ # path used in local dev
   # ignore code generated links from the GitHub component
+  - https://github.com/mosn/htnn/tree/main/site/content
   - https://github.com/mosn/htnn/edit/
   - https://github.com/mosn/htnn/new
   - https://github.com/mosn/htnn/issues/new

site/content/en/docs/developer-guide/get_involved.md

@@ -0,0 +1,125 @@
+---
+title: How to develop HTNN to fit your purpose
+---
+
+HTNN's functional code is mainly located in the following modules:
+
+* api/: This is the most basic module. It is referenced by other modules and provides interfaces needed to develop HTNN plugins.
+* types/: Modules such as the control plane, data plane, and console all depend on this module to provide plugin metadata, CRD definitions, and other internal common data.
+* controller/: Control plane
+* plugins/: Data plane (plugin hub)
+
+For users who want to develop their functionalities, it is generally only necessary to refer to the implementations in `controller/` and `plugins/`.
+
+## Controller
+
+HTNN in the `controller/` module mainly does the following:
+
+* Reconcile HTNN's CRD
+* Provide a Native Plugin framework to allow modifications to the xDS sent to the data plane through plugins.
+* Provide a Service Registry mechanism to integrate the external service discovery systems in the form of registries, converting them into upstream information within the data plane.
+
+Plugin and Registry are closely related to developers. It is advisable to read the code in the `plugins/` and `registries/` directories to see how to develop functionalities using HTNN's interfaces.
+
+Traditionally, there are two ways to alter the behavior of Istio:
+
+1. Translate your resources into EnvoyFilter and write them into k8s to let Istio process the EnvoyFilter.
+2. Become an MCP server yourself. Reconcile your own resources, then deliver the results to Istio through the MCP protocol.
+
+HTNN does not use the methods above but instead modifies Istio to embed its own process into Istio. The specific changes can be referred to in the `patch/istio` directory's patch files. The reason for not using method 1 is that writing stateful EnvoyFilter leads to observability and high availability becoming more challenging and prone to problems. Method 2 is not used because to ensure the consistency of network policy and network routing, the network routing part also needs to be integrated into the MCP server, otherwise, there will be a situation where the policy is still reconciling and the routing has been published online. Providing network routing configurations in MCP server would undoubtedly be a significant workload.
+
+If you develop your own Native Plugin or Service Registry in a third-party repository and want to use it in HTNN, you can use a patch to introduce that repository into Istio. For details, refer to this patch: https://github.com/mosn/htnn/blob/main/patch/istio/1.21/20240410-htnn-go-mod.patch. When importing the repository, make sure to place it after the HTNN official plugin package to avoid being overwritten by plugins with the same name.
+
+If you want to run the HTNN control plane, you can refer to the implementation of `make e2e-prepare-controller-image` in the `e2e/` directory to see how to package the Istio embedded with HTNN into an image.
+
+## Plugins
+
+HTNN places data plane-running Go Plugins and their shared pkg libraries in the `plugins/` module.
+
+If you want to write a Go Plugin in a third-party repository, you can refer to this example: https://github.com/mosn/htnn/tree/main/examples/dev_your_plugin. Go Plugins are compiled into shared libraries and deployed to the data plane. When you want to integrate a Go Plugin into the data plane, this file can serve as a template: https://github.com/mosn/htnn/blob/main/examples/dev_your_plugin/cmd/libgolang/main.go.
+
+It is recommended to read the existing plugins' code before developing your own plugin, especially plugins that are similar to the one you're developing – at the very least, you should look at the [demo](https://github.com/mosn/htnn/tree/main/plugins/plugins/demo) plugin. Plugin code is located in the `plugins/` directory. For more information on plugin development, please refer to the [Plugin Development](./plugin_development) documentation.
+
+HTNN provides a [Plugin Integration Test Framework](./plugin_integration_test_framework) that allows the testing of Go Plugin logic to run with only the data plane running. The `dev_your_plugin` example also demonstrates how to run the integration test framework in a third-party repository.
+
+Each plugin contains two types of objects: `config` and `filter`. Where `config` is responsible for configuration management and `filter` is responsible for executing request-level logic.
+
+### Config
+
+In plugin development, features closely related to configuration should all be placed in `config`.
+
+The granularity of `config` can be per-route, per-gateway, or per-consumer, depending on the scope of the plugin configuration. For example, configuring the Gateway-level `limitCountRedis`:
+
+```yaml
+apiVersion: htnn.mosn.io/v1
+kind: FilterPolicy
+metadata:
+  name: policy
+  namespace: istio-system
+spec:
+  targetRef:
+    group: networking.istio.io
+    kind: Gateway
+    name: default
+  filters:
+    limitCountRedis:
+      config:
+        address: "redis:6379"
+        rules:
+        - count: 1
+          timeWindow: "60s"
+```
+
+When this occurs, a `limitCountRedis.config` object is created and shared among various routes in the Gateway `default`.
+
+The lifecycle of `config` depends on the plugin configuration itself and the object it acts upon. For example, if we configure `limitCountRedis` on route `vs`:
+
+```yaml
+apiVersion: htnn.mosn.io/v1
+kind: FilterPolicy
+metadata:
+  name: policy
+  namespace: istio-system
+spec:
+  targetRef:
+    group: networking.istio.io
+    kind: VirtualService
+    name: vs
+  filters:
+    limitCountRedis:
+      config:
+        address: "redis:6379"
+        rules:
+        - count: 1
+          timeWindow: "60s"
+```
+
+In two cases, the `limitCountRedis.config` object will be recreated:
+1. Any changes to the routing under any Gateway that `vs` belongs to
+2. Any changes into the specification of FilterPolicy that points to routes mentioned in the previous case
+
+This is because new [RouteConfiguration](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route.proto#envoy-v3-api-msg-config-route-v3-routeconfiguration) in the above cases causes the `limitCountRedis.config` object to be re-created.
+
+(TODO: Support incremental update of routes so that only changes to `vs` itself cause the object to be re-created)
+
+As `config` is shared across multiple requests in different Envoy workers, read and write operations to `config` must be locked.
+
+### Filter
+
+In plugin development, features related to each request should be placed in `filter`. Each request creates a `filter` object for each plugin that is executed.
+
+`filter` mainly defines the following methods:
+
+1. DecodeHeaders
+2. DecodeData
+3. EncodeHeaders
+4. EncodeData
+5. OnLog
+
+Normally, the above methods are executed from top to bottom. However, there are exceptions:
+
+1. If there is no body, the corresponding DecodeData and EncodeData methods will not be executed.
+2. Since the OnLog operation is triggered by the client interrupting the request, OnLog may execute concurrently with other methods if the client interrupts prematurely.
+3. In requests like bidirectional streams, it is possible to handle the request body and upstream response at the same time, so DecodeData may execute concurrently with EncodeHeaders or EncodeData.
+
+So, when reading and writing `filter`, there is a risk of concurrent access and lock consideration is necessary.

site/content/en/docs/developer-guide/plugin_development.md

@@ -13,7 +13,7 @@ Assume you are at the root of this project.
 3. Refer to plugins of the same type and decide on the type and order of your plugin.
 4. Add your plugin's package into `./types/plugins/plugins.go`.
 5. Create a directory under `./plugins/plugins/`, with the same name created in step one. Finish the plugin. Don't forget to write tests. If your plugin is simple, you can write integration test only. You can take `./plugins/plugins/demo` as an example. The doc of the API used in the plugin is in their comments.
-6. Add the doc of the plugin in the `site/content/$your_language/docs/reference/plugins/$your_plugin.md`. You can choose to write doc under Simpilified Chinese or English, depends on which is your prime language. We have [tool](https://github.com/mosn/htnn/tree/main/site#cmdtranslator) to translate it to other languages.
+6. Add the doc of the plugin in the `site/content/$your_language/docs/reference/plugins/$your_plugin.md`. You can choose to write doc under Simplified Chinese or English, depending on which is your prime language. We have [tool](https://github.com/mosn/htnn/tree/main/site#cmdtranslator) to translate it to other languages.
 7. Add your plugin's package into `./plugins/plugins.go`. Go to `./plugins`, then run `make build-so`. Now the plugin is compiled into `libgolang.so` under the current directory.
 8. Add integration test in the `./plugins/tests/integration/`. For how to run the integration test, please read [Plugin Integration Test Framework](../plugin_integration_test_framework).
 
@@ -40,7 +40,11 @@ For plugins which have the same operation, they are sorted by alphabetical order
 
 Here are the order group (sorted from first to last):
 
-* `Outer`: First position. It's reserved for Native plugins.
+The first three order groups are reserved for Native plugins.
+
+* `Listener`: plugins relative to [Envoy listener filters](https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/listener_filters/listener_filters).
+* `Network`: plugins relative to [Envoy network filters](https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/network_filters).
+* `Outer`: First position for plugins running in HTTP.
 
 Now goes the Go plugins:
 

site/content/zh-hans/docs/developer-guide/get_involved.md

@@ -0,0 +1,125 @@
+---
+title: 如何二次开发 HTNN
+---
+
+HTNN 功能代码主要位于下面的模块：
+
+* api/：最基础的模块。它被其他模块引用，并对外提供开发 HTNN 插件所需的接口。
+* types/：控制面、数据面和 console 都会依赖该模块提供插件元数据、CRD 定义等内部通用的数据。
+* controller/：控制面
+* plugins/：数据面（plugin hub）
+
+对于二次开发的用户，一般都只需参考 `controller/` 和 `plugins/` 里面的实现。
+
+## controller
+
+HTNN 在 `controller/` 模块主要做了下面的事：
+
+* 调和 HTNN 的 CRD
+* 提供 Native Plugin 框架，允许以插件的方式修改发送给数据面的 xDS
+* 提供 Service Registry 机制，允许以 registry 的方式对接外部服务发现系统，转换成数据面内的上游信息
+
+其中和开发者紧密相关的是 Plugin 和 Registry。建议阅读 `plugins/` 和 `registries/` 两个目录下的代码，看看如何使用 HTNN 提供的接口开发功能。
+
+传统上想要修改 istio 的行为有两种方法，
+
+1. 将自己的资源翻译成 EnvoyFilter 写入到 k8s 里，让 istio 处理 EnvoyFilter。
+2. 让自己成为 MCP server。调和自己的资源，再把结果通过 MCP 协议给 istio。
+
+HTNN 没有用上述方法，而是通过修改 istio，把自己的流程嵌入到 istio 当中。具体的改动可参考 `patch/istio` 目录下的 patch 文件。之所以不用方法 1，是因为写入带状态的 EnvoyFilter 会导致可观测性和高可用变得更加困难，容易出问题。不用方法 2 的原因在于要想保证策略和路由的一致性，需要将路由也接入到 MCP server 中，否则会出现策略还在调和过程中，路由已经发布上线的情况。将路由也一并在 MCP server 里处理，无疑会是很大的工作量。
+
+如果你在第三方仓库下开发了自己的 Native Plugin 或 Service Registry，想要在 HTNN 中使用，可以通过 patch 的方式将该仓库引入到 istio 当中来。具体参考 https://github.com/mosn/htnn/blob/main/patch/istio/1.21/20240410-htnn-go-mod.patch 这个 patch。注意 import 该仓库时需要把它放到 HTNN 官方插件 package 的后面，这样才能避免被同名的插件覆盖掉。
+
+如果要想运行 HTNN 的控制面，可以参考 `e2e/` 目录下 `make e2e-prepare-controller-image` 的实现，看看如何将嵌入了 HTNN 的 istio 打包成镜像。
+
+## plugins
+
+HTNN 在 `plugins/` 模块下放置在数据面上运行的 Go Plugin 以及它们共享的 pkg 库。
+
+如果你想要在第三方仓库编写 Go Plugin，可以参考 https://github.com/mosn/htnn/tree/main/examples/dev_your_plugin 这个范例。Go Plugin 在数据面上是编译成 shared library 来部署到。当你想要集成 Go Plugin 到数据面时，https://github.com/mosn/htnn/blob/main/examples/dev_your_plugin/cmd/libgolang/main.go 这个文件可以作为模版。
+
+在开发自己的插件之前建议读一下现有插件的代码，尤其是和你开发的功能相似的插件，至少应该看看 [demo](https://github.com/mosn/htnn/tree/main/plugins/plugins/demo) 插件。插件代码位于 `plugins/` 目录下面。关于插件开发的更多信息，请参考[插件开发](./plugin_development)文档。
+
+HTNN 提供了一个[插件集成测试框架](./plugin_integration_test_framework)，允许在只运行数据面的情况下测试 Go Plugin 的逻辑。在 `dev_your_plugin` 这个范例里也展示了如何在第三方仓库中运行集成测试框架。
+
+每个插件都包含两类对象：`config` 和 `filter`。其中 `config` 负责配置管理，`filter` 负责执行请求级别的逻辑。
+
+### config
+
+在插件开发时，和配置紧密相关的功能都应当放到 `config` 里。
+
+`config` 的粒度可以是 per-route 的，也可以是 per-gateway 或者 per-consumer，取决于插件配置的范围。比如配置 Gateway 级别的 `limitCountRedis`:
+
+```yaml
+apiVersion: htnn.mosn.io/v1
+kind: FilterPolicy
+metadata:
+  name: policy
+  namespace: istio-system
+spec:
+  targetRef:
+    group: networking.istio.io
+    kind: Gateway
+    name: default
+  filters:
+    limitCountRedis:
+      config:
+        address: "redis:6379"
+        rules:
+        - count: 1
+          timeWindow: "60s"
+```
+
+这时候会创建一个 `limitCountRedis.config` 对象，且这个对象在整个 Gateway `default` 下各个路由里是共享的。
+
+`config` 的生命周期取决于插件配置本身和它所作用的对象。举个例子，如果我们在路由 `vs` 上配置了 `limitCountRedis`:
+
+```yaml
+apiVersion: htnn.mosn.io/v1
+kind: FilterPolicy
+metadata:
+  name: policy
+  namespace: istio-system
+spec:
+  targetRef:
+    group: networking.istio.io
+    kind: VirtualService
+    name: vs
+  filters:
+    limitCountRedis:
+      config:
+        address: "redis:6379"
+        rules:
+        - count: 1
+          timeWindow: "60s"
+```
+
+在两种情况下 `limitCountRedis.config` 对象会被重新创建：
+1. 任何 `vs` 所在的 Gateway 下的路由发生变化
+2. 任何指向上一种情况里面的路由的 FilterPolicy 的 spec 发生变化
+
+这是因为上述情况下会创建新的 [RouteConfiguration](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route.proto#envoy-v3-api-msg-config-route-v3-routeconfiguration)，导致 `limitCountRedis.config` 对象被重建。
+
+（TODO：支持增量更新路由，这样只有 `vs` 自己的变化才会引起对象被重建）
+
+由于 `config` 在不同的 Envoy worker 线程的多个请求间是共享的，所以对 `config` 的读写操作必须加锁。
+
+### filter
+
+在插件开发时，和每请求相关的功能都应当放到 `filter` 里。每个请求都会给每个执行的插件创建一个 `filter` 对象。
+
+`filter` 里主要定义下面的方法：
+
+1. DecodeHeaders
+2. DecodeData
+3. EncodeHeaders
+4. EncodeData
+5. OnLog
+
+正常情况下，会从上到下执行上述方法。但存在以下特例：
+
+1. 没有 body 则不会执行对应的 DecodeData 和 EncodeData 方法。
+2. 由于客户端中断请求时会触发 OnLog 操作，所以当客户端提前中断请求时，OnLog 方法可能和其他方法同时执行。
+3. 在 bidirectional stream 之类的请求里，有可能出现同时处理请求体和上游响应的情况，所以 DecodeData 可能和 EncodeHeaders 或 EncodeData 同时执行。
+
+所以当读写 `filter` 时可能遇到并发访问的风险时需要考虑加锁。

site/content/zh-hans/docs/developer-guide/plugin_development.md

@@ -39,7 +39,11 @@ HTNN 的插件分成两种：Go 插件和 Native 插件。Native 插件在运行
 对于操作相同的插件，它们按字母顺序排序。
 以下是顺序组（从第一个到最后一个排序）：
 
-* `Outer`：首位。它为 Native 插件保留。
+前三个顺序组为 Native 插件保留：
+
+* `Listener`: 和 [Envoy listener filters](https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/listener_filters/listener_filters) 相关的插件。
+* `Network`: 和 [Envoy network filters](https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/network_filters) 相关的插件。
+* `Outer`：在 HTTP 上运行最前端的插件。
 
 现在开始是 Go 插件：